2024-04-25T19:05:43.408 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T19:05:43.408 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/iozone}} --archive /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673301 --owner scheduled_pdonnell@teuthology --verbose --name pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673301/orig.config.yaml 2024-04-25T19:05:43.455 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T19:05:43.551 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673301 branch: wip-pdonnell-testing-20240425.015853-debug description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/iozone}} email: pdonnell@redhat.com first_in_suite: false job_id: '7673301' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240425.015853-debug ceph: cephfs: max_mds: 5 session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: true mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: true mds_sleep_rank_change: 5000000.0 mon client directed command retry: 5 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds osd ms inject socket failures: 2500 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon client directed command retry: 5 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds ms inject socket failures: 2500 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - overall HEALTH_ - \(MON_DOWN\) sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240425.015853-debug sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 1880 sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 sleep_before_teardown: 0 subset: 178/512 suite: fs suite_branch: wip-pdonnell-testing-20240425.015853-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi076.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAxDBH2a+eJqO9WYzP6gBhYbaBMx+CsbA0x+/qDLMUG+r0jSg1bwhDhzQG4L6CyVWNyaLtPKAOr5DrbnmDUSNqM= smithi105.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBGv52NDt/WUrxy6zTqWRWDJEYvTHKMwbILe6dTAmaf/dTjfiFMYcSCLUEts0v1B1vhWyV8lNbTxvW/gL6lcZs0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - mon_thrash: check_mds_failover: true revive_delay: 20 thrash_delay: 10 - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_10:00:23 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T19:05:43.551 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa; will attempt to use it 2024-04-25T19:05:43.552 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks 2024-04-25T19:05:43.552 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T19:05:43.553 INFO:teuthology.task.internal:Checking packages... 2024-04-25T19:05:43.576 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '994e2f20102a20fa9a27966c14b625cc01884df8' 2024-04-25T19:05:43.577 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T19:05:43.577 INFO:teuthology.packaging:ref: None 2024-04-25T19:05:43.577 INFO:teuthology.packaging:tag: None 2024-04-25T19:05:43.577 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240425.015853-debug 2024-04-25T19:05:43.577 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T19:05:43.577 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240425.015853-debug 2024-04-25T19:05:43.797 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3291.g994e2f20 2024-04-25T19:05:43.799 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T19:05:43.806 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T19:05:43.806 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T19:05:43.824 INFO:teuthology.task.internal:Saving configuration 2024-04-25T19:05:43.843 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T19:05:43.849 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T19:05:43.873 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi076.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673301', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 19:00:19.952252', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAxDBH2a+eJqO9WYzP6gBhYbaBMx+CsbA0x+/qDLMUG+r0jSg1bwhDhzQG4L6CyVWNyaLtPKAOr5DrbnmDUSNqM='} 2024-04-25T19:05:43.898 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673301', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 19:00:19.950715', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBGv52NDt/WUrxy6zTqWRWDJEYvTHKMwbILe6dTAmaf/dTjfiFMYcSCLUEts0v1B1vhWyV8lNbTxvW/gL6lcZs0='} 2024-04-25T19:05:43.898 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T19:05:43.904 INFO:teuthology.task.internal:roles: ubuntu@smithi076.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-04-25T19:05:43.904 INFO:teuthology.task.internal:roles: ubuntu@smithi105.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-04-25T19:05:43.904 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T19:05:44.032 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f853cacef70>, signals=[15]) 2024-04-25T19:05:44.032 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T19:05:44.037 INFO:teuthology.task.internal:Opening connections... 2024-04-25T19:05:44.038 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi076.front.sepia.ceph.com 2024-04-25T19:05:44.039 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T19:05:44.115 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi105.front.sepia.ceph.com 2024-04-25T19:05:44.115 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T19:05:44.190 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T19:05:44.197 DEBUG:teuthology.orchestra.run.smithi076:> uname -m 2024-04-25T19:05:44.214 INFO:teuthology.orchestra.run.smithi076.stdout:x86_64 2024-04-25T19:05:44.215 DEBUG:teuthology.orchestra.run.smithi076:> cat /etc/os-release 2024-04-25T19:05:44.271 INFO:teuthology.orchestra.run.smithi076.stdout:NAME="CentOS Stream" 2024-04-25T19:05:44.271 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION="9" 2024-04-25T19:05:44.271 INFO:teuthology.orchestra.run.smithi076.stdout:ID="centos" 2024-04-25T19:05:44.272 INFO:teuthology.orchestra.run.smithi076.stdout:ID_LIKE="rhel fedora" 2024-04-25T19:05:44.272 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION_ID="9" 2024-04-25T19:05:44.272 INFO:teuthology.orchestra.run.smithi076.stdout:PLATFORM_ID="platform:el9" 2024-04-25T19:05:44.272 INFO:teuthology.orchestra.run.smithi076.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T19:05:44.272 INFO:teuthology.orchestra.run.smithi076.stdout:ANSI_COLOR="0;31" 2024-04-25T19:05:44.272 INFO:teuthology.orchestra.run.smithi076.stdout:LOGO="fedora-logo-icon" 2024-04-25T19:05:44.272 INFO:teuthology.orchestra.run.smithi076.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T19:05:44.272 INFO:teuthology.orchestra.run.smithi076.stdout:HOME_URL="https://centos.org/" 2024-04-25T19:05:44.272 INFO:teuthology.orchestra.run.smithi076.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T19:05:44.272 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T19:05:44.272 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T19:05:44.273 INFO:teuthology.lock.ops:Updating smithi076.front.sepia.ceph.com on lock server 2024-04-25T19:05:44.296 DEBUG:teuthology.orchestra.run.smithi105:> uname -m 2024-04-25T19:05:44.314 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64 2024-04-25T19:05:44.314 DEBUG:teuthology.orchestra.run.smithi105:> cat /etc/os-release 2024-04-25T19:05:44.372 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="CentOS Stream" 2024-04-25T19:05:44.372 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="9" 2024-04-25T19:05:44.372 INFO:teuthology.orchestra.run.smithi105.stdout:ID="centos" 2024-04-25T19:05:44.372 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="rhel fedora" 2024-04-25T19:05:44.372 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="9" 2024-04-25T19:05:44.372 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el9" 2024-04-25T19:05:44.372 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T19:05:44.373 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31" 2024-04-25T19:05:44.373 INFO:teuthology.orchestra.run.smithi105.stdout:LOGO="fedora-logo-icon" 2024-04-25T19:05:44.373 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T19:05:44.373 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://centos.org/" 2024-04-25T19:05:44.373 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T19:05:44.373 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T19:05:44.374 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T19:05:44.374 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2024-04-25T19:05:44.395 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T19:05:44.406 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T19:05:44.412 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T19:05:44.412 DEBUG:teuthology.orchestra.run.smithi076:> test '!' -e /home/ubuntu/cephtest 2024-04-25T19:05:44.415 DEBUG:teuthology.orchestra.run.smithi105:> test '!' -e /home/ubuntu/cephtest 2024-04-25T19:05:44.431 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T19:05:44.437 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T19:05:44.437 DEBUG:teuthology.orchestra.run.smithi076:> test -z $(ls -A /var/lib/ceph) 2024-04-25T19:05:44.471 DEBUG:teuthology.orchestra.run.smithi105:> test -z $(ls -A /var/lib/ceph) 2024-04-25T19:05:44.549 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T19:05:44.605 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T19:05:44.619 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-25T19:05:44.619 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-25T19:05:44.620 DEBUG:teuthology.orchestra.run.smithi076:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T19:05:44.620 DEBUG:teuthology.orchestra.run.smithi105:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T19:05:44.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:05:44.635 DEBUG:teuthology.orchestra.run.smithi076:> uname -r 2024-04-25T19:05:44.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:05:44.637 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2024-04-25T19:05:44.691 INFO:teuthology.orchestra.run.smithi076.stdout:5.14.0-437.el9.x86_64 2024-04-25T19:05:44.692 INFO:teuthology.task.kernel:Running kernel on smithi076: 5.14.0-437.el9.x86_64 2024-04-25T19:05:44.692 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum install -y kernel 2024-04-25T19:05:44.694 INFO:teuthology.orchestra.run.smithi105.stdout:5.14.0-437.el9.x86_64 2024-04-25T19:05:44.694 INFO:teuthology.task.kernel:Running kernel on smithi105: 5.14.0-437.el9.x86_64 2024-04-25T19:05:44.694 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum install -y kernel 2024-04-25T19:05:47.513 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - BaseOS 7.3 MB/s | 8.0 MB 00:01 2024-04-25T19:05:47.771 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - BaseOS 5.7 MB/s | 8.0 MB 00:01 2024-04-25T19:05:49.306 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-04-25T19:05:49.780 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-25T19:05:53.490 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-25T19:05:53.943 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-25T19:05:55.154 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 16 kB 00:00 2024-04-25T19:05:55.729 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-04-25T19:05:57.175 INFO:teuthology.orchestra.run.smithi076.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-25T19:05:58.573 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 6.4 MB/s | 21 MB 00:03 2024-04-25T19:06:02.247 INFO:teuthology.orchestra.run.smithi076.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-25T19:06:03.639 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-25T19:06:04.104 INFO:teuthology.orchestra.run.smithi076.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T19:06:04.104 INFO:teuthology.orchestra.run.smithi076.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T19:06:04.194 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-04-25T19:06:04.195 INFO:teuthology.orchestra.run.smithi076.stdout:Nothing to do. 2024-04-25T19:06:04.195 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-04-25T19:06:04.302 DEBUG:teuthology.orchestra.run.smithi076:> echo no | sudo yum reinstall kernel || true 2024-04-25T19:06:04.839 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:00:02 ago on Thu 25 Apr 2024 07:06:02 PM UTC. 2024-04-25T19:06:04.957 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-04-25T19:06:04.971 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-04-25T19:06:04.971 INFO:teuthology.orchestra.run.smithi076.stdout: Package Architecture Version Repository Size 2024-04-25T19:06:04.971 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-04-25T19:06:04.971 INFO:teuthology.orchestra.run.smithi076.stdout:Reinstalling: 2024-04-25T19:06:04.971 INFO:teuthology.orchestra.run.smithi076.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T19:06:04.971 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:06:04.971 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-04-25T19:06:04.971 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-04-25T19:06:04.971 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:06:04.971 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 255 k 2024-04-25T19:06:04.971 INFO:teuthology.orchestra.run.smithi076.stdout:Installed size: 0 2024-04-25T19:06:04.972 INFO:teuthology.orchestra.run.smithi076.stderr:Operation aborted. 2024-04-25T19:06:05.042 INFO:teuthology.orchestra.run.smithi076.stdout:Is this ok [y/N]: 2024-04-25T19:06:05.043 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum reinstall -y kernel || true 2024-04-25T19:06:05.531 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 07:06:02 PM UTC. 2024-04-25T19:06:05.537 INFO:teuthology.orchestra.run.smithi105.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T19:06:05.537 INFO:teuthology.orchestra.run.smithi105.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T19:06:05.626 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2024-04-25T19:06:05.627 INFO:teuthology.orchestra.run.smithi105.stdout:Nothing to do. 2024-04-25T19:06:05.627 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2024-04-25T19:06:05.646 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-04-25T19:06:05.646 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-04-25T19:06:05.647 INFO:teuthology.orchestra.run.smithi076.stdout: Package Architecture Version Repository Size 2024-04-25T19:06:05.647 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-04-25T19:06:05.647 INFO:teuthology.orchestra.run.smithi076.stdout:Reinstalling: 2024-04-25T19:06:05.647 INFO:teuthology.orchestra.run.smithi076.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T19:06:05.647 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:06:05.647 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-04-25T19:06:05.647 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-04-25T19:06:05.647 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:06:05.648 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 255 k 2024-04-25T19:06:05.648 INFO:teuthology.orchestra.run.smithi076.stdout:Installed size: 0 2024-04-25T19:06:05.648 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-04-25T19:06:05.728 DEBUG:teuthology.orchestra.run.smithi105:> echo no | sudo yum reinstall kernel || true 2024-04-25T19:06:05.925 INFO:teuthology.orchestra.run.smithi076.stdout:kernel-5.14.0-437.el9.x86_64.rpm 2.0 MB/s | 255 kB 00:00 2024-04-25T19:06:05.925 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-04-25T19:06:05.926 INFO:teuthology.orchestra.run.smithi076.stdout:Total 915 kB/s | 255 kB 00:00 2024-04-25T19:06:05.943 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-04-25T19:06:06.115 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-04-25T19:06:06.115 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-04-25T19:06:06.120 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-04-25T19:06:06.120 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-04-25T19:06:06.227 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 07:06:03 PM UTC. 2024-04-25T19:06:06.249 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-04-25T19:06:06.318 INFO:teuthology.orchestra.run.smithi076.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T19:06:06.318 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T19:06:06.347 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2024-04-25T19:06:06.357 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-04-25T19:06:06.357 INFO:teuthology.orchestra.run.smithi105.stdout: Package Architecture Version Repository Size 2024-04-25T19:06:06.357 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-04-25T19:06:06.357 INFO:teuthology.orchestra.run.smithi105.stdout:Reinstalling: 2024-04-25T19:06:06.357 INFO:teuthology.orchestra.run.smithi105.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T19:06:06.357 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:06:06.357 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2024-04-25T19:06:06.357 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-04-25T19:06:06.357 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:06:06.358 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 255 k 2024-04-25T19:06:06.358 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 0 2024-04-25T19:06:06.358 INFO:teuthology.orchestra.run.smithi105.stderr:Operation aborted. 2024-04-25T19:06:06.414 INFO:teuthology.orchestra.run.smithi105.stdout:Is this ok [y/N]: 2024-04-25T19:06:06.414 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum reinstall -y kernel || true 2024-04-25T19:06:06.904 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 07:06:03 PM UTC. 2024-04-25T19:06:07.020 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2024-04-25T19:06:07.022 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-04-25T19:06:07.022 INFO:teuthology.orchestra.run.smithi105.stdout: Package Architecture Version Repository Size 2024-04-25T19:06:07.022 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-04-25T19:06:07.022 INFO:teuthology.orchestra.run.smithi105.stdout:Reinstalling: 2024-04-25T19:06:07.022 INFO:teuthology.orchestra.run.smithi105.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T19:06:07.023 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:06:07.023 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2024-04-25T19:06:07.023 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-04-25T19:06:07.023 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:06:07.023 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 255 k 2024-04-25T19:06:07.023 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 0 2024-04-25T19:06:07.023 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2024-04-25T19:06:07.026 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T19:06:07.026 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T19:06:07.233 INFO:teuthology.orchestra.run.smithi105.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.8 MB/s | 255 kB 00:00 2024-04-25T19:06:07.233 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2024-04-25T19:06:07.234 INFO:teuthology.orchestra.run.smithi105.stdout:Total 1.2 MB/s | 255 kB 00:00 2024-04-25T19:06:07.249 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2024-04-25T19:06:07.427 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2024-04-25T19:06:07.427 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2024-04-25T19:06:07.432 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2024-04-25T19:06:07.432 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2024-04-25T19:06:07.570 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2024-04-25T19:06:07.634 INFO:teuthology.orchestra.run.smithi105.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T19:06:07.634 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T19:06:07.765 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T19:06:07.765 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:06:07.765 INFO:teuthology.orchestra.run.smithi076.stdout:Reinstalled: 2024-04-25T19:06:07.765 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T19:06:07.765 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:06:07.765 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-04-25T19:06:07.947 DEBUG:teuthology.orchestra.run.smithi076:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T19:06:07.994 INFO:teuthology.orchestra.run.smithi076.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T19:06:07.994 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T19:06:07.994 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T19:06:07.995 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-25T19:06:07.995 DEBUG:teuthology.orchestra.run.smithi076:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T19:06:08.032 INFO:teuthology.orchestra.run.smithi076.stdout:ttyS1 2024-04-25T19:06:08.057 DEBUG:teuthology.parallel:result is None 2024-04-25T19:06:08.305 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T19:06:08.306 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T19:06:08.995 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T19:06:08.995 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:06:08.996 INFO:teuthology.orchestra.run.smithi105.stdout:Reinstalled: 2024-04-25T19:06:08.996 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T19:06:08.996 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:06:08.996 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2024-04-25T19:06:09.163 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T19:06:09.209 INFO:teuthology.orchestra.run.smithi105.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T19:06:09.210 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T19:06:09.210 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T19:06:09.210 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-25T19:06:09.210 DEBUG:teuthology.orchestra.run.smithi105:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T19:06:09.237 INFO:teuthology.orchestra.run.smithi105.stdout:ttyS1 2024-04-25T19:06:09.260 DEBUG:teuthology.parallel:result is None 2024-04-25T19:06:09.260 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T19:06:09.268 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T19:06:09.268 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T19:06:09.271 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T19:06:09.315 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T19:06:09.322 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T19:06:09.333 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T19:06:09.333 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T19:06:09.336 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T19:06:09.385 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T19:06:09.391 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T19:06:09.391 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T19:06:09.399 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T19:06:09.433 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T19:06:09.443 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T19:06:09.451 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T19:06:09.460 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T19:06:09.462 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T19:06:09.469 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T19:06:09.469 DEBUG:teuthology.orchestra.run.smithi076:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T19:06:09.488 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T19:06:09.525 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T19:06:09.534 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T19:06:09.534 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T19:06:09.554 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T19:06:09.580 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T19:06:09.651 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T19:06:09.696 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T19:06:09.781 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T19:06:09.818 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:06:09.818 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T19:06:09.882 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T19:06:09.919 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T19:06:09.970 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T19:06:10.051 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T19:06:10.090 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-25T19:06:10.091 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T19:06:10.154 DEBUG:teuthology.orchestra.run.smithi076:> sudo service rsyslog restart 2024-04-25T19:06:10.156 DEBUG:teuthology.orchestra.run.smithi105:> sudo service rsyslog restart 2024-04-25T19:06:10.194 INFO:teuthology.orchestra.run.smithi076.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T19:06:10.234 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T19:06:10.717 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T19:06:10.726 INFO:teuthology.task.internal:Starting timer... 2024-04-25T19:06:10.726 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T19:06:10.783 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T19:06:10.797 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-25T19:06:10.798 DEBUG:teuthology.orchestra.run.smithi076:> sudo service auditd rotate 2024-04-25T19:06:10.850 INFO:teuthology.orchestra.run.smithi076.stdout:Rotating logs: 2024-04-25T19:06:10.852 DEBUG:teuthology.orchestra.run.smithi105:> sudo service auditd rotate 2024-04-25T19:06:10.899 INFO:teuthology.orchestra.run.smithi105.stdout:Rotating logs: 2024-04-25T19:06:10.901 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T19:06:10.901 DEBUG:teuthology.orchestra.run.smithi076:> /usr/sbin/getenforce 2024-04-25T19:06:10.924 INFO:teuthology.orchestra.run.smithi076.stdout:Permissive 2024-04-25T19:06:10.925 DEBUG:teuthology.orchestra.run.smithi105:> /usr/sbin/getenforce 2024-04-25T19:06:10.941 INFO:teuthology.orchestra.run.smithi105.stdout:Permissive 2024-04-25T19:06:10.942 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi076.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi105.front.sepia.ceph.com': 'permissive'} 2024-04-25T19:06:10.942 DEBUG:teuthology.orchestra.run.smithi076:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T19:06:10.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:06:10.988 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T19:06:11.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:06:11.012 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T19:06:11.012 DEBUG:teuthology.orchestra.run.smithi076:> sudo /usr/sbin/setenforce permissive 2024-04-25T19:06:11.052 DEBUG:teuthology.orchestra.run.smithi105:> sudo /usr/sbin/setenforce permissive 2024-04-25T19:06:11.085 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T19:06:11.094 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T19:06:11.110 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T19:06:11.111 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi076.front.sepia.ceph.com,smithi105.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T19:09:37.248 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi076.front.sepia.ceph.com'), Remote(name='ubuntu@smithi105.front.sepia.ceph.com')] 2024-04-25T19:09:37.249 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-04-25T19:09:37.251 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T19:09:37.330 DEBUG:teuthology.orchestra.run.smithi076:> true 2024-04-25T19:09:37.409 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-04-25T19:09:37.410 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-04-25T19:09:37.410 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T19:09:37.493 DEBUG:teuthology.orchestra.run.smithi105:> true 2024-04-25T19:09:37.528 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-04-25T19:09:37.529 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T19:09:37.541 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T19:09:37.541 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T19:09:37.541 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T19:09:37.543 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T19:09:37.544 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T19:09:37.569 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T19:09:37.582 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T19:09:37.596 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T19:09:37.604 INFO:teuthology.orchestra.run.smithi076.stderr:sudo: ntpd: command not found 2024-04-25T19:09:37.609 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T19:09:37.614 INFO:teuthology.orchestra.run.smithi076.stdout:506 Cannot talk to daemon 2024-04-25T19:09:37.627 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T19:09:37.632 INFO:teuthology.orchestra.run.smithi105.stderr:sudo: ntpd: command not found 2024-04-25T19:09:37.640 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T19:09:37.642 INFO:teuthology.orchestra.run.smithi105.stdout:506 Cannot talk to daemon 2024-04-25T19:09:37.655 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T19:09:37.667 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T19:09:37.693 INFO:teuthology.orchestra.run.smithi076.stderr:bash: line 1: ntpq: command not found 2024-04-25T19:09:37.696 INFO:teuthology.orchestra.run.smithi076.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T19:09:37.696 INFO:teuthology.orchestra.run.smithi076.stdout:=============================================================================== 2024-04-25T19:09:37.696 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T19:09:37.696 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T19:09:37.696 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T19:09:37.696 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T19:09:37.735 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: ntpq: command not found 2024-04-25T19:09:37.738 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T19:09:37.739 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2024-04-25T19:09:37.739 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T19:09:37.739 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T19:09:37.739 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T19:09:37.739 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T19:09:37.739 INFO:teuthology.run_tasks:Running task install... 2024-04-25T19:09:37.747 DEBUG:teuthology.task.install:project ceph 2024-04-25T19:09:37.747 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}} 2024-04-25T19:09:37.747 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'} 2024-04-25T19:09:37.748 INFO:teuthology.task.install:Using flavor: default 2024-04-25T19:09:37.757 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T19:09:37.757 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T19:09:37.757 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T19:09:37.758 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T19:09:37.761 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T19:09:37.761 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T19:09:37.910 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T19:09:37.910 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T19:09:37.924 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T19:09:37.924 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T19:09:38.026 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T19:09:38.026 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:09:38.026 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T19:09:38.051 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T19:09:38.052 DEBUG:teuthology.orchestra.run.smithi076:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T19:09:38.093 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T19:09:38.093 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-25T19:09:38.093 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T19:09:38.117 DEBUG:teuthology.orchestra.run.smithi076:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T19:09:38.118 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T19:09:38.119 DEBUG:teuthology.orchestra.run.smithi105:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T19:09:38.182 DEBUG:teuthology.orchestra.run.smithi105:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T19:09:38.190 DEBUG:teuthology.orchestra.run.smithi076:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T19:09:38.253 INFO:teuthology.orchestra.run.smithi076.stdout:check_obsoletes = 1 2024-04-25T19:09:38.255 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum clean all 2024-04-25T19:09:38.256 DEBUG:teuthology.orchestra.run.smithi105:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T19:09:38.319 INFO:teuthology.orchestra.run.smithi105.stdout:check_obsoletes = 1 2024-04-25T19:09:38.320 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum clean all 2024-04-25T19:09:38.478 INFO:teuthology.orchestra.run.smithi076.stdout:48 files removed 2024-04-25T19:09:38.505 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T19:09:38.542 INFO:teuthology.orchestra.run.smithi105.stdout:48 files removed 2024-04-25T19:09:38.569 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T19:09:39.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph packages for x86_64 182 kB/s | 84 kB 00:00 2024-04-25T19:09:39.240 INFO:teuthology.orchestra.run.smithi105.stdout:ceph packages for x86_64 204 kB/s | 84 kB 00:00 2024-04-25T19:09:39.616 INFO:teuthology.orchestra.run.smithi105.stdout:ceph noarch packages 35 kB/s | 12 kB 00:00 2024-04-25T19:09:39.684 INFO:teuthology.orchestra.run.smithi076.stdout:ceph noarch packages 28 kB/s | 12 kB 00:00 2024-04-25T19:09:40.041 INFO:teuthology.orchestra.run.smithi105.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-04-25T19:09:40.093 INFO:teuthology.orchestra.run.smithi076.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-04-25T19:09:40.739 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-25T19:09:40.852 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-25T19:09:42.841 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-04-25T19:09:43.229 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-04-25T19:09:47.100 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.2 MB 00:00 2024-04-25T19:09:47.808 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - CRB 8.9 MB/s | 6.2 MB 00:00 2024-04-25T19:09:48.565 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 16 kB 00:00 2024-04-25T19:09:49.411 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-04-25T19:09:49.705 INFO:teuthology.orchestra.run.smithi076.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-25T19:09:50.899 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-25T19:09:54.651 INFO:teuthology.orchestra.run.smithi076.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-25T19:09:55.971 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-04-25T19:09:56.778 INFO:teuthology.orchestra.run.smithi076.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T19:09:56.779 INFO:teuthology.orchestra.run.smithi076.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T19:09:56.790 INFO:teuthology.orchestra.run.smithi076.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T19:09:56.791 INFO:teuthology.orchestra.run.smithi076.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T19:09:56.791 INFO:teuthology.orchestra.run.smithi076.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T19:09:56.792 INFO:teuthology.orchestra.run.smithi076.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T19:09:56.794 INFO:teuthology.orchestra.run.smithi076.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T19:09:56.796 INFO:teuthology.orchestra.run.smithi076.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T19:09:56.797 INFO:teuthology.orchestra.run.smithi076.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T19:09:56.858 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-04-25T19:09:56.870 INFO:teuthology.orchestra.run.smithi076.stdout:======================================================================================= 2024-04-25T19:09:56.870 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repository Size 2024-04-25T19:09:56.870 INFO:teuthology.orchestra.run.smithi076.stdout:======================================================================================= 2024-04-25T19:09:56.870 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-04-25T19:09:56.870 INFO:teuthology.orchestra.run.smithi076.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T19:09:56.870 INFO:teuthology.orchestra.run.smithi076.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T19:09:56.870 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T19:09:56.870 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T19:09:56.870 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T19:09:56.870 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T19:09:56.870 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T19:09:56.871 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T19:09:56.871 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T19:09:56.871 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T19:09:56.871 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T19:09:56.871 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T19:09:56.871 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T19:09:56.871 INFO:teuthology.orchestra.run.smithi076.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T19:09:56.871 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T19:09:56.871 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T19:09:56.871 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T19:09:56.871 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T19:09:56.871 INFO:teuthology.orchestra.run.smithi076.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T19:09:56.871 INFO:teuthology.orchestra.run.smithi076.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T19:09:56.871 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi076.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi076.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi076.stdout:Upgrading: 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi076.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi076.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi076.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi076.stdout:Installing dependencies: 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi076.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi076.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi076.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T19:09:56.874 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T19:09:56.874 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T19:09:56.874 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T19:09:56.874 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T19:09:56.874 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T19:09:56.874 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T19:09:56.874 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T19:09:56.874 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T19:09:56.874 INFO:teuthology.orchestra.run.smithi076.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T19:09:56.874 INFO:teuthology.orchestra.run.smithi076.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T19:09:56.874 INFO:teuthology.orchestra.run.smithi076.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T19:09:56.874 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T19:09:56.875 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T19:09:56.875 INFO:teuthology.orchestra.run.smithi076.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T19:09:56.875 INFO:teuthology.orchestra.run.smithi076.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T19:09:56.875 INFO:teuthology.orchestra.run.smithi076.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T19:09:56.875 INFO:teuthology.orchestra.run.smithi076.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T19:09:56.875 INFO:teuthology.orchestra.run.smithi076.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T19:09:56.875 INFO:teuthology.orchestra.run.smithi076.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T19:09:56.875 INFO:teuthology.orchestra.run.smithi076.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi076.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi076.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi076.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi076.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi076.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi076.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi076.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi076.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi076.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi076.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T19:09:56.877 INFO:teuthology.orchestra.run.smithi076.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T19:09:56.877 INFO:teuthology.orchestra.run.smithi076.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T19:09:56.877 INFO:teuthology.orchestra.run.smithi076.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T19:09:56.877 INFO:teuthology.orchestra.run.smithi076.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T19:09:56.877 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T19:09:56.877 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T19:09:56.877 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T19:09:56.877 INFO:teuthology.orchestra.run.smithi076.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T19:09:56.877 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T19:09:56.877 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T19:09:56.878 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T19:09:56.878 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T19:09:56.878 INFO:teuthology.orchestra.run.smithi076.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T19:09:56.878 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T19:09:56.878 INFO:teuthology.orchestra.run.smithi076.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T19:09:56.878 INFO:teuthology.orchestra.run.smithi076.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T19:09:56.878 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T19:09:56.878 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T19:09:56.878 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T19:09:56.878 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T19:09:56.879 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T19:09:56.879 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T19:09:56.879 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T19:09:56.879 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T19:09:56.879 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T19:09:56.879 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T19:09:56.879 INFO:teuthology.orchestra.run.smithi076.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T19:09:56.879 INFO:teuthology.orchestra.run.smithi076.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T19:09:56.879 INFO:teuthology.orchestra.run.smithi076.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T19:09:56.879 INFO:teuthology.orchestra.run.smithi076.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T19:09:56.879 INFO:teuthology.orchestra.run.smithi076.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi076.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi076.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi076.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi076.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi076.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi076.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T19:09:56.881 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T19:09:56.881 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T19:09:56.881 INFO:teuthology.orchestra.run.smithi076.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T19:09:56.881 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T19:09:56.881 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T19:09:56.881 INFO:teuthology.orchestra.run.smithi076.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T19:09:56.881 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T19:09:56.881 INFO:teuthology.orchestra.run.smithi076.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T19:09:56.881 INFO:teuthology.orchestra.run.smithi076.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T19:09:56.881 INFO:teuthology.orchestra.run.smithi076.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T19:09:56.881 INFO:teuthology.orchestra.run.smithi076.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T19:09:56.881 INFO:teuthology.orchestra.run.smithi076.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T19:09:56.881 INFO:teuthology.orchestra.run.smithi076.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T19:09:56.881 INFO:teuthology.orchestra.run.smithi076.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T19:09:56.881 INFO:teuthology.orchestra.run.smithi076.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T19:09:56.882 INFO:teuthology.orchestra.run.smithi076.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T19:09:56.882 INFO:teuthology.orchestra.run.smithi076.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T19:09:56.882 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T19:09:56.882 INFO:teuthology.orchestra.run.smithi076.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T19:09:56.882 INFO:teuthology.orchestra.run.smithi076.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T19:09:56.882 INFO:teuthology.orchestra.run.smithi076.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T19:09:56.882 INFO:teuthology.orchestra.run.smithi076.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T19:09:56.882 INFO:teuthology.orchestra.run.smithi076.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T19:09:56.882 INFO:teuthology.orchestra.run.smithi076.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T19:09:56.882 INFO:teuthology.orchestra.run.smithi076.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T19:09:56.882 INFO:teuthology.orchestra.run.smithi076.stdout:Installing weak dependencies: 2024-04-25T19:09:56.882 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T19:09:56.882 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:09:56.882 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-04-25T19:09:56.883 INFO:teuthology.orchestra.run.smithi076.stdout:======================================================================================= 2024-04-25T19:09:56.883 INFO:teuthology.orchestra.run.smithi076.stdout:Install 148 Packages 2024-04-25T19:09:56.883 INFO:teuthology.orchestra.run.smithi076.stdout:Upgrade 4 Packages 2024-04-25T19:09:56.883 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:09:56.883 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 214 M 2024-04-25T19:09:56.883 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-04-25T19:09:57.744 INFO:teuthology.orchestra.run.smithi076.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-04-25T19:09:58.078 INFO:teuthology.orchestra.run.smithi076.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 2.5 MB/s | 863 kB 00:00 2024-04-25T19:09:58.114 INFO:teuthology.orchestra.run.smithi105.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T19:09:58.115 INFO:teuthology.orchestra.run.smithi105.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T19:09:58.126 INFO:teuthology.orchestra.run.smithi105.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T19:09:58.127 INFO:teuthology.orchestra.run.smithi105.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T19:09:58.128 INFO:teuthology.orchestra.run.smithi105.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T19:09:58.128 INFO:teuthology.orchestra.run.smithi105.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T19:09:58.130 INFO:teuthology.orchestra.run.smithi105.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T19:09:58.132 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T19:09:58.133 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T19:09:58.194 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2024-04-25T19:09:58.203 INFO:teuthology.orchestra.run.smithi105.stdout:======================================================================================= 2024-04-25T19:09:58.203 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2024-04-25T19:09:58.203 INFO:teuthology.orchestra.run.smithi105.stdout:======================================================================================= 2024-04-25T19:09:58.203 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2024-04-25T19:09:58.203 INFO:teuthology.orchestra.run.smithi105.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T19:09:58.203 INFO:teuthology.orchestra.run.smithi105.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T19:09:58.203 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T19:09:58.203 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T19:09:58.203 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T19:09:58.203 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T19:09:58.203 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T19:09:58.203 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T19:09:58.203 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T19:09:58.204 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T19:09:58.204 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T19:09:58.204 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T19:09:58.204 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T19:09:58.204 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T19:09:58.204 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T19:09:58.204 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T19:09:58.204 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T19:09:58.204 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T19:09:58.204 INFO:teuthology.orchestra.run.smithi105.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T19:09:58.204 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T19:09:58.204 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T19:09:58.204 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T19:09:58.204 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T19:09:58.205 INFO:teuthology.orchestra.run.smithi105.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T19:09:58.205 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T19:09:58.205 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T19:09:58.205 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T19:09:58.205 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T19:09:58.205 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T19:09:58.205 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T19:09:58.205 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T19:09:58.205 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T19:09:58.205 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T19:09:58.205 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T19:09:58.205 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T19:09:58.205 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T19:09:58.205 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T19:09:58.205 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrading: 2024-04-25T19:09:58.206 INFO:teuthology.orchestra.run.smithi105.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T19:09:58.206 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T19:09:58.206 INFO:teuthology.orchestra.run.smithi105.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T19:09:58.206 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T19:09:58.206 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2024-04-25T19:09:58.206 INFO:teuthology.orchestra.run.smithi105.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T19:09:58.206 INFO:teuthology.orchestra.run.smithi105.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T19:09:58.206 INFO:teuthology.orchestra.run.smithi105.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T19:09:58.206 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T19:09:58.206 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T19:09:58.206 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T19:09:58.206 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T19:09:58.207 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T19:09:58.207 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T19:09:58.207 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T19:09:58.207 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T19:09:58.207 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T19:09:58.207 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T19:09:58.207 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T19:09:58.207 INFO:teuthology.orchestra.run.smithi105.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T19:09:58.207 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T19:09:58.207 INFO:teuthology.orchestra.run.smithi105.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T19:09:58.207 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T19:09:58.207 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T19:09:58.207 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T19:09:58.207 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T19:09:58.207 INFO:teuthology.orchestra.run.smithi105.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T19:09:58.208 INFO:teuthology.orchestra.run.smithi105.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T19:09:58.208 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T19:09:58.208 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T19:09:58.208 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T19:09:58.208 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T19:09:58.208 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T19:09:58.208 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T19:09:58.208 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T19:09:58.208 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T19:09:58.208 INFO:teuthology.orchestra.run.smithi105.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T19:09:58.208 INFO:teuthology.orchestra.run.smithi105.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T19:09:58.209 INFO:teuthology.orchestra.run.smithi105.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T19:09:58.209 INFO:teuthology.orchestra.run.smithi105.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T19:09:58.209 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T19:09:58.209 INFO:teuthology.orchestra.run.smithi105.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T19:09:58.209 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T19:09:58.209 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T19:09:58.209 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T19:09:58.209 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T19:09:58.209 INFO:teuthology.orchestra.run.smithi105.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T19:09:58.209 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T19:09:58.209 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T19:09:58.209 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T19:09:58.209 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T19:09:58.210 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T19:09:58.210 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T19:09:58.210 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T19:09:58.210 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T19:09:58.210 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T19:09:58.210 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T19:09:58.210 INFO:teuthology.orchestra.run.smithi105.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T19:09:58.210 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T19:09:58.210 INFO:teuthology.orchestra.run.smithi105.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T19:09:58.210 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T19:09:58.210 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T19:09:58.210 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T19:09:58.211 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T19:09:58.211 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T19:09:58.211 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T19:09:58.211 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T19:09:58.211 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T19:09:58.211 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T19:09:58.211 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T19:09:58.211 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T19:09:58.211 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T19:09:58.211 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T19:09:58.211 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T19:09:58.211 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T19:09:58.211 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T19:09:58.211 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T19:09:58.212 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T19:09:58.212 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T19:09:58.212 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T19:09:58.212 INFO:teuthology.orchestra.run.smithi105.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T19:09:58.212 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T19:09:58.212 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T19:09:58.212 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T19:09:58.212 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T19:09:58.212 INFO:teuthology.orchestra.run.smithi105.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T19:09:58.212 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T19:09:58.212 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T19:09:58.212 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T19:09:58.212 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T19:09:58.212 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T19:09:58.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T19:09:58.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T19:09:58.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T19:09:58.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T19:09:58.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T19:09:58.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T19:09:58.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T19:09:58.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T19:09:58.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T19:09:58.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T19:09:58.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T19:09:58.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T19:09:58.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T19:09:58.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T19:09:58.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T19:09:58.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T19:09:58.214 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T19:09:58.214 INFO:teuthology.orchestra.run.smithi105.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T19:09:58.214 INFO:teuthology.orchestra.run.smithi105.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T19:09:58.214 INFO:teuthology.orchestra.run.smithi105.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T19:09:58.214 INFO:teuthology.orchestra.run.smithi105.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T19:09:58.214 INFO:teuthology.orchestra.run.smithi105.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T19:09:58.214 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T19:09:58.214 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T19:09:58.214 INFO:teuthology.orchestra.run.smithi105.stdout:Installing weak dependencies: 2024-04-25T19:09:58.215 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T19:09:58.215 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:09:58.215 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2024-04-25T19:09:58.215 INFO:teuthology.orchestra.run.smithi105.stdout:======================================================================================= 2024-04-25T19:09:58.215 INFO:teuthology.orchestra.run.smithi105.stdout:Install 148 Packages 2024-04-25T19:09:58.215 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrade 4 Packages 2024-04-25T19:09:58.215 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:09:58.216 INFO:teuthology.orchestra.run.smithi076.stdout:(3/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 8.6 MB/s | 5.4 MB 00:00 2024-04-25T19:09:58.216 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 214 M 2024-04-25T19:09:58.216 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2024-04-25T19:09:58.228 INFO:teuthology.orchestra.run.smithi076.stdout:(4/152): ceph-immutable-object-cache-19.0.0-329 979 kB/s | 147 kB 00:00 2024-04-25T19:09:58.437 INFO:teuthology.orchestra.run.smithi076.stdout:(5/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 9.7 MB/s | 2.3 MB 00:00 2024-04-25T19:09:58.654 INFO:teuthology.orchestra.run.smithi076.stdout:(6/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 3.5 MB/s | 1.5 MB 00:00 2024-04-25T19:09:58.896 INFO:teuthology.orchestra.run.smithi076.stdout:(7/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 11 MB/s | 4.9 MB 00:00 2024-04-25T19:09:58.931 INFO:teuthology.orchestra.run.smithi105.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 30 kB/s | 6.6 kB 00:00 2024-04-25T19:09:59.648 INFO:teuthology.orchestra.run.smithi105.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 1.2 MB/s | 863 kB 00:00 2024-04-25T19:09:59.723 INFO:teuthology.orchestra.run.smithi105.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 1.9 MB/s | 147 kB 00:00 2024-04-25T19:09:59.755 INFO:teuthology.orchestra.run.smithi076.stdout:(8/152): ceph-common-19.0.0-3291.g994e2f20.el9. 9.0 MB/s | 20 MB 00:02 2024-04-25T19:09:59.880 INFO:teuthology.orchestra.run.smithi076.stdout:(9/152): ceph-radosgw-19.0.0-3291.g994e2f20.el9 10 MB/s | 10 MB 00:00 2024-04-25T19:09:59.905 INFO:teuthology.orchestra.run.smithi076.stdout:(10/152): ceph-selinux-19.0.0-3291.g994e2f20.el 168 kB/s | 25 kB 00:00 2024-04-25T19:10:00.006 INFO:teuthology.orchestra.run.smithi076.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 2.2 MB/s | 224 kB 00:00 2024-04-25T19:10:00.098 INFO:teuthology.orchestra.run.smithi076.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 295 kB/s | 27 kB 00:00 2024-04-25T19:10:00.173 INFO:teuthology.orchestra.run.smithi076.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 424 kB/s | 32 kB 00:00 2024-04-25T19:10:00.249 INFO:teuthology.orchestra.run.smithi105.stdout:(4/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 4.3 MB/s | 2.3 MB 00:00 2024-04-25T19:10:00.340 INFO:teuthology.orchestra.run.smithi076.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 4.4 MB/s | 756 kB 00:00 2024-04-25T19:10:00.432 INFO:teuthology.orchestra.run.smithi076.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.7 MB/s | 158 kB 00:00 2024-04-25T19:10:00.507 INFO:teuthology.orchestra.run.smithi076.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 1.7 MB/s | 127 kB 00:00 2024-04-25T19:10:00.607 INFO:teuthology.orchestra.run.smithi076.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 4.9 MB/s | 497 kB 00:00 2024-04-25T19:10:00.682 INFO:teuthology.orchestra.run.smithi105.stdout:(5/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 3.5 MB/s | 1.5 MB 00:00 2024-04-25T19:10:01.508 INFO:teuthology.orchestra.run.smithi076.stdout:(18/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 5.7 MB/s | 5.2 MB 00:00 2024-04-25T19:10:01.574 INFO:teuthology.orchestra.run.smithi076.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 678 kB/s | 45 kB 00:00 2024-04-25T19:10:01.582 INFO:teuthology.orchestra.run.smithi105.stdout:(6/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 5.4 MB/s | 4.9 MB 00:00 2024-04-25T19:10:01.649 INFO:teuthology.orchestra.run.smithi105.stdout:(7/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 1.8 MB/s | 5.4 MB 00:02 2024-04-25T19:10:01.767 INFO:teuthology.orchestra.run.smithi076.stdout:(20/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 5.7 MB/s | 18 MB 00:03 2024-04-25T19:10:01.792 INFO:teuthology.orchestra.run.smithi076.stdout:(21/152): python3-ceph-common-19.0.0-3291.g994e 621 kB/s | 135 kB 00:00 2024-04-25T19:10:01.851 INFO:teuthology.orchestra.run.smithi076.stdout:(22/152): python3-cephfs-19.0.0-3291.g994e2f20. 1.9 MB/s | 161 kB 00:00 2024-04-25T19:10:01.893 INFO:teuthology.orchestra.run.smithi076.stdout:(23/152): python3-rados-19.0.0-3291.g994e2f20.e 3.1 MB/s | 321 kB 00:00 2024-04-25T19:10:01.960 INFO:teuthology.orchestra.run.smithi076.stdout:(24/152): python3-rbd-19.0.0-3291.g994e2f20.el9 2.7 MB/s | 298 kB 00:00 2024-04-25T19:10:01.985 INFO:teuthology.orchestra.run.smithi076.stdout:(25/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.1 MB/s | 100 kB 00:00 2024-04-25T19:10:02.044 INFO:teuthology.orchestra.run.smithi076.stdout:(26/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.0 MB/s | 86 kB 00:00 2024-04-25T19:10:02.144 INFO:teuthology.orchestra.run.smithi076.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 1.7 MB/s | 169 kB 00:00 2024-04-25T19:10:02.236 INFO:teuthology.orchestra.run.smithi076.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 372 kB/s | 34 kB 00:00 2024-04-25T19:10:02.336 INFO:teuthology.orchestra.run.smithi076.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.4 MB/s | 142 kB 00:00 2024-04-25T19:10:02.620 INFO:teuthology.orchestra.run.smithi076.stdout:(30/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 4.8 MB/s | 3.0 MB 00:00 2024-04-25T19:10:03.341 INFO:teuthology.orchestra.run.smithi105.stdout:(8/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x86 10 MB/s | 18 MB 00:01 2024-04-25T19:10:03.399 INFO:teuthology.orchestra.run.smithi105.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 434 kB/s | 25 kB 00:00 2024-04-25T19:10:03.412 INFO:teuthology.orchestra.run.smithi076.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 3.3 MB/s | 3.5 MB 00:01 2024-04-25T19:10:03.579 INFO:teuthology.orchestra.run.smithi076.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 1.5 MB/s | 249 kB 00:00 2024-04-25T19:10:03.680 INFO:teuthology.orchestra.run.smithi076.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 489 kB/s | 49 kB 00:00 2024-04-25T19:10:03.772 INFO:teuthology.orchestra.run.smithi076.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 179 kB/s | 16 kB 00:00 2024-04-25T19:10:03.891 INFO:teuthology.orchestra.run.smithi105.stdout:(10/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 4.5 MB/s | 10 MB 00:02 2024-04-25T19:10:03.914 INFO:teuthology.orchestra.run.smithi076.stdout:(35/152): ceph-volume-19.0.0-3291.g994e2f20.el9 1.9 MB/s | 273 kB 00:00 2024-04-25T19:10:03.983 INFO:teuthology.orchestra.run.smithi105.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 2.4 MB/s | 224 kB 00:00 2024-04-25T19:10:04.048 INFO:teuthology.orchestra.run.smithi076.stdout:(36/152): ceph-mgr-diskprediction-local-19.0.0- 5.2 MB/s | 7.4 MB 00:01 2024-04-25T19:10:04.066 INFO:teuthology.orchestra.run.smithi105.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 324 kB/s | 27 kB 00:00 2024-04-25T19:10:04.115 INFO:teuthology.orchestra.run.smithi076.stdout:(37/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 274 kB/s | 18 kB 00:00 2024-04-25T19:10:04.158 INFO:teuthology.orchestra.run.smithi105.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 346 kB/s | 32 kB 00:00 2024-04-25T19:10:04.181 INFO:teuthology.orchestra.run.smithi076.stdout:(38/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 2.8 MB/s | 762 kB 00:00 2024-04-25T19:10:04.290 INFO:teuthology.orchestra.run.smithi076.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 7.3 MB/s | 803 kB 00:00 2024-04-25T19:10:04.316 INFO:teuthology.orchestra.run.smithi076.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 400 kB/s | 80 kB 00:00 2024-04-25T19:10:04.325 INFO:teuthology.orchestra.run.smithi105.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 4.4 MB/s | 756 kB 00:00 2024-04-25T19:10:04.333 INFO:teuthology.orchestra.run.smithi076.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-25T19:10:04.366 INFO:teuthology.orchestra.run.smithi076.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-04-25T19:10:04.418 INFO:teuthology.orchestra.run.smithi105.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.7 MB/s | 158 kB 00:00 2024-04-25T19:10:04.433 INFO:teuthology.orchestra.run.smithi076.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-04-25T19:10:04.459 INFO:teuthology.orchestra.run.smithi076.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-04-25T19:10:04.484 INFO:teuthology.orchestra.run.smithi076.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-25T19:10:04.493 INFO:teuthology.orchestra.run.smithi105.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 1.7 MB/s | 127 kB 00:00 2024-04-25T19:10:04.510 INFO:teuthology.orchestra.run.smithi076.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-25T19:10:04.535 INFO:teuthology.orchestra.run.smithi076.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T19:10:04.569 INFO:teuthology.orchestra.run.smithi076.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 12 MB/s | 689 kB 00:00 2024-04-25T19:10:04.602 INFO:teuthology.orchestra.run.smithi076.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-04-25T19:10:04.628 INFO:teuthology.orchestra.run.smithi076.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-25T19:10:04.651 INFO:teuthology.orchestra.run.smithi105.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 3.1 MB/s | 497 kB 00:00 2024-04-25T19:10:04.653 INFO:teuthology.orchestra.run.smithi076.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 428 kB/s | 21 kB 00:00 2024-04-25T19:10:04.678 INFO:teuthology.orchestra.run.smithi076.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T19:10:04.695 INFO:teuthology.orchestra.run.smithi076.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 792 kB/s | 33 kB 00:00 2024-04-25T19:10:04.737 INFO:teuthology.orchestra.run.smithi076.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-04-25T19:10:04.796 INFO:teuthology.orchestra.run.smithi076.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 25 MB/s | 3.0 MB 00:00 2024-04-25T19:10:04.813 INFO:teuthology.orchestra.run.smithi076.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 152 kB/s | 11 kB 00:00 2024-04-25T19:10:04.838 INFO:teuthology.orchestra.run.smithi076.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 160 kB 00:00 2024-04-25T19:10:04.864 INFO:teuthology.orchestra.run.smithi076.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-04-25T19:10:04.889 INFO:teuthology.orchestra.run.smithi076.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-25T19:10:04.914 INFO:teuthology.orchestra.run.smithi076.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 12 MB/s | 582 kB 00:00 2024-04-25T19:10:04.940 INFO:teuthology.orchestra.run.smithi076.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 515 kB/s | 25 kB 00:00 2024-04-25T19:10:04.965 INFO:teuthology.orchestra.run.smithi076.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-04-25T19:10:04.990 INFO:teuthology.orchestra.run.smithi076.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T19:10:05.016 INFO:teuthology.orchestra.run.smithi076.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T19:10:05.041 INFO:teuthology.orchestra.run.smithi076.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 705 kB/s | 35 kB 00:00 2024-04-25T19:10:05.192 INFO:teuthology.orchestra.run.smithi076.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-04-25T19:10:05.250 INFO:teuthology.orchestra.run.smithi076.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-04-25T19:10:05.292 INFO:teuthology.orchestra.run.smithi076.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-25T19:10:05.334 INFO:teuthology.orchestra.run.smithi076.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-04-25T19:10:05.360 INFO:teuthology.orchestra.run.smithi076.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-25T19:10:05.477 INFO:teuthology.orchestra.run.smithi076.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 8.7 MB/s | 1.0 MB 00:00 2024-04-25T19:10:05.544 INFO:teuthology.orchestra.run.smithi076.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 28 MB/s | 5.8 MB 00:00 2024-04-25T19:10:05.569 INFO:teuthology.orchestra.run.smithi076.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 976 kB/s | 87 kB 00:00 2024-04-25T19:10:05.611 INFO:teuthology.orchestra.run.smithi076.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 6.0 MB/s | 244 kB 00:00 2024-04-25T19:10:05.626 INFO:teuthology.orchestra.run.smithi105.stdout:(18/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 5.3 MB/s | 5.2 MB 00:00 2024-04-25T19:10:05.662 INFO:teuthology.orchestra.run.smithi076.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 5.1 MB/s | 248 kB 00:00 2024-04-25T19:10:05.718 INFO:teuthology.orchestra.run.smithi105.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 493 kB/s | 45 kB 00:00 2024-04-25T19:10:05.737 INFO:teuthology.orchestra.run.smithi076.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 31 MB/s | 6.0 MB 00:00 2024-04-25T19:10:05.762 INFO:teuthology.orchestra.run.smithi076.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 261 kB/s | 26 kB 00:00 2024-04-25T19:10:05.779 INFO:teuthology.orchestra.run.smithi076.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 456 kB/s | 19 kB 00:00 2024-04-25T19:10:05.804 INFO:teuthology.orchestra.run.smithi076.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-04-25T19:10:05.818 INFO:teuthology.orchestra.run.smithi105.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 1.3 MB/s | 135 kB 00:00 2024-04-25T19:10:05.821 INFO:teuthology.orchestra.run.smithi076.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 833 kB/s | 35 kB 00:00 2024-04-25T19:10:05.863 INFO:teuthology.orchestra.run.smithi076.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-04-25T19:10:05.910 INFO:teuthology.orchestra.run.smithi105.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 1.7 MB/s | 161 kB 00:00 2024-04-25T19:10:05.914 INFO:teuthology.orchestra.run.smithi076.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.6 MB/s | 222 kB 00:00 2024-04-25T19:10:05.981 INFO:teuthology.orchestra.run.smithi076.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 5.1 MB 00:00 2024-04-25T19:10:06.002 INFO:teuthology.orchestra.run.smithi105.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 3.4 MB/s | 321 kB 00:00 2024-04-25T19:10:06.006 INFO:teuthology.orchestra.run.smithi076.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 848 kB/s | 77 kB 00:00 2024-04-25T19:10:06.031 INFO:teuthology.orchestra.run.smithi076.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-04-25T19:10:06.057 INFO:teuthology.orchestra.run.smithi076.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-25T19:10:06.074 INFO:teuthology.orchestra.run.smithi076.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.7 MB/s | 159 kB 00:00 2024-04-25T19:10:06.100 INFO:teuthology.orchestra.run.smithi076.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 6.5 MB/s | 279 kB 00:00 2024-04-25T19:10:06.124 INFO:teuthology.orchestra.run.smithi076.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T19:10:06.150 INFO:teuthology.orchestra.run.smithi076.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T19:10:06.184 INFO:teuthology.orchestra.run.smithi076.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 42 kB 00:00 2024-04-25T19:10:06.226 INFO:teuthology.orchestra.run.smithi076.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.1 MB/s | 44 kB 00:00 2024-04-25T19:10:06.268 INFO:teuthology.orchestra.run.smithi076.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 5.5 MB/s | 221 kB 00:00 2024-04-25T19:10:06.302 INFO:teuthology.orchestra.run.smithi076.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 1.1 MB/s | 36 kB 00:00 2024-04-25T19:10:06.420 INFO:teuthology.orchestra.run.smithi105.stdout:(23/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 14 MB/s | 43 MB 00:03 2024-04-25T19:10:06.445 INFO:teuthology.orchestra.run.smithi105.stdout:(24/152): python3-rbd-19.0.0-3291.g994e2f20.el9 674 kB/s | 298 kB 00:00 2024-04-25T19:10:06.470 INFO:teuthology.orchestra.run.smithi105.stdout:(25/152): python3-rgw-19.0.0-3291.g994e2f20.el9 2.0 MB/s | 100 kB 00:00 2024-04-25T19:10:06.504 INFO:teuthology.orchestra.run.smithi105.stdout:(26/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.4 MB/s | 86 kB 00:00 2024-04-25T19:10:06.621 INFO:teuthology.orchestra.run.smithi105.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 1.4 MB/s | 169 kB 00:00 2024-04-25T19:10:06.688 INFO:teuthology.orchestra.run.smithi105.stdout:(28/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 14 MB/s | 3.0 MB 00:00 2024-04-25T19:10:06.713 INFO:teuthology.orchestra.run.smithi105.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3291.g 372 kB/s | 34 kB 00:00 2024-04-25T19:10:06.727 INFO:teuthology.orchestra.run.smithi076.stdout:(95/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 6.3 MB/s | 43 MB 00:06 2024-04-25T19:10:06.739 INFO:teuthology.orchestra.run.smithi105.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 2.8 MB/s | 142 kB 00:00 2024-04-25T19:10:06.752 INFO:teuthology.orchestra.run.smithi076.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 106 kB/s | 47 kB 00:00 2024-04-25T19:10:06.961 INFO:teuthology.orchestra.run.smithi076.stdout:(97/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-04-25T19:10:06.986 INFO:teuthology.orchestra.run.smithi076.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 1.2 MB/s | 305 kB 00:00 2024-04-25T19:10:07.011 INFO:teuthology.orchestra.run.smithi076.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 226 kB/s | 58 kB 00:00 2024-04-25T19:10:07.028 INFO:teuthology.orchestra.run.smithi076.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.4 MB/s | 94 kB 00:00 2024-04-25T19:10:07.054 INFO:teuthology.orchestra.run.smithi076.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-04-25T19:10:07.079 INFO:teuthology.orchestra.run.smithi076.stdout:(102/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 948 kB/s | 64 kB 00:00 2024-04-25T19:10:07.104 INFO:teuthology.orchestra.run.smithi076.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 296 kB/s | 22 kB 00:00 2024-04-25T19:10:07.146 INFO:teuthology.orchestra.run.smithi076.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 9.2 MB/s | 863 kB 00:00 2024-04-25T19:10:07.224 INFO:teuthology.orchestra.run.smithi105.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 15 MB/s | 7.4 MB 00:00 2024-04-25T19:10:07.300 INFO:teuthology.orchestra.run.smithi105.stdout:(32/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 6.0 MB/s | 3.5 MB 00:00 2024-04-25T19:10:07.325 INFO:teuthology.orchestra.run.smithi105.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3291.g99 2.4 MB/s | 249 kB 00:00 2024-04-25T19:10:07.350 INFO:teuthology.orchestra.run.smithi105.stdout:(34/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 986 kB/s | 49 kB 00:00 2024-04-25T19:10:07.367 INFO:teuthology.orchestra.run.smithi105.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3291.g9 392 kB/s | 16 kB 00:00 2024-04-25T19:10:07.388 INFO:teuthology.orchestra.run.smithi076.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 80 kB/s | 19 kB 00:00 2024-04-25T19:10:07.417 INFO:teuthology.orchestra.run.smithi105.stdout:(36/152): ceph-volume-19.0.0-3291.g994e2f20.el9 4.0 MB/s | 273 kB 00:00 2024-04-25T19:10:07.459 INFO:teuthology.orchestra.run.smithi105.stdout:(37/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 8.1 MB/s | 762 kB 00:00 2024-04-25T19:10:07.484 INFO:teuthology.orchestra.run.smithi105.stdout:(38/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 272 kB/s | 18 kB 00:00 2024-04-25T19:10:07.597 INFO:teuthology.orchestra.run.smithi076.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 629 kB/s | 309 kB 00:00 2024-04-25T19:10:07.610 INFO:teuthology.orchestra.run.smithi105.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 535 kB/s | 80 kB 00:00 2024-04-25T19:10:07.647 INFO:teuthology.orchestra.run.smithi076.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-04-25T19:10:07.668 INFO:teuthology.orchestra.run.smithi105.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 4.3 MB/s | 803 kB 00:00 2024-04-25T19:10:07.689 INFO:teuthology.orchestra.run.smithi076.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 282 kB/s | 26 kB 00:00 2024-04-25T19:10:07.694 INFO:teuthology.orchestra.run.smithi105.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-04-25T19:10:07.719 INFO:teuthology.orchestra.run.smithi105.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-04-25T19:10:07.739 INFO:teuthology.orchestra.run.smithi076.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 528 kB/s | 49 kB 00:00 2024-04-25T19:10:07.781 INFO:teuthology.orchestra.run.smithi076.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 732 kB/s | 67 kB 00:00 2024-04-25T19:10:07.832 INFO:teuthology.orchestra.run.smithi076.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-04-25T19:10:07.911 INFO:teuthology.orchestra.run.smithi105.stdout:(43/152): ceph-common-19.0.0-3291.g994e2f20.el9 2.1 MB/s | 20 MB 00:09 2024-04-25T19:10:07.940 INFO:teuthology.orchestra.run.smithi076.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-04-25T19:10:07.944 INFO:teuthology.orchestra.run.smithi105.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 4.9 MB/s | 1.2 MB 00:00 2024-04-25T19:10:07.970 INFO:teuthology.orchestra.run.smithi105.stdout:(45/152): python3-ply-3.11-14.el9.noarch.rpm 426 kB/s | 106 kB 00:00 2024-04-25T19:10:07.982 INFO:teuthology.orchestra.run.smithi076.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.1 MB/s | 838 kB 00:00 2024-04-25T19:10:07.995 INFO:teuthology.orchestra.run.smithi105.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-04-25T19:10:08.020 INFO:teuthology.orchestra.run.smithi105.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 2.5 MB/s | 125 kB 00:00 2024-04-25T19:10:08.045 INFO:teuthology.orchestra.run.smithi105.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-04-25T19:10:08.058 INFO:teuthology.orchestra.run.smithi076.stdout:(114/152): libarrow-9.0.0-10.el9.x86_64.rpm 6.6 MB/s | 4.4 MB 00:00 2024-04-25T19:10:08.070 INFO:teuthology.orchestra.run.smithi105.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 8.9 MB/s | 689 kB 00:00 2024-04-25T19:10:08.075 INFO:teuthology.orchestra.run.smithi076.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 324 kB/s | 43 kB 00:00 2024-04-25T19:10:08.087 INFO:teuthology.orchestra.run.smithi105.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 2.6 MB/s | 107 kB 00:00 2024-04-25T19:10:08.100 INFO:teuthology.orchestra.run.smithi076.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 274 kB/s | 32 kB 00:00 2024-04-25T19:10:08.121 INFO:teuthology.orchestra.run.smithi105.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 943 kB 00:00 2024-04-25T19:10:08.142 INFO:teuthology.orchestra.run.smithi076.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-04-25T19:10:08.146 INFO:teuthology.orchestra.run.smithi105.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 285 kB/s | 21 kB 00:00 2024-04-25T19:10:08.171 INFO:teuthology.orchestra.run.smithi105.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-04-25T19:10:08.192 INFO:teuthology.orchestra.run.smithi076.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-04-25T19:10:08.197 INFO:teuthology.orchestra.run.smithi105.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 662 kB/s | 33 kB 00:00 2024-04-25T19:10:08.217 INFO:teuthology.orchestra.run.smithi076.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-04-25T19:10:08.222 INFO:teuthology.orchestra.run.smithi105.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-04-25T19:10:08.243 INFO:teuthology.orchestra.run.smithi076.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-04-25T19:10:08.247 INFO:teuthology.orchestra.run.smithi105.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 225 kB/s | 11 kB 00:00 2024-04-25T19:10:08.285 INFO:teuthology.orchestra.run.smithi076.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 382 kB/s | 35 kB 00:00 2024-04-25T19:10:08.306 INFO:teuthology.orchestra.run.smithi105.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-04-25T19:10:08.319 INFO:teuthology.orchestra.run.smithi076.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-04-25T19:10:08.331 INFO:teuthology.orchestra.run.smithi105.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.5 MB/s | 160 kB 00:00 2024-04-25T19:10:08.356 INFO:teuthology.orchestra.run.smithi105.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 425 kB/s | 45 kB 00:00 2024-04-25T19:10:08.381 INFO:teuthology.orchestra.run.smithi105.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 582 kB 00:00 2024-04-25T19:10:08.386 INFO:teuthology.orchestra.run.smithi076.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-25T19:10:08.411 INFO:teuthology.orchestra.run.smithi076.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-04-25T19:10:08.415 INFO:teuthology.orchestra.run.smithi105.stdout:(61/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 662 kB 00:00 2024-04-25T19:10:08.441 INFO:teuthology.orchestra.run.smithi105.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 305 kB/s | 25 kB 00:00 2024-04-25T19:10:08.466 INFO:teuthology.orchestra.run.smithi105.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 566 kB/s | 47 kB 00:00 2024-04-25T19:10:08.478 INFO:teuthology.orchestra.run.smithi076.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-04-25T19:10:08.491 INFO:teuthology.orchestra.run.smithi105.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T19:10:08.511 INFO:teuthology.orchestra.run.smithi076.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-04-25T19:10:08.516 INFO:teuthology.orchestra.run.smithi105.stdout:(65/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-25T19:10:08.541 INFO:teuthology.orchestra.run.smithi105.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 470 kB/s | 35 kB 00:00 2024-04-25T19:10:08.578 INFO:teuthology.orchestra.run.smithi076.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-04-25T19:10:08.620 INFO:teuthology.orchestra.run.smithi076.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-04-25T19:10:08.659 INFO:teuthology.orchestra.run.smithi105.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 28 MB/s | 4.7 MB 00:00 2024-04-25T19:10:08.696 INFO:teuthology.orchestra.run.smithi076.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-04-25T19:10:08.700 INFO:teuthology.orchestra.run.smithi105.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-04-25T19:10:08.721 INFO:teuthology.orchestra.run.smithi076.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 461 kB/s | 46 kB 00:00 2024-04-25T19:10:08.767 INFO:teuthology.orchestra.run.smithi105.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-04-25T19:10:08.788 INFO:teuthology.orchestra.run.smithi076.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 857 kB/s | 79 kB 00:00 2024-04-25T19:10:08.801 INFO:teuthology.orchestra.run.smithi105.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 6.3 MB/s | 911 kB 00:00 2024-04-25T19:10:08.818 INFO:teuthology.orchestra.run.smithi076.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 595 kB/s | 58 kB 00:00 2024-04-25T19:10:08.826 INFO:teuthology.orchestra.run.smithi105.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-04-25T19:10:08.876 INFO:teuthology.orchestra.run.smithi105.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 14 MB/s | 1.0 MB 00:00 2024-04-25T19:10:08.901 INFO:teuthology.orchestra.run.smithi105.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-04-25T19:10:08.906 INFO:teuthology.orchestra.run.smithi076.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-04-25T19:10:08.931 INFO:teuthology.orchestra.run.smithi076.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 146 kB/s | 16 kB 00:00 2024-04-25T19:10:08.993 INFO:teuthology.orchestra.run.smithi105.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 30 MB/s | 5.8 MB 00:00 2024-04-25T19:10:09.019 INFO:teuthology.orchestra.run.smithi105.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.1 MB/s | 244 kB 00:00 2024-04-25T19:10:09.023 INFO:teuthology.orchestra.run.smithi076.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 769 kB/s | 90 kB 00:00 2024-04-25T19:10:09.048 INFO:teuthology.orchestra.run.smithi076.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 241 kB/s | 28 kB 00:00 2024-04-25T19:10:09.102 INFO:teuthology.orchestra.run.smithi105.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 6.0 MB 00:00 2024-04-25T19:10:09.124 INFO:teuthology.orchestra.run.smithi076.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-04-25T19:10:09.128 INFO:teuthology.orchestra.run.smithi105.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.8 MB/s | 248 kB 00:00 2024-04-25T19:10:09.149 INFO:teuthology.orchestra.run.smithi076.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-25T19:10:09.152 INFO:teuthology.orchestra.run.smithi105.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 195 kB/s | 26 kB 00:00 2024-04-25T19:10:09.178 INFO:teuthology.orchestra.run.smithi105.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 255 kB/s | 19 kB 00:00 2024-04-25T19:10:09.203 INFO:teuthology.orchestra.run.smithi105.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-04-25T19:10:09.330 INFO:teuthology.orchestra.run.smithi105.stdout:(81/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-04-25T19:10:09.330 INFO:teuthology.orchestra.run.smithi105.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.3 MB/s | 250 kB 00:00 2024-04-25T19:10:09.330 INFO:teuthology.orchestra.run.smithi105.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-04-25T19:10:09.330 INFO:teuthology.orchestra.run.smithi076.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 642 kB/s | 59 kB 00:00 2024-04-25T19:10:09.330 INFO:teuthology.orchestra.run.smithi076.stdout:(140/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.0 MB/s | 2.0 MB 00:01 2024-04-25T19:10:09.330 INFO:teuthology.orchestra.run.smithi076.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 267 kB/s | 36 kB 00:00 2024-04-25T19:10:09.331 INFO:teuthology.orchestra.run.smithi076.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 552 kB/s | 51 kB 00:00 2024-04-25T19:10:09.354 INFO:teuthology.orchestra.run.smithi105.stdout:(84/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 5.1 MB 00:00 2024-04-25T19:10:09.359 INFO:teuthology.orchestra.run.smithi076.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-25T19:10:09.379 INFO:teuthology.orchestra.run.smithi105.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 672 kB/s | 77 kB 00:00 2024-04-25T19:10:09.385 INFO:teuthology.orchestra.run.smithi076.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 887 kB/s | 90 kB 00:00 2024-04-25T19:10:09.405 INFO:teuthology.orchestra.run.smithi105.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 367 kB/s | 42 kB 00:00 2024-04-25T19:10:09.410 INFO:teuthology.orchestra.run.smithi076.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-04-25T19:10:09.430 INFO:teuthology.orchestra.run.smithi105.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T19:10:09.452 INFO:teuthology.orchestra.run.smithi076.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 216 kB/s | 20 kB 00:00 2024-04-25T19:10:09.455 INFO:teuthology.orchestra.run.smithi105.stdout:(88/152): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-04-25T19:10:09.480 INFO:teuthology.orchestra.run.smithi105.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-04-25T19:10:09.485 INFO:teuthology.orchestra.run.smithi076.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-25T19:10:09.506 INFO:teuthology.orchestra.run.smithi105.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 683 kB/s | 51 kB 00:00 2024-04-25T19:10:09.531 INFO:teuthology.orchestra.run.smithi105.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-04-25T19:10:09.536 INFO:teuthology.orchestra.run.smithi076.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-25T19:10:09.556 INFO:teuthology.orchestra.run.smithi105.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 830 kB/s | 42 kB 00:00 2024-04-25T19:10:09.581 INFO:teuthology.orchestra.run.smithi105.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 891 kB/s | 44 kB 00:00 2024-04-25T19:10:09.606 INFO:teuthology.orchestra.run.smithi105.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.5 MB/s | 221 kB 00:00 2024-04-25T19:10:09.611 INFO:teuthology.orchestra.run.smithi076.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 18 MB/s | 1.3 MB 00:00 2024-04-25T19:10:09.649 INFO:teuthology.orchestra.run.smithi105.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 556 kB/s | 36 kB 00:00 2024-04-25T19:10:09.674 INFO:teuthology.orchestra.run.smithi105.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 736 kB/s | 47 kB 00:00 2024-04-25T19:10:09.733 INFO:teuthology.orchestra.run.smithi105.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.7 MB/s | 305 kB 00:00 2024-04-25T19:10:09.745 INFO:teuthology.orchestra.run.smithi076.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-04-25T19:10:09.758 INFO:teuthology.orchestra.run.smithi105.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 710 kB/s | 58 kB 00:00 2024-04-25T19:10:09.800 INFO:teuthology.orchestra.run.smithi105.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-04-25T19:10:09.870 INFO:teuthology.orchestra.run.smithi076.stdout:(151/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 8.0 MB/s | 3.1 MB 00:00 2024-04-25T19:10:09.959 INFO:teuthology.orchestra.run.smithi105.stdout:(100/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-04-25T19:10:09.962 INFO:teuthology.orchestra.run.smithi076.stdout:(152/152): librados2-19.0.0-3291.g994e2f20.el9. 6.5 MB/s | 3.3 MB 00:00 2024-04-25T19:10:09.966 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-04-25T19:10:09.966 INFO:teuthology.orchestra.run.smithi076.stdout:Total 16 MB/s | 214 MB 00:13 2024-04-25T19:10:09.984 INFO:teuthology.orchestra.run.smithi105.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 285 kB/s | 64 kB 00:00 2024-04-25T19:10:10.009 INFO:teuthology.orchestra.run.smithi105.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 216 kB/s | 45 kB 00:00 2024-04-25T19:10:10.034 INFO:teuthology.orchestra.run.smithi105.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 447 kB/s | 22 kB 00:00 2024-04-25T19:10:10.059 INFO:teuthology.orchestra.run.smithi105.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 11 MB/s | 863 kB 00:00 2024-04-25T19:10:11.081 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-04-25T19:10:11.228 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-04-25T19:10:11.228 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-04-25T19:10:12.951 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-04-25T19:10:12.951 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-04-25T19:10:18.868 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-04-25T19:10:18.949 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T19:10:18.987 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T19:10:19.049 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T19:10:19.083 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T19:10:19.109 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T19:10:19.169 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T19:10:19.504 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T19:10:19.620 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T19:10:19.744 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T19:10:19.771 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T19:10:19.862 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T19:10:19.892 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T19:10:20.049 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T19:10:20.098 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T19:10:20.126 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T19:10:20.222 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T19:10:20.252 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T19:10:20.315 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T19:10:20.337 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T19:10:20.374 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T19:10:20.596 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T19:10:20.629 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T19:10:20.752 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T19:10:20.852 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T19:10:20.895 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T19:10:20.930 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T19:10:21.004 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T19:10:21.038 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T19:10:21.082 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T19:10:21.115 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T19:10:21.142 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T19:10:21.212 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T19:10:21.265 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T19:10:21.295 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T19:10:21.339 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T19:10:21.425 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T19:10:21.459 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T19:10:21.502 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T19:10:21.531 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T19:10:21.602 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T19:10:21.646 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T19:10:21.720 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T19:10:21.862 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T19:10:21.915 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T19:10:22.443 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T19:10:22.528 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T19:10:22.557 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T19:10:22.600 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T19:10:22.638 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T19:10:22.680 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T19:10:22.842 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T19:10:22.892 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T19:10:22.947 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T19:10:23.019 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T19:10:23.051 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T19:10:23.088 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T19:10:23.135 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T19:10:23.171 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T19:10:23.213 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T19:10:23.255 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T19:10:23.394 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T19:10:23.454 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T19:10:23.484 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T19:10:23.525 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T19:10:23.620 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T19:10:23.659 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T19:10:23.699 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T19:10:23.725 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T19:10:23.750 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T19:10:23.825 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T19:10:23.953 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T19:10:23.983 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T19:10:24.011 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T19:10:24.046 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T19:10:24.085 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T19:10:24.116 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T19:10:24.146 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T19:10:24.273 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T19:10:24.374 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T19:10:24.416 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T19:10:24.450 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T19:10:24.538 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T19:10:24.580 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T19:10:24.620 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T19:10:24.667 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T19:10:24.765 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T19:10:24.813 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T19:10:25.535 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T19:10:25.587 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T19:10:25.620 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T19:10:25.721 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T19:10:25.778 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T19:10:25.865 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T19:10:25.914 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T19:10:26.091 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T19:10:26.121 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T19:10:26.198 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T19:10:26.839 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T19:10:26.952 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T19:10:28.195 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T19:10:28.287 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T19:10:28.315 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T19:10:28.348 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T19:10:28.667 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T19:10:28.720 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T19:10:28.809 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T19:10:28.833 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T19:10:28.869 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T19:10:29.358 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T19:10:29.381 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T19:10:29.452 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T19:10:29.475 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T19:10:31.712 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T19:10:31.747 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T19:10:32.307 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T19:10:32.352 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T19:10:32.452 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T19:10:32.452 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T19:10:32.452 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T19:10:32.452 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:10:32.487 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T19:10:44.100 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T19:10:44.101 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /sys 2024-04-25T19:10:44.101 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /proc 2024-04-25T19:10:44.101 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /mnt 2024-04-25T19:10:44.101 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /var/tmp 2024-04-25T19:10:44.101 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /home 2024-04-25T19:10:44.101 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /root 2024-04-25T19:10:44.101 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /tmp 2024-04-25T19:10:44.101 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:10:44.272 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T19:10:44.287 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T19:10:44.287 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:10:44.288 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T19:10:44.288 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:10:45.823 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T19:10:45.849 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T19:10:45.849 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:10:45.849 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T19:10:45.849 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T19:10:45.849 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T19:10:45.849 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:10:46.073 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T19:10:46.096 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T19:10:46.097 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:10:46.097 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T19:10:46.097 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T19:10:46.097 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T19:10:46.097 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:10:46.558 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T19:10:46.581 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T19:10:46.581 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:10:46.581 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T19:10:46.581 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T19:10:46.581 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T19:10:46.581 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:10:46.688 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T19:10:46.737 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T19:10:46.801 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T19:10:46.990 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T19:10:47.034 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T19:10:47.113 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T19:10:47.226 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T19:10:47.366 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T19:10:47.410 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T19:10:47.442 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T19:10:47.470 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T19:10:47.922 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T19:10:47.953 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T19:10:47.985 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T19:10:48.038 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T19:10:48.068 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T19:10:49.100 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T19:10:49.137 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T19:10:50.104 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T19:10:50.129 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T19:10:50.232 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T19:10:50.395 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T19:10:50.431 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T19:10:50.454 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T19:10:50.455 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:10:50.455 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T19:10:50.455 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T19:10:50.455 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T19:10:50.455 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:10:50.499 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T19:10:50.510 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T19:10:50.611 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T19:10:50.657 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T19:10:51.638 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T19:10:51.672 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T19:10:51.697 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T19:10:51.697 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:10:51.697 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T19:10:51.697 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T19:10:51.697 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T19:10:51.697 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:10:51.740 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T19:10:51.763 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T19:10:51.763 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:10:51.763 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T19:10:51.763 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:10:51.816 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T19:10:51.839 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T19:10:51.839 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:10:51.839 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T19:10:51.839 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T19:10:51.839 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T19:10:51.839 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:10:52.150 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T19:10:52.173 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T19:10:52.173 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:10:52.173 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T19:10:52.173 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T19:10:52.173 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T19:10:52.173 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:10:56.439 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T19:10:56.480 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T19:10:56.521 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T19:10:56.553 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T19:10:56.580 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T19:10:56.762 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T19:10:56.802 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T19:10:56.832 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T19:10:57.454 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T19:10:57.499 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T19:10:57.519 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T19:10:58.371 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T19:10:58.453 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T19:10:58.493 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T19:10:58.575 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T19:10:58.607 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T19:10:58.607 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T19:10:58.691 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T19:10:58.724 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T19:10:58.724 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T19:10:58.819 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T19:10:58.889 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T19:11:04.879 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T19:11:04.880 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T19:11:04.880 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T19:11:04.880 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T19:11:04.880 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T19:11:04.880 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T19:11:04.880 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T19:11:04.880 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T19:11:04.880 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T19:11:04.880 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T19:11:04.881 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T19:11:04.881 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T19:11:04.881 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T19:11:04.881 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T19:11:04.881 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T19:11:04.881 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T19:11:04.881 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T19:11:04.881 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T19:11:04.881 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T19:11:04.881 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T19:11:04.881 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T19:11:04.881 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T19:11:04.881 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T19:11:04.882 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T19:11:04.882 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T19:11:04.882 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T19:11:04.882 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T19:11:04.882 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T19:11:04.882 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T19:11:04.882 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T19:11:04.896 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T19:11:04.896 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T19:11:04.896 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T19:11:04.896 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T19:11:04.896 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T19:11:04.896 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T19:11:04.896 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T19:11:04.896 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T19:11:04.896 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T19:11:04.896 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T19:11:04.896 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T19:11:04.896 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T19:11:04.897 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T19:11:04.897 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T19:11:04.897 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T19:11:04.897 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T19:11:04.897 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T19:11:04.897 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T19:11:04.897 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T19:11:04.897 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T19:11:04.897 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T19:11:04.897 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T19:11:04.897 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T19:11:04.897 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T19:11:04.897 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T19:11:04.897 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T19:11:04.898 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T19:11:04.898 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T19:11:04.898 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T19:11:04.898 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T19:11:04.898 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T19:11:04.898 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T19:11:04.898 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T19:11:04.898 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T19:11:04.898 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T19:11:04.898 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T19:11:04.899 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T19:11:04.899 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T19:11:04.899 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T19:11:04.899 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T19:11:04.899 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T19:11:04.899 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T19:11:04.899 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T19:11:04.899 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T19:11:04.899 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T19:11:04.899 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T19:11:04.899 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T19:11:04.899 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T19:11:04.899 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T19:11:04.900 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T19:11:04.900 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T19:11:04.900 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T19:11:04.900 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T19:11:04.900 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T19:11:04.900 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T19:11:04.900 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T19:11:04.900 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T19:11:04.900 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T19:11:04.900 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T19:11:04.900 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T19:11:04.900 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T19:11:04.900 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T19:11:04.901 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T19:11:04.901 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T19:11:04.901 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T19:11:04.901 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T19:11:04.901 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T19:11:04.901 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T19:11:04.901 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T19:11:04.901 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T19:11:04.901 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T19:11:04.902 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T19:11:04.902 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T19:11:04.902 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T19:11:04.902 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T19:11:04.902 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T19:11:04.902 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T19:11:04.902 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T19:11:04.902 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T19:11:04.902 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T19:11:04.902 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T19:11:04.902 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T19:11:04.902 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T19:11:04.902 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T19:11:04.902 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T19:11:04.903 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T19:11:04.903 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T19:11:04.903 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T19:11:04.903 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T19:11:04.903 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T19:11:04.903 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T19:11:04.903 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T19:11:04.903 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T19:11:04.903 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T19:11:04.904 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T19:11:04.904 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T19:11:04.904 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T19:11:04.904 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T19:11:04.904 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T19:11:04.904 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T19:11:04.904 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T19:11:04.904 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T19:11:04.904 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T19:11:04.904 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T19:11:04.904 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T19:11:04.904 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T19:11:04.904 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T19:11:04.904 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T19:11:04.905 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T19:11:04.905 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T19:11:04.905 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T19:11:04.905 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T19:11:04.905 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T19:11:04.905 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T19:11:04.905 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T19:11:04.905 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T19:11:04.905 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T19:11:04.905 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T19:11:04.906 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T19:11:04.906 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T19:11:04.906 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T19:11:04.906 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T19:11:04.906 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T19:11:04.906 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T19:11:04.906 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T19:11:04.906 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T19:11:06.994 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T19:11:06.995 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:11:06.995 INFO:teuthology.orchestra.run.smithi076.stdout:Upgraded: 2024-04-25T19:11:06.995 INFO:teuthology.orchestra.run.smithi076.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:06.995 INFO:teuthology.orchestra.run.smithi076.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:06.995 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T19:11:06.995 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T19:11:06.995 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-04-25T19:11:06.995 INFO:teuthology.orchestra.run.smithi076.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T19:11:06.995 INFO:teuthology.orchestra.run.smithi076.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T19:11:06.995 INFO:teuthology.orchestra.run.smithi076.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T19:11:06.995 INFO:teuthology.orchestra.run.smithi076.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T19:11:06.995 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:06.996 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:06.996 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:06.996 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:06.996 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:11:06.996 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:06.996 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:06.996 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:06.996 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:11:06.996 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:11:06.996 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:11:06.996 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:11:06.996 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:11:06.996 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:06.996 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:06.997 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:11:06.997 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:06.997 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:06.997 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:06.997 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:11:06.997 INFO:teuthology.orchestra.run.smithi076.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:11:06.997 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:06.997 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:06.997 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:11:06.997 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T19:11:06.998 INFO:teuthology.orchestra.run.smithi076.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T19:11:06.998 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T19:11:06.998 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T19:11:06.998 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T19:11:06.998 INFO:teuthology.orchestra.run.smithi076.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T19:11:06.998 INFO:teuthology.orchestra.run.smithi076.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T19:11:06.998 INFO:teuthology.orchestra.run.smithi076.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T19:11:06.998 INFO:teuthology.orchestra.run.smithi076.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T19:11:06.998 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T19:11:06.998 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T19:11:06.998 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:06.998 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:06.998 INFO:teuthology.orchestra.run.smithi076.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:06.998 INFO:teuthology.orchestra.run.smithi076.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T19:11:06.999 INFO:teuthology.orchestra.run.smithi076.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T19:11:06.999 INFO:teuthology.orchestra.run.smithi076.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T19:11:06.999 INFO:teuthology.orchestra.run.smithi076.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T19:11:06.999 INFO:teuthology.orchestra.run.smithi076.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T19:11:06.999 INFO:teuthology.orchestra.run.smithi076.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:06.999 INFO:teuthology.orchestra.run.smithi076.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:06.999 INFO:teuthology.orchestra.run.smithi076.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T19:11:06.999 INFO:teuthology.orchestra.run.smithi076.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:06.999 INFO:teuthology.orchestra.run.smithi076.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T19:11:06.999 INFO:teuthology.orchestra.run.smithi076.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T19:11:07.000 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T19:11:07.000 INFO:teuthology.orchestra.run.smithi076.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T19:11:07.000 INFO:teuthology.orchestra.run.smithi076.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T19:11:07.000 INFO:teuthology.orchestra.run.smithi076.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T19:11:07.000 INFO:teuthology.orchestra.run.smithi076.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T19:11:07.000 INFO:teuthology.orchestra.run.smithi076.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T19:11:07.000 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T19:11:07.000 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T19:11:07.000 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T19:11:07.000 INFO:teuthology.orchestra.run.smithi076.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T19:11:07.000 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T19:11:07.000 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T19:11:07.000 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T19:11:07.001 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T19:11:07.001 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T19:11:07.001 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T19:11:07.001 INFO:teuthology.orchestra.run.smithi076.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T19:11:07.001 INFO:teuthology.orchestra.run.smithi076.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T19:11:07.001 INFO:teuthology.orchestra.run.smithi076.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T19:11:07.001 INFO:teuthology.orchestra.run.smithi076.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T19:11:07.001 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T19:11:07.001 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:07.001 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:07.001 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:07.002 INFO:teuthology.orchestra.run.smithi076.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T19:11:07.002 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T19:11:07.002 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T19:11:07.002 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T19:11:07.002 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T19:11:07.002 INFO:teuthology.orchestra.run.smithi076.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T19:11:07.002 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T19:11:07.002 INFO:teuthology.orchestra.run.smithi076.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T19:11:07.002 INFO:teuthology.orchestra.run.smithi076.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T19:11:07.002 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T19:11:07.002 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T19:11:07.002 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T19:11:07.003 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T19:11:07.003 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T19:11:07.003 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T19:11:07.003 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T19:11:07.003 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T19:11:07.003 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T19:11:07.003 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T19:11:07.003 INFO:teuthology.orchestra.run.smithi076.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T19:11:07.003 INFO:teuthology.orchestra.run.smithi076.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T19:11:07.003 INFO:teuthology.orchestra.run.smithi076.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T19:11:07.003 INFO:teuthology.orchestra.run.smithi076.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T19:11:07.003 INFO:teuthology.orchestra.run.smithi076.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T19:11:07.003 INFO:teuthology.orchestra.run.smithi076.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T19:11:07.004 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T19:11:07.004 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T19:11:07.004 INFO:teuthology.orchestra.run.smithi076.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T19:11:07.004 INFO:teuthology.orchestra.run.smithi076.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T19:11:07.004 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T19:11:07.004 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T19:11:07.004 INFO:teuthology.orchestra.run.smithi076.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T19:11:07.004 INFO:teuthology.orchestra.run.smithi076.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T19:11:07.004 INFO:teuthology.orchestra.run.smithi076.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T19:11:07.004 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T19:11:07.004 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T19:11:07.005 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T19:11:07.005 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T19:11:07.005 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T19:11:07.005 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T19:11:07.005 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:07.005 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:07.005 INFO:teuthology.orchestra.run.smithi076.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T19:11:07.005 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T19:11:07.005 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T19:11:07.005 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:07.005 INFO:teuthology.orchestra.run.smithi076.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T19:11:07.005 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T19:11:07.005 INFO:teuthology.orchestra.run.smithi076.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T19:11:07.005 INFO:teuthology.orchestra.run.smithi076.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T19:11:07.006 INFO:teuthology.orchestra.run.smithi076.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T19:11:07.006 INFO:teuthology.orchestra.run.smithi076.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T19:11:07.006 INFO:teuthology.orchestra.run.smithi076.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T19:11:07.006 INFO:teuthology.orchestra.run.smithi076.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T19:11:07.006 INFO:teuthology.orchestra.run.smithi076.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T19:11:07.006 INFO:teuthology.orchestra.run.smithi076.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T19:11:07.006 INFO:teuthology.orchestra.run.smithi076.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T19:11:07.006 INFO:teuthology.orchestra.run.smithi076.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T19:11:07.006 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T19:11:07.006 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T19:11:07.006 INFO:teuthology.orchestra.run.smithi076.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T19:11:07.006 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:07.007 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:07.007 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:11:07.007 INFO:teuthology.orchestra.run.smithi076.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T19:11:07.007 INFO:teuthology.orchestra.run.smithi076.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T19:11:07.007 INFO:teuthology.orchestra.run.smithi076.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T19:11:07.007 INFO:teuthology.orchestra.run.smithi076.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T19:11:07.007 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T19:11:07.007 INFO:teuthology.orchestra.run.smithi076.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T19:11:07.007 INFO:teuthology.orchestra.run.smithi076.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T19:11:07.008 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:11:07.008 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-04-25T19:11:07.405 DEBUG:teuthology.parallel:result is None 2024-04-25T19:11:14.962 INFO:teuthology.orchestra.run.smithi105.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 306 B/s | 19 kB 01:04 2024-04-25T19:11:15.295 INFO:teuthology.orchestra.run.smithi105.stdout:(106/152): libarrow-9.0.0-10.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-04-25T19:11:15.354 INFO:teuthology.orchestra.run.smithi105.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 446 kB/s | 26 kB 00:00 2024-04-25T19:11:15.412 INFO:teuthology.orchestra.run.smithi105.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 835 kB/s | 49 kB 00:00 2024-04-25T19:11:15.471 INFO:teuthology.orchestra.run.smithi105.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-25T19:11:15.538 INFO:teuthology.orchestra.run.smithi105.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-04-25T19:11:15.621 INFO:teuthology.orchestra.run.smithi105.stdout:(111/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.8 MB/s | 838 kB 00:00 2024-04-25T19:11:15.705 INFO:teuthology.orchestra.run.smithi105.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-04-25T19:11:15.772 INFO:teuthology.orchestra.run.smithi105.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 651 kB/s | 43 kB 00:00 2024-04-25T19:11:15.822 INFO:teuthology.orchestra.run.smithi105.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 646 kB/s | 32 kB 00:00 2024-04-25T19:11:15.872 INFO:teuthology.orchestra.run.smithi105.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-04-25T19:11:15.931 INFO:teuthology.orchestra.run.smithi105.stdout:(116/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-25T19:11:15.998 INFO:teuthology.orchestra.run.smithi105.stdout:(117/152): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-04-25T19:11:16.065 INFO:teuthology.orchestra.run.smithi105.stdout:(118/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-04-25T19:11:16.115 INFO:teuthology.orchestra.run.smithi105.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 706 kB/s | 35 kB 00:00 2024-04-25T19:11:16.182 INFO:teuthology.orchestra.run.smithi105.stdout:(120/152): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-04-25T19:11:16.290 INFO:teuthology.orchestra.run.smithi105.stdout:(121/152): python3-grpcio-1.46.7-10.el9.x86_64. 19 MB/s | 2.0 MB 00:00 2024-04-25T19:11:16.366 INFO:teuthology.orchestra.run.smithi105.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-25T19:11:16.424 INFO:teuthology.orchestra.run.smithi105.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 183 kB/s | 11 kB 00:00 2024-04-25T19:11:16.483 INFO:teuthology.orchestra.run.smithi105.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 303 kB/s | 18 kB 00:00 2024-04-25T19:11:16.542 INFO:teuthology.orchestra.run.smithi105.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 397 kB/s | 23 kB 00:00 2024-04-25T19:11:16.600 INFO:teuthology.orchestra.run.smithi105.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 333 kB/s | 19 kB 00:00 2024-04-25T19:11:16.659 INFO:teuthology.orchestra.run.smithi105.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 338 kB/s | 20 kB 00:00 2024-04-25T19:11:16.743 INFO:teuthology.orchestra.run.smithi105.stdout:(128/152): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-04-25T19:11:16.801 INFO:teuthology.orchestra.run.smithi105.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 795 kB/s | 46 kB 00:00 2024-04-25T19:11:16.868 INFO:teuthology.orchestra.run.smithi105.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-04-25T19:11:16.927 INFO:teuthology.orchestra.run.smithi105.stdout:(131/152): python3-natsort-7.1.1-5.el9.noarch.r 987 kB/s | 58 kB 00:00 2024-04-25T19:11:17.018 INFO:teuthology.orchestra.run.smithi105.stdout:(132/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-25T19:11:17.110 INFO:teuthology.orchestra.run.smithi105.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-04-25T19:11:17.219 INFO:teuthology.orchestra.run.smithi105.stdout:(134/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 829 kB/s | 90 kB 00:00 2024-04-25T19:11:17.294 INFO:teuthology.orchestra.run.smithi105.stdout:(135/152): python3-pyperclip-1.8.2-2.el9.noarch 377 kB/s | 28 kB 00:00 2024-04-25T19:11:17.420 INFO:teuthology.orchestra.run.smithi105.stdout:(136/152): python3-repoze-lru-0.7-16.el9.noarch 246 kB/s | 31 kB 00:00 2024-04-25T19:11:17.503 INFO:teuthology.orchestra.run.smithi105.stdout:(137/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-25T19:11:17.587 INFO:teuthology.orchestra.run.smithi105.stdout:(138/152): python3-rsa-4.9-2.el9.noarch.rpm 710 kB/s | 59 kB 00:00 2024-04-25T19:11:17.670 INFO:teuthology.orchestra.run.smithi105.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 429 kB/s | 36 kB 00:00 2024-04-25T19:11:17.737 INFO:teuthology.orchestra.run.smithi105.stdout:(140/152): python3-typing-extensions-4.4.0-2.el 765 kB/s | 51 kB 00:00 2024-04-25T19:11:17.804 INFO:teuthology.orchestra.run.smithi105.stdout:(141/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-04-25T19:11:17.879 INFO:teuthology.orchestra.run.smithi105.stdout:(142/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-25T19:11:17.955 INFO:teuthology.orchestra.run.smithi105.stdout:(143/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-04-25T19:11:18.039 INFO:teuthology.orchestra.run.smithi105.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 239 kB/s | 20 kB 00:00 2024-04-25T19:11:18.139 INFO:teuthology.orchestra.run.smithi105.stdout:(145/152): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-25T19:11:18.230 INFO:teuthology.orchestra.run.smithi105.stdout:(146/152): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-25T19:11:18.631 INFO:teuthology.orchestra.run.smithi105.stdout:(147/152): librados2-19.0.0-3291.g994e2f20.el9. 8.3 MB/s | 3.3 MB 00:00 2024-04-25T19:11:18.806 INFO:teuthology.orchestra.run.smithi105.stdout:(148/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 18 MB/s | 3.1 MB 00:00 2024-04-25T19:11:18.890 INFO:teuthology.orchestra.run.smithi105.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 16 MB/s | 1.3 MB 00:00 2024-04-25T19:11:19.015 INFO:teuthology.orchestra.run.smithi105.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-04-25T19:12:21.893 INFO:teuthology.orchestra.run.smithi105.stdout:[MIRROR] abseil-cpp-20211102.0-3.el9.x86_64.rpm: Curl error (28): Timeout was reached for http://epel.stl.us.ssimn.org/9/Everything/x86_64/Packages/a/abseil-cpp-20211102.0-3.el9.x86_64.rpm [Failed to connect to epel.stl.us.ssimn.org port 80: Connection timed out] 2024-04-25T19:12:21.893 INFO:teuthology.orchestra.run.smithi105.stdout:[MIRROR] gperftools-libs-2.9.1-2.el9.x86_64.rpm: Curl error (28): Timeout was reached for http://epel.stl.us.ssimn.org/9/Everything/x86_64/Packages/g/gperftools-libs-2.9.1-2.el9.x86_64.rpm [Failed to connect to epel.stl.us.ssimn.org port 80: Connection timed out] 2024-04-25T19:12:37.973 INFO:teuthology.orchestra.run.smithi105.stdout:(151/152): gperftools-libs-2.9.1-2.el9.x86_64.r 2.1 kB/s | 309 kB 02:27 2024-04-25T19:13:26.384 INFO:teuthology.orchestra.run.smithi105.stdout:(152/152): abseil-cpp-20211102.0-3.el9.x86_64.r 2.8 kB/s | 548 kB 03:16 2024-04-25T19:13:26.388 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2024-04-25T19:13:26.388 INFO:teuthology.orchestra.run.smithi105.stdout:Total 1.0 MB/s | 214 MB 03:28 2024-04-25T19:13:27.431 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2024-04-25T19:13:27.574 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2024-04-25T19:13:27.574 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2024-04-25T19:13:29.366 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2024-04-25T19:13:29.366 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2024-04-25T19:13:35.300 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2024-04-25T19:13:35.387 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T19:13:35.425 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T19:13:35.478 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T19:13:35.512 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T19:13:35.538 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T19:13:35.581 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T19:13:35.913 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T19:13:36.030 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T19:13:36.154 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T19:13:36.207 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T19:13:36.281 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T19:13:36.319 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T19:13:36.469 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T19:13:36.516 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T19:13:36.544 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T19:13:36.648 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T19:13:36.669 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T19:13:36.732 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T19:13:36.754 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T19:13:36.779 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T19:13:36.978 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T19:13:37.020 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T19:13:37.127 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T19:13:37.222 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T19:13:37.252 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T19:13:37.287 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T19:13:37.351 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T19:13:37.386 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T19:13:37.429 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T19:13:37.462 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T19:13:37.490 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T19:13:37.558 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T19:13:37.612 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T19:13:37.643 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T19:13:37.678 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T19:13:37.762 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T19:13:37.797 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T19:13:37.832 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T19:13:37.852 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T19:13:37.923 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T19:13:37.975 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T19:13:38.050 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T19:13:38.184 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T19:13:38.236 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T19:13:38.760 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T19:13:38.840 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T19:13:38.869 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T19:13:38.903 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T19:13:38.942 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T19:13:38.983 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T19:13:39.135 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T19:13:39.211 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T19:13:39.266 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T19:13:39.338 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T19:13:39.369 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T19:13:39.406 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T19:13:39.452 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T19:13:39.497 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T19:13:39.540 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T19:13:39.582 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T19:13:39.718 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T19:13:39.771 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T19:13:39.802 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T19:13:39.843 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T19:13:39.937 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T19:13:39.968 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T19:13:40.008 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T19:13:40.033 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T19:13:40.058 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T19:13:40.133 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T19:13:40.260 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T19:13:40.290 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T19:13:40.318 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T19:13:40.352 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T19:13:40.391 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T19:13:40.423 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T19:13:40.453 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T19:13:40.487 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T19:13:40.520 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T19:13:40.571 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T19:13:40.605 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T19:13:40.692 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T19:13:40.744 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T19:13:40.792 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T19:13:40.839 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T19:13:40.937 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T19:13:40.993 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T19:13:41.725 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T19:13:41.773 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T19:13:41.807 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T19:13:41.908 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T19:13:41.973 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T19:13:42.062 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T19:13:42.100 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T19:13:42.279 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T19:13:42.306 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T19:13:42.385 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T19:13:43.036 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T19:13:43.147 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T19:13:44.383 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T19:13:44.485 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T19:13:44.513 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T19:13:44.564 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T19:13:44.891 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T19:13:44.934 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T19:13:45.025 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T19:13:45.055 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T19:13:45.091 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T19:13:45.581 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T19:13:45.620 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T19:13:45.691 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T19:13:45.713 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T19:13:47.889 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T19:13:47.934 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T19:13:48.492 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T19:13:48.538 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T19:13:48.631 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T19:13:48.631 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T19:13:48.631 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T19:13:48.631 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:13:48.664 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T19:14:00.277 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T19:14:00.277 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /sys 2024-04-25T19:14:00.277 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /proc 2024-04-25T19:14:00.277 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /mnt 2024-04-25T19:14:00.278 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /var/tmp 2024-04-25T19:14:00.278 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /home 2024-04-25T19:14:00.278 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /root 2024-04-25T19:14:00.278 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /tmp 2024-04-25T19:14:00.278 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:00.453 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T19:14:00.523 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T19:14:00.523 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:14:00.523 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T19:14:00.523 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:02.054 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T19:14:02.079 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T19:14:02.079 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:14:02.079 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T19:14:02.079 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T19:14:02.079 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T19:14:02.079 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:02.296 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T19:14:02.318 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T19:14:02.318 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:14:02.318 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T19:14:02.318 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T19:14:02.318 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T19:14:02.319 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:02.780 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T19:14:02.802 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T19:14:02.803 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:14:02.803 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T19:14:02.803 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T19:14:02.803 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T19:14:02.803 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:02.910 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T19:14:02.959 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T19:14:03.014 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T19:14:03.199 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T19:14:03.247 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T19:14:03.317 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T19:14:03.406 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T19:14:03.554 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T19:14:03.597 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T19:14:03.630 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T19:14:03.666 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T19:14:04.092 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T19:14:04.114 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T19:14:04.146 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T19:14:04.200 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T19:14:04.229 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T19:14:05.260 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T19:14:05.297 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T19:14:06.252 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T19:14:06.281 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T19:14:06.382 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T19:14:06.539 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T19:14:06.575 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T19:14:06.597 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T19:14:06.597 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:14:06.597 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T19:14:06.597 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T19:14:06.597 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T19:14:06.597 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:06.642 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T19:14:06.653 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T19:14:06.751 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T19:14:06.799 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T19:14:07.728 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T19:14:07.769 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T19:14:07.791 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T19:14:07.791 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:14:07.791 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T19:14:07.791 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T19:14:07.791 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T19:14:07.792 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:07.837 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T19:14:07.858 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T19:14:07.858 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:14:07.858 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T19:14:07.858 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:07.906 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T19:14:07.926 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T19:14:07.926 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:14:07.926 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T19:14:07.926 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T19:14:07.926 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T19:14:07.926 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:08.230 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T19:14:08.252 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T19:14:08.252 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T19:14:08.252 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T19:14:08.252 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T19:14:08.252 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T19:14:08.252 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:12.501 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T19:14:12.541 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T19:14:12.582 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T19:14:12.614 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T19:14:12.641 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T19:14:12.827 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T19:14:12.879 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T19:14:12.909 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T19:14:13.517 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T19:14:13.573 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T19:14:13.592 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T19:14:14.424 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T19:14:14.511 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T19:14:14.553 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T19:14:14.634 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T19:14:14.665 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T19:14:14.665 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T19:14:14.749 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T19:14:14.782 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T19:14:14.782 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T19:14:14.878 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T19:14:14.957 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T19:14:21.037 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T19:14:21.037 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T19:14:21.038 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T19:14:21.038 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T19:14:21.038 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T19:14:21.038 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T19:14:21.038 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T19:14:21.038 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T19:14:21.038 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T19:14:21.038 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T19:14:21.038 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T19:14:21.038 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T19:14:21.038 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T19:14:21.038 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T19:14:21.039 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T19:14:21.039 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T19:14:21.039 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T19:14:21.039 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T19:14:21.039 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T19:14:21.039 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T19:14:21.039 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T19:14:21.039 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T19:14:21.039 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T19:14:21.039 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T19:14:21.039 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T19:14:21.039 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T19:14:21.039 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T19:14:21.040 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T19:14:21.040 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T19:14:21.040 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T19:14:21.040 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T19:14:21.040 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T19:14:21.040 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T19:14:21.040 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T19:14:21.040 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T19:14:21.040 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T19:14:21.040 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T19:14:21.040 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T19:14:21.040 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T19:14:21.040 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T19:14:21.040 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T19:14:21.041 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T19:14:21.041 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T19:14:21.041 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T19:14:21.041 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T19:14:21.041 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T19:14:21.041 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T19:14:21.041 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T19:14:21.041 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T19:14:21.041 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T19:14:21.041 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T19:14:21.041 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T19:14:21.041 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T19:14:21.041 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T19:14:21.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T19:14:21.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T19:14:21.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T19:14:21.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T19:14:21.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T19:14:21.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T19:14:21.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T19:14:21.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T19:14:21.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T19:14:21.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T19:14:21.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T19:14:21.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T19:14:21.043 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T19:14:21.043 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T19:14:21.043 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T19:14:21.043 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T19:14:21.043 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T19:14:21.043 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T19:14:21.043 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T19:14:21.043 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T19:14:21.043 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T19:14:21.043 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T19:14:21.043 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T19:14:21.043 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T19:14:21.044 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T19:14:21.044 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T19:14:21.044 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T19:14:21.044 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T19:14:21.044 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T19:14:21.044 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T19:14:21.044 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T19:14:21.044 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T19:14:21.044 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T19:14:21.044 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T19:14:21.044 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T19:14:21.044 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T19:14:21.044 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T19:14:21.045 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T19:14:21.045 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T19:14:21.045 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T19:14:21.045 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T19:14:21.045 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T19:14:21.045 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T19:14:21.045 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T19:14:21.045 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T19:14:21.045 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T19:14:21.045 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T19:14:21.045 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T19:14:21.045 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T19:14:21.045 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T19:14:21.046 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T19:14:21.046 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T19:14:21.046 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T19:14:21.046 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T19:14:21.046 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T19:14:21.046 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T19:14:21.046 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T19:14:21.046 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T19:14:21.047 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T19:14:21.047 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T19:14:21.047 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T19:14:21.047 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T19:14:21.047 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T19:14:21.047 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T19:14:21.047 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T19:14:21.047 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T19:14:21.047 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T19:14:21.047 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T19:14:21.047 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T19:14:21.047 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T19:14:21.047 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T19:14:21.047 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T19:14:21.048 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T19:14:21.048 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T19:14:21.048 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T19:14:21.048 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T19:14:21.048 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T19:14:21.048 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T19:14:21.048 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T19:14:21.048 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T19:14:21.048 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T19:14:21.048 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T19:14:21.048 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T19:14:21.048 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T19:14:21.048 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T19:14:21.049 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T19:14:21.049 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T19:14:21.049 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T19:14:21.049 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T19:14:21.049 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T19:14:21.049 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T19:14:21.049 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T19:14:21.049 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T19:14:21.049 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T19:14:21.049 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T19:14:21.049 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T19:14:21.049 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T19:14:21.049 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T19:14:21.049 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T19:14:21.050 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T19:14:21.050 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T19:14:21.050 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T19:14:23.163 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T19:14:23.163 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:23.164 INFO:teuthology.orchestra.run.smithi105.stdout:Upgraded: 2024-04-25T19:14:23.164 INFO:teuthology.orchestra.run.smithi105.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.164 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.164 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T19:14:23.164 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T19:14:23.164 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2024-04-25T19:14:23.164 INFO:teuthology.orchestra.run.smithi105.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T19:14:23.164 INFO:teuthology.orchestra.run.smithi105.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T19:14:23.164 INFO:teuthology.orchestra.run.smithi105.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T19:14:23.164 INFO:teuthology.orchestra.run.smithi105.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T19:14:23.164 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.164 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.164 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.164 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.165 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:14:23.165 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.165 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.165 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.165 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:14:23.165 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:14:23.165 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:14:23.165 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:14:23.165 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:14:23.165 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.165 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.165 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:14:23.165 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.166 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.166 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.166 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:14:23.166 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:14:23.166 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.166 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.166 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T19:14:23.166 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T19:14:23.166 INFO:teuthology.orchestra.run.smithi105.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T19:14:23.166 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T19:14:23.166 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T19:14:23.167 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T19:14:23.167 INFO:teuthology.orchestra.run.smithi105.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T19:14:23.167 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T19:14:23.167 INFO:teuthology.orchestra.run.smithi105.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T19:14:23.167 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T19:14:23.167 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T19:14:23.167 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T19:14:23.167 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.167 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.167 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.167 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T19:14:23.167 INFO:teuthology.orchestra.run.smithi105.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T19:14:23.167 INFO:teuthology.orchestra.run.smithi105.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T19:14:23.167 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T19:14:23.167 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T19:14:23.168 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.168 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.168 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T19:14:23.168 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.168 INFO:teuthology.orchestra.run.smithi105.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T19:14:23.168 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T19:14:23.168 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T19:14:23.168 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T19:14:23.168 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T19:14:23.168 INFO:teuthology.orchestra.run.smithi105.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T19:14:23.168 INFO:teuthology.orchestra.run.smithi105.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T19:14:23.168 INFO:teuthology.orchestra.run.smithi105.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T19:14:23.168 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T19:14:23.168 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T19:14:23.169 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T19:14:23.169 INFO:teuthology.orchestra.run.smithi105.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T19:14:23.169 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T19:14:23.169 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T19:14:23.169 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T19:14:23.169 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T19:14:23.169 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T19:14:23.169 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T19:14:23.169 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T19:14:23.169 INFO:teuthology.orchestra.run.smithi105.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T19:14:23.169 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T19:14:23.169 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T19:14:23.169 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T19:14:23.169 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.170 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.170 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.170 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T19:14:23.170 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T19:14:23.170 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T19:14:23.170 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T19:14:23.170 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T19:14:23.170 INFO:teuthology.orchestra.run.smithi105.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T19:14:23.170 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T19:14:23.170 INFO:teuthology.orchestra.run.smithi105.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T19:14:23.170 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T19:14:23.170 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T19:14:23.170 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T19:14:23.170 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T19:14:23.171 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T19:14:23.171 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T19:14:23.171 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T19:14:23.171 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T19:14:23.171 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T19:14:23.171 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T19:14:23.171 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T19:14:23.171 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T19:14:23.171 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T19:14:23.171 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T19:14:23.171 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T19:14:23.171 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T19:14:23.171 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T19:14:23.171 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T19:14:23.172 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T19:14:23.172 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T19:14:23.172 INFO:teuthology.orchestra.run.smithi105.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T19:14:23.172 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T19:14:23.172 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T19:14:23.172 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T19:14:23.172 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T19:14:23.172 INFO:teuthology.orchestra.run.smithi105.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T19:14:23.172 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T19:14:23.172 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T19:14:23.172 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T19:14:23.172 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T19:14:23.172 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T19:14:23.172 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T19:14:23.173 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.173 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.173 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T19:14:23.173 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T19:14:23.173 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T19:14:23.173 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.173 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T19:14:23.173 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T19:14:23.173 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T19:14:23.173 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T19:14:23.173 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T19:14:23.173 INFO:teuthology.orchestra.run.smithi105.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T19:14:23.173 INFO:teuthology.orchestra.run.smithi105.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T19:14:23.173 INFO:teuthology.orchestra.run.smithi105.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T19:14:23.174 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T19:14:23.174 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T19:14:23.174 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T19:14:23.174 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T19:14:23.174 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T19:14:23.174 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T19:14:23.174 INFO:teuthology.orchestra.run.smithi105.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T19:14:23.174 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.174 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.174 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T19:14:23.174 INFO:teuthology.orchestra.run.smithi105.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T19:14:23.175 INFO:teuthology.orchestra.run.smithi105.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T19:14:23.175 INFO:teuthology.orchestra.run.smithi105.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T19:14:23.175 INFO:teuthology.orchestra.run.smithi105.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T19:14:23.175 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T19:14:23.175 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T19:14:23.175 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T19:14:23.175 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:23.175 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2024-04-25T19:14:23.574 DEBUG:teuthology.parallel:result is None 2024-04-25T19:14:23.574 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T19:14:23.808 DEBUG:teuthology.orchestra.run.smithi076:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T19:14:23.834 INFO:teuthology.orchestra.run.smithi076.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T19:14:23.834 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T19:14:23.835 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T19:14:23.836 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T19:14:23.975 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T19:14:23.996 INFO:teuthology.orchestra.run.smithi105.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T19:14:23.997 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T19:14:23.997 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T19:14:23.999 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T19:14:24.000 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:14:24.000 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T19:14:24.029 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-25T19:14:24.029 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T19:14:24.063 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T19:14:24.063 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:14:24.064 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T19:14:24.096 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T19:14:24.162 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-25T19:14:24.162 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T19:14:24.185 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T19:14:24.247 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T19:14:24.248 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:14:24.248 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T19:14:24.274 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T19:14:24.339 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-25T19:14:24.339 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T19:14:24.362 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T19:14:24.426 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T19:14:24.426 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:14:24.426 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T19:14:24.452 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T19:14:24.517 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-25T19:14:24.517 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T19:14:24.540 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T19:14:24.603 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T19:14:24.689 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T19:14:24.690 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 777 /var/log/ceph 2024-04-25T19:14:24.692 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 777 /var/log/ceph 2024-04-25T19:14:24.715 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T19:14:24.715 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T19:14:24.756 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T19:14:24.779 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T19:14:24.779 DEBUG:teuthology.orchestra.run.smithi076:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T19:14:24.822 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T19:14:24.851 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T19:14:24.851 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T19:14:24.851 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T19:14:24.851 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:14:24.852 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T19:14:24.900 DEBUG:teuthology.orchestra.run.smithi076:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T19:14:25.008 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T19:14:25.045 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-25T19:14:25.046 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T19:14:25.060 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T19:14:25.168 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T19:14:25.212 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T19:14:25.212 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'overall HEALTH_', '\\(MON_DOWN\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T19:14:25.212 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673301', 'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/iozone}}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7673301', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240425.015853-debug'}, 'ceph': {'cephfs': {'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'overall HEALTH_', '\\(MON_DOWN\\)'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 85, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 1880, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'sleep_before_teardown': 0, 'subset': '178/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240425.015853-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi076.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAxDBH2a+eJqO9WYzP6gBhYbaBMx+CsbA0x+/qDLMUG+r0jSg1bwhDhzQG4L6CyVWNyaLtPKAOr5DrbnmDUSNqM=', 'smithi105.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBGv52NDt/WUrxy6zTqWRWDJEYvTHKMwbILe6dTAmaf/dTjfiFMYcSCLUEts0v1B1vhWyV8lNbTxvW/gL6lcZs0='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'overall HEALTH_', '\\(MON_DOWN\\)'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'mon_thrash': {'check_mds_failover': True, 'revive_delay': 20, 'thrash_delay': 10}}, {'workunit': {'clients': {'all': ['suites/iozone.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_10:00:23', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T19:14:25.213 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T19:14:25.215 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T19:14:25.261 DEBUG:teuthology.orchestra.run.smithi076:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T19:14:25.269 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T19:14:25.325 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:14:25.325 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T19:14:25.345 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T19:14:25.346 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_1 2024-04-25T19:14:25.400 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T19:14:25.400 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T19:14:25.400 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-25T19:14:25.400 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T19:14:25.400 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T19:14:25.400 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-04-25 19:10:59.244448762 +0000 2024-04-25T19:14:25.400 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-04-25 19:09:18.954577443 +0000 2024-04-25T19:14:25.400 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-04-25 19:09:18.954577443 +0000 2024-04-25T19:14:25.400 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-04-25 19:09:18.954577443 +0000 2024-04-25T19:14:25.400 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T19:14:25.462 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-04-25T19:14:25.463 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-04-25T19:14:25.463 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000326834 s, 1.6 MB/s 2024-04-25T19:14:25.464 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T19:14:25.520 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_2 2024-04-25T19:14:25.575 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T19:14:25.575 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T19:14:25.575 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-25T19:14:25.575 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T19:14:25.575 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T19:14:25.575 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-04-25 19:10:59.244448762 +0000 2024-04-25T19:14:25.575 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-04-25 19:09:19.286567152 +0000 2024-04-25T19:14:25.575 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-04-25 19:09:19.286567152 +0000 2024-04-25T19:14:25.575 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-04-25 19:09:19.286567152 +0000 2024-04-25T19:14:25.575 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T19:14:25.637 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-04-25T19:14:25.637 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-04-25T19:14:25.637 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000172393 s, 3.0 MB/s 2024-04-25T19:14:25.638 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T19:14:25.693 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_3 2024-04-25T19:14:25.747 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T19:14:25.747 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T19:14:25.747 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-25T19:14:25.747 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T19:14:25.748 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T19:14:25.748 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-04-25 19:10:59.244448762 +0000 2024-04-25T19:14:25.748 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-04-25 19:09:19.623556705 +0000 2024-04-25T19:14:25.748 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-04-25 19:09:19.623556705 +0000 2024-04-25T19:14:25.748 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-04-25 19:09:19.623556705 +0000 2024-04-25T19:14:25.748 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T19:14:25.809 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-04-25T19:14:25.809 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-04-25T19:14:25.809 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000208321 s, 2.5 MB/s 2024-04-25T19:14:25.810 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T19:14:25.865 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_4 2024-04-25T19:14:25.919 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T19:14:25.920 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T19:14:25.920 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T19:14:25.920 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T19:14:25.920 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T19:14:25.920 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-04-25 19:10:59.245448732 +0000 2024-04-25T19:14:25.920 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-04-25 19:09:19.973545855 +0000 2024-04-25T19:14:25.920 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-04-25 19:09:19.973545855 +0000 2024-04-25T19:14:25.920 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-04-25 19:09:19.973545855 +0000 2024-04-25T19:14:25.920 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T19:14:25.981 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-04-25T19:14:25.981 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-04-25T19:14:25.981 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000173008 s, 3.0 MB/s 2024-04-25T19:14:25.983 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T19:14:26.038 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T19:14:26.039 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-25T19:14:26.039 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T19:14:26.054 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T19:14:26.054 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_1 2024-04-25T19:14:26.109 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T19:14:26.109 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T19:14:26.109 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T19:14:26.110 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T19:14:26.110 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T19:14:26.110 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-04-25 19:14:15.352517905 +0000 2024-04-25T19:14:26.110 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-04-25 19:09:18.958229144 +0000 2024-04-25T19:14:26.110 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-04-25 19:09:18.958229144 +0000 2024-04-25T19:14:26.110 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-04-25 19:09:18.958229144 +0000 2024-04-25T19:14:26.110 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T19:14:26.172 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-04-25T19:14:26.172 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-04-25T19:14:26.172 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000268928 s, 1.9 MB/s 2024-04-25T19:14:26.173 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T19:14:26.230 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_2 2024-04-25T19:14:26.290 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T19:14:26.290 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T19:14:26.291 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T19:14:26.291 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T19:14:26.291 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T19:14:26.291 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-04-25 19:14:15.353517885 +0000 2024-04-25T19:14:26.291 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-04-25 19:09:19.284222344 +0000 2024-04-25T19:14:26.291 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-04-25 19:09:19.284222344 +0000 2024-04-25T19:14:26.291 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-04-25 19:09:19.284222344 +0000 2024-04-25T19:14:26.291 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T19:14:26.364 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-04-25T19:14:26.364 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-04-25T19:14:26.364 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000183716 s, 2.8 MB/s 2024-04-25T19:14:26.372 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T19:14:26.437 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_3 2024-04-25T19:14:26.498 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T19:14:26.498 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T19:14:26.498 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T19:14:26.499 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T19:14:26.499 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T19:14:26.499 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-04-25 19:14:15.353517885 +0000 2024-04-25T19:14:26.499 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-04-25 19:09:19.634215043 +0000 2024-04-25T19:14:26.499 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-04-25 19:09:19.634215043 +0000 2024-04-25T19:14:26.499 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-04-25 19:09:19.634215043 +0000 2024-04-25T19:14:26.499 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T19:14:26.566 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-04-25T19:14:26.566 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-04-25T19:14:26.566 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000188557 s, 2.7 MB/s 2024-04-25T19:14:26.567 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T19:14:26.630 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_4 2024-04-25T19:14:26.689 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T19:14:26.689 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T19:14:26.689 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T19:14:26.689 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T19:14:26.689 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T19:14:26.689 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-04-25 19:14:15.354517865 +0000 2024-04-25T19:14:26.689 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-04-25 19:09:19.954208368 +0000 2024-04-25T19:14:26.689 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-04-25 19:09:19.954208368 +0000 2024-04-25T19:14:26.689 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-04-25 19:09:19.954208368 +0000 2024-04-25T19:14:26.689 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T19:14:26.751 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-04-25T19:14:26.751 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-04-25T19:14:26.751 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000212828 s, 2.4 MB/s 2024-04-25T19:14:26.753 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T19:14:26.809 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T19:14:26.810 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi076.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi105.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T19:14:26.810 INFO:tasks.ceph:Generating config... 2024-04-25T19:14:26.814 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T19:14:26.814 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-04-25T19:14:26.814 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T19:14:26.815 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T19:14:26.815 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T19:14:26.815 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T19:14:26.815 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T19:14:26.815 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T19:14:26.815 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T19:14:26.815 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2024-04-25T19:14:26.815 INFO:tasks.ceph:[mds] mds bal merge size = 5 2024-04-25T19:14:26.815 INFO:tasks.ceph:[mds] mds bal split bits = 3 2024-04-25T19:14:26.815 INFO:tasks.ceph:[mds] mds bal split size = 100 2024-04-25T19:14:26.815 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T19:14:26.815 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T19:14:26.815 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = True 2024-04-25T19:14:26.815 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T19:14:26.815 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T19:14:26.816 INFO:tasks.ceph:[mds] mds_dir_prefetch = True 2024-04-25T19:14:26.816 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2024-04-25T19:14:26.816 INFO:tasks.ceph:[mds] mon client directed command retry = 5 2024-04-25T19:14:26.816 INFO:tasks.ceph:[mds] ms inject delay max = 1 2024-04-25T19:14:26.816 INFO:tasks.ceph:[mds] ms inject delay probability = 0.005 2024-04-25T19:14:26.816 INFO:tasks.ceph:[mds] ms inject delay type = client mds osd 2024-04-25T19:14:26.816 INFO:tasks.ceph:[mds] ms inject socket failures = 2500 2024-04-25T19:14:26.816 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T19:14:26.816 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T19:14:26.816 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T19:14:26.816 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-25T19:14:26.816 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T19:14:26.816 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T19:14:26.816 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T19:14:26.816 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T19:14:26.817 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T19:14:26.817 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-25T19:14:26.817 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T19:14:26.817 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T19:14:26.817 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-04-25T19:14:26.817 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T19:14:26.817 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-25T19:14:26.817 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-25T19:14:26.817 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T19:14:26.817 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T19:14:26.817 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-25T19:14:26.817 INFO:tasks.ceph:[osd] mon client directed command retry = 5 2024-04-25T19:14:26.817 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T19:14:26.817 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T19:14:26.817 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T19:14:26.818 INFO:tasks.ceph:[osd] ms inject delay max = 1 2024-04-25T19:14:26.818 INFO:tasks.ceph:[osd] ms inject delay probability = 0.005 2024-04-25T19:14:26.818 INFO:tasks.ceph:[osd] ms inject delay type = client mds 2024-04-25T19:14:26.818 INFO:tasks.ceph:[osd] ms inject socket failures = 2500 2024-04-25T19:14:26.818 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-04-25T19:14:26.818 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T19:14:26.818 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-04-25T19:14:26.818 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T19:14:26.818 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T19:14:26.818 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T19:14:26.819 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T19:14:26.892 INFO:teuthology.orchestra.run.smithi076.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T19:14:26.895 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T19:14:26.948 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T19:14:26.971 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.76'), ('mon.b', '172.21.15.105'), ('mon.c', '[v2:172.21.15.105:3301,v1:172.21.15.105:6790]')] 2024-04-25T19:14:26.972 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.76,172.21.15.105,[v2:172.21.15.105:3301,v1:172.21.15.105:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T19:14:26.973 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:14:26.973 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T19:14:27.027 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.76 --add b 172.21.15.105 --addv c '[v2:172.21.15.105:3301,v1:172.21.15.105:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:14:27.122 INFO:teuthology.orchestra.run.smithi076.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:14:27.122 INFO:teuthology.orchestra.run.smithi076.stdout:monmaptool: generated fsid d5adf865-03c7-4588-a151-044ededbff0b 2024-04-25T19:14:27.122 INFO:teuthology.orchestra.run.smithi076.stdout:setting min_mon_release = quincy 2024-04-25T19:14:27.123 INFO:teuthology.orchestra.run.smithi076.stdout:epoch 0 2024-04-25T19:14:27.123 INFO:teuthology.orchestra.run.smithi076.stdout:fsid d5adf865-03c7-4588-a151-044ededbff0b 2024-04-25T19:14:27.123 INFO:teuthology.orchestra.run.smithi076.stdout:last_changed 2024-04-25T19:14:27.121750+0000 2024-04-25T19:14:27.123 INFO:teuthology.orchestra.run.smithi076.stdout:created 2024-04-25T19:14:27.121750+0000 2024-04-25T19:14:27.123 INFO:teuthology.orchestra.run.smithi076.stdout:min_mon_release 17 (quincy) 2024-04-25T19:14:27.123 INFO:teuthology.orchestra.run.smithi076.stdout:election_strategy: 1 2024-04-25T19:14:27.123 INFO:teuthology.orchestra.run.smithi076.stdout:0: [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon.a 2024-04-25T19:14:27.123 INFO:teuthology.orchestra.run.smithi076.stdout:1: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.b 2024-04-25T19:14:27.123 INFO:teuthology.orchestra.run.smithi076.stdout:2: [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] mon.c 2024-04-25T19:14:27.123 INFO:teuthology.orchestra.run.smithi076.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T19:14:27.125 DEBUG:teuthology.orchestra.run.smithi076:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T19:14:27.139 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID d5adf865-03c7-4588-a151-044ededbff0b... 2024-04-25T19:14:27.140 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T19:14:27.182 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T19:14:27.219 INFO:teuthology.orchestra.run.smithi076.stdout:[global] 2024-04-25T19:14:27.219 INFO:teuthology.orchestra.run.smithi076.stdout: chdir = "" 2024-04-25T19:14:27.219 INFO:teuthology.orchestra.run.smithi076.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T19:14:27.220 INFO:teuthology.orchestra.run.smithi076.stdout: auth supported = cephx 2024-04-25T19:14:27.220 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.220 INFO:teuthology.orchestra.run.smithi076.stdout: filestore xattr use omap = true 2024-04-25T19:14:27.220 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.220 INFO:teuthology.orchestra.run.smithi076.stdout: mon clock drift allowed = 1.000 2024-04-25T19:14:27.220 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.220 INFO:teuthology.orchestra.run.smithi076.stdout: osd crush chooseleaf type = 0 2024-04-25T19:14:27.220 INFO:teuthology.orchestra.run.smithi076.stdout: auth debug = true 2024-04-25T19:14:27.220 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.220 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on old message = true 2024-04-25T19:14:27.220 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on bug = true 2024-04-25T19:14:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T19:14:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: mon pg warn max object skew = 0 2024-04-25T19:14:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T19:14:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T19:14:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: osd pool default size = 2 2024-04-25T19:14:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd allow primary affinity = true 2024-04-25T19:14:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd allow pg remap = true 2024-04-25T19:14:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on legacy crush tunables = false 2024-04-25T19:14:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on crush straw calc version zero = false 2024-04-25T19:14:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on no sortbitwise = false 2024-04-25T19:14:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on osd down out interval zero = false 2024-04-25T19:14:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on too few osds = false 2024-04-25T19:14:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T19:14:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T19:14:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: mon_allow_pool_size_one = true 2024-04-25T19:14:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T19:14:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: osd default data pool replay window = 5 2024-04-25T19:14:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: mon allow pool delete = true 2024-04-25T19:14:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: mon cluster log file level = debug 2024-04-25T19:14:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: debug asserts on shutdown = true 2024-04-25T19:14:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: mon health detail to clog = false 2024-04-25T19:14:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: mon host = "172.21.15.76,172.21.15.105,[v2:172.21.15.105:3301,v1:172.21.15.105:6790]" 2024-04-25T19:14:27.223 INFO:teuthology.orchestra.run.smithi076.stdout: fsid = d5adf865-03c7-4588-a151-044ededbff0b 2024-04-25T19:14:27.223 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.223 INFO:teuthology.orchestra.run.smithi076.stdout:[osd] 2024-04-25T19:14:27.223 INFO:teuthology.orchestra.run.smithi076.stdout: osd journal size = 100 2024-04-25T19:14:27.223 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.223 INFO:teuthology.orchestra.run.smithi076.stdout: osd scrub load threshold = 5.0 2024-04-25T19:14:27.223 INFO:teuthology.orchestra.run.smithi076.stdout: osd scrub max interval = 600 2024-04-25T19:14:27.223 INFO:teuthology.orchestra.run.smithi076.stdout: osd mclock profile = high_recovery_ops 2024-04-25T19:14:27.223 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.223 INFO:teuthology.orchestra.run.smithi076.stdout: osd recover clone overlap = true 2024-04-25T19:14:27.223 INFO:teuthology.orchestra.run.smithi076.stdout: osd recovery max chunk = 1048576 2024-04-25T19:14:27.223 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.223 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug shutdown = true 2024-04-25T19:14:27.223 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug op order = true 2024-04-25T19:14:27.223 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug verify stray on activate = true 2024-04-25T19:14:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug trim objects = true 2024-04-25T19:14:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: osd open classes on start = true 2024-04-25T19:14:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug pg log writeout = true 2024-04-25T19:14:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: osd deep scrub update digest min age = 30 2024-04-25T19:14:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: osd map max advance = 10 2024-04-25T19:14:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: journal zero on create = true 2024-04-25T19:14:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: filestore ondisk finisher threads = 3 2024-04-25T19:14:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: filestore apply finisher threads = 3 2024-04-25T19:14:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: bdev debug aio = true 2024-04-25T19:14:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug misdirected ops = true 2024-04-25T19:14:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore block size = 96636764160 2024-04-25T19:14:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore compression mode = aggressive 2024-04-25T19:14:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore fsck on mount = True 2024-04-25T19:14:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: debug bluefs = 20 2024-04-25T19:14:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: debug bluestore = 20 2024-04-25T19:14:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-25T19:14:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: debug osd = 20 2024-04-25T19:14:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: debug rocksdb = 10 2024-04-25T19:14:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: mon client directed command retry = 5 2024-04-25T19:14:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T19:14:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd full ratio = 0.9 2024-04-25T19:14:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd nearfull ratio = 0.8 2024-04-25T19:14:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: ms inject delay max = 1 2024-04-25T19:14:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: ms inject delay probability = 0.005 2024-04-25T19:14:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: ms inject delay type = client mds 2024-04-25T19:14:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: ms inject socket failures = 2500 2024-04-25T19:14:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: osd blocked scrub grace period = 3600 2024-04-25T19:14:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: osd failsafe full ratio = 0.95 2024-04-25T19:14:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: osd max markdown count = 1000 2024-04-25T19:14:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: osd objectstore = bluestore 2024-04-25T19:14:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: osd op complaint time = 180 2024-04-25T19:14:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.226 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr] 2024-04-25T19:14:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-25T19:14:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: debug mgr = 20 2024-04-25T19:14:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: debug mon = 20 2024-04-25T19:14:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: debug auth = 20 2024-04-25T19:14:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min pgs per osd = 4 2024-04-25T19:14:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min bytes per osd = 10 2024-04-25T19:14:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: mgr/telemetry/nag = false 2024-04-25T19:14:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: debug client = 20 2024-04-25T19:14:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.227 INFO:teuthology.orchestra.run.smithi076.stdout:[mon] 2024-04-25T19:14:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-25T19:14:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: debug mon = 20 2024-04-25T19:14:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: debug paxos = 20 2024-04-25T19:14:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: debug auth = 20 2024-04-25T19:14:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: mon data avail warn = 5 2024-04-25T19:14:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: mon mgr mkfs grace = 240 2024-04-25T19:14:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min pgs per osd = 4 2024-04-25T19:14:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd reporter subtree level = osd 2024-04-25T19:14:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd prime pg temp = true 2024-04-25T19:14:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min bytes per osd = 10 2024-04-25T19:14:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T19:14:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T19:14:27.228 INFO:teuthology.orchestra.run.smithi076.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T19:14:27.228 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.228 INFO:teuthology.orchestra.run.smithi076.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T19:14:27.228 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T19:14:27.228 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T19:14:27.228 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.228 INFO:teuthology.orchestra.run.smithi076.stdout: # 1m isn't quite enough 2024-04-25T19:14:27.228 INFO:teuthology.orchestra.run.smithi076.stdout: mon_down_mkfs_grace = 2m 2024-04-25T19:14:27.228 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.228 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_filestore_osds = false 2024-04-25T19:14:27.228 INFO:teuthology.orchestra.run.smithi076.stdout: mon down mkfs grace = 300 2024-04-25T19:14:27.228 INFO:teuthology.orchestra.run.smithi076.stdout: mon op complaint time = 120 2024-04-25T19:14:27.228 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:14:27.228 INFO:teuthology.orchestra.run.smithi076.stdout:[client] 2024-04-25T19:14:27.228 INFO:teuthology.orchestra.run.smithi076.stdout: rgw cache enabled = true 2024-04-25T19:14:27.229 INFO:teuthology.orchestra.run.smithi076.stdout: rgw enable ops log = true 2024-04-25T19:14:27.229 INFO:teuthology.orchestra.run.smithi076.stdout: rgw enable usage log = true 2024-04-25T19:14:27.229 INFO:teuthology.orchestra.run.smithi076.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T19:14:27.229 INFO:teuthology.orchestra.run.smithi076.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T19:14:27.229 INFO:teuthology.orchestra.run.smithi076.stdout: client mount timeout = 600 2024-04-25T19:14:27.229 INFO:teuthology.orchestra.run.smithi076.stdout: client_shutdown_timeout = 120 2024-04-25T19:14:27.229 INFO:teuthology.orchestra.run.smithi076.stdout: debug client = 20 2024-04-25T19:14:27.229 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-25T19:14:27.229 INFO:teuthology.orchestra.run.smithi076.stdout: rados mon op timeout = 900 2024-04-25T19:14:27.229 INFO:teuthology.orchestra.run.smithi076.stdout: rados osd op timeout = 900 2024-04-25T19:14:27.229 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.a] 2024-04-25T19:14:27.229 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.b] 2024-04-25T19:14:27.229 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.c] 2024-04-25T19:14:27.229 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.a] 2024-04-25T19:14:27.229 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.c] 2024-04-25T19:14:27.229 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.e] 2024-04-25T19:14:27.230 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.b] 2024-04-25T19:14:27.230 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.d] 2024-04-25T19:14:27.230 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.f] 2024-04-25T19:14:27.230 INFO:teuthology.orchestra.run.smithi076.stdout:[mds] 2024-04-25T19:14:27.230 INFO:teuthology.orchestra.run.smithi076.stdout: debug mds = 20 2024-04-25T19:14:27.230 INFO:teuthology.orchestra.run.smithi076.stdout: debug mds balancer = 20 2024-04-25T19:14:27.230 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-04-25T19:14:27.230 INFO:teuthology.orchestra.run.smithi076.stdout: mds bal fragment size max = 10000 2024-04-25T19:14:27.230 INFO:teuthology.orchestra.run.smithi076.stdout: mds bal merge size = 5 2024-04-25T19:14:27.230 INFO:teuthology.orchestra.run.smithi076.stdout: mds bal split bits = 3 2024-04-25T19:14:27.230 INFO:teuthology.orchestra.run.smithi076.stdout: mds bal split size = 100 2024-04-25T19:14:27.230 INFO:teuthology.orchestra.run.smithi076.stdout: mds debug frag = True 2024-04-25T19:14:27.230 INFO:teuthology.orchestra.run.smithi076.stdout: mds debug scatterstat = True 2024-04-25T19:14:27.230 INFO:teuthology.orchestra.run.smithi076.stdout: mds oft prefetch dirfrags = True 2024-04-25T19:14:27.230 INFO:teuthology.orchestra.run.smithi076.stdout: mds op complaint time = 180 2024-04-25T19:14:27.231 INFO:teuthology.orchestra.run.smithi076.stdout: mds verify scatter = True 2024-04-25T19:14:27.231 INFO:teuthology.orchestra.run.smithi076.stdout: mds_dir_prefetch = True 2024-04-25T19:14:27.231 INFO:teuthology.orchestra.run.smithi076.stdout: mds_sleep_rank_change = 5000000.0 2024-04-25T19:14:27.231 INFO:teuthology.orchestra.run.smithi076.stdout: mon client directed command retry = 5 2024-04-25T19:14:27.231 INFO:teuthology.orchestra.run.smithi076.stdout: ms inject delay max = 1 2024-04-25T19:14:27.231 INFO:teuthology.orchestra.run.smithi076.stdout: ms inject delay probability = 0.005 2024-04-25T19:14:27.231 INFO:teuthology.orchestra.run.smithi076.stdout: ms inject delay type = client mds osd 2024-04-25T19:14:27.231 INFO:teuthology.orchestra.run.smithi076.stdout: ms inject socket failures = 2500 2024-04-25T19:14:27.231 INFO:teuthology.orchestra.run.smithi076.stdout: osd op complaint time = 180 2024-04-25T19:14:27.231 INFO:teuthology.orchestra.run.smithi076.stdout: rados mon op timeout = 900 2024-04-25T19:14:27.231 INFO:teuthology.orchestra.run.smithi076.stdout: rados osd op timeout = 900 2024-04-25T19:14:27.238 INFO:teuthology.orchestra.run.smithi105.stdout:[global] 2024-04-25T19:14:27.238 INFO:teuthology.orchestra.run.smithi105.stdout: chdir = "" 2024-04-25T19:14:27.238 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T19:14:27.238 INFO:teuthology.orchestra.run.smithi105.stdout: auth supported = cephx 2024-04-25T19:14:27.238 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.238 INFO:teuthology.orchestra.run.smithi105.stdout: filestore xattr use omap = true 2024-04-25T19:14:27.238 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.238 INFO:teuthology.orchestra.run.smithi105.stdout: mon clock drift allowed = 1.000 2024-04-25T19:14:27.238 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.238 INFO:teuthology.orchestra.run.smithi105.stdout: osd crush chooseleaf type = 0 2024-04-25T19:14:27.238 INFO:teuthology.orchestra.run.smithi105.stdout: auth debug = true 2024-04-25T19:14:27.238 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.238 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on old message = true 2024-04-25T19:14:27.238 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on bug = true 2024-04-25T19:14:27.239 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.239 INFO:teuthology.orchestra.run.smithi105.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T19:14:27.239 INFO:teuthology.orchestra.run.smithi105.stdout: mon pg warn max object skew = 0 2024-04-25T19:14:27.239 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.239 INFO:teuthology.orchestra.run.smithi105.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T19:14:27.239 INFO:teuthology.orchestra.run.smithi105.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T19:14:27.239 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.239 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default size = 2 2024-04-25T19:14:27.239 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.239 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow primary affinity = true 2024-04-25T19:14:27.239 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow pg remap = true 2024-04-25T19:14:27.239 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on legacy crush tunables = false 2024-04-25T19:14:27.239 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on crush straw calc version zero = false 2024-04-25T19:14:27.239 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on no sortbitwise = false 2024-04-25T19:14:27.239 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on osd down out interval zero = false 2024-04-25T19:14:27.239 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on too few osds = false 2024-04-25T19:14:27.240 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T19:14:27.240 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T19:14:27.240 INFO:teuthology.orchestra.run.smithi105.stdout: mon_allow_pool_size_one = true 2024-04-25T19:14:27.240 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.240 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T19:14:27.240 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.240 INFO:teuthology.orchestra.run.smithi105.stdout: osd default data pool replay window = 5 2024-04-25T19:14:27.240 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.240 INFO:teuthology.orchestra.run.smithi105.stdout: mon allow pool delete = true 2024-04-25T19:14:27.240 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.240 INFO:teuthology.orchestra.run.smithi105.stdout: mon cluster log file level = debug 2024-04-25T19:14:27.240 INFO:teuthology.orchestra.run.smithi105.stdout: debug asserts on shutdown = true 2024-04-25T19:14:27.240 INFO:teuthology.orchestra.run.smithi105.stdout: mon health detail to clog = false 2024-04-25T19:14:27.240 INFO:teuthology.orchestra.run.smithi105.stdout: mon host = "172.21.15.76,172.21.15.105,[v2:172.21.15.105:3301,v1:172.21.15.105:6790]" 2024-04-25T19:14:27.240 INFO:teuthology.orchestra.run.smithi105.stdout: fsid = d5adf865-03c7-4588-a151-044ededbff0b 2024-04-25T19:14:27.241 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.241 INFO:teuthology.orchestra.run.smithi105.stdout:[osd] 2024-04-25T19:14:27.241 INFO:teuthology.orchestra.run.smithi105.stdout: osd journal size = 100 2024-04-25T19:14:27.241 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.241 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub load threshold = 5.0 2024-04-25T19:14:27.241 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub max interval = 600 2024-04-25T19:14:27.241 INFO:teuthology.orchestra.run.smithi105.stdout: osd mclock profile = high_recovery_ops 2024-04-25T19:14:27.241 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.241 INFO:teuthology.orchestra.run.smithi105.stdout: osd recover clone overlap = true 2024-04-25T19:14:27.241 INFO:teuthology.orchestra.run.smithi105.stdout: osd recovery max chunk = 1048576 2024-04-25T19:14:27.241 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.241 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug shutdown = true 2024-04-25T19:14:27.241 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug op order = true 2024-04-25T19:14:27.241 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug verify stray on activate = true 2024-04-25T19:14:27.241 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug trim objects = true 2024-04-25T19:14:27.242 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.242 INFO:teuthology.orchestra.run.smithi105.stdout: osd open classes on start = true 2024-04-25T19:14:27.242 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug pg log writeout = true 2024-04-25T19:14:27.242 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.242 INFO:teuthology.orchestra.run.smithi105.stdout: osd deep scrub update digest min age = 30 2024-04-25T19:14:27.242 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.242 INFO:teuthology.orchestra.run.smithi105.stdout: osd map max advance = 10 2024-04-25T19:14:27.242 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.242 INFO:teuthology.orchestra.run.smithi105.stdout: journal zero on create = true 2024-04-25T19:14:27.242 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.242 INFO:teuthology.orchestra.run.smithi105.stdout: filestore ondisk finisher threads = 3 2024-04-25T19:14:27.242 INFO:teuthology.orchestra.run.smithi105.stdout: filestore apply finisher threads = 3 2024-04-25T19:14:27.242 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.242 INFO:teuthology.orchestra.run.smithi105.stdout: bdev debug aio = true 2024-04-25T19:14:27.243 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug misdirected ops = true 2024-04-25T19:14:27.243 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore block size = 96636764160 2024-04-25T19:14:27.243 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore compression mode = aggressive 2024-04-25T19:14:27.243 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore fsck on mount = True 2024-04-25T19:14:27.243 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluefs = 20 2024-04-25T19:14:27.243 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluestore = 20 2024-04-25T19:14:27.243 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-04-25T19:14:27.243 INFO:teuthology.orchestra.run.smithi105.stdout: debug osd = 20 2024-04-25T19:14:27.243 INFO:teuthology.orchestra.run.smithi105.stdout: debug rocksdb = 10 2024-04-25T19:14:27.243 INFO:teuthology.orchestra.run.smithi105.stdout: mon client directed command retry = 5 2024-04-25T19:14:27.243 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T19:14:27.243 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd full ratio = 0.9 2024-04-25T19:14:27.243 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd nearfull ratio = 0.8 2024-04-25T19:14:27.243 INFO:teuthology.orchestra.run.smithi105.stdout: ms inject delay max = 1 2024-04-25T19:14:27.243 INFO:teuthology.orchestra.run.smithi105.stdout: ms inject delay probability = 0.005 2024-04-25T19:14:27.243 INFO:teuthology.orchestra.run.smithi105.stdout: ms inject delay type = client mds 2024-04-25T19:14:27.244 INFO:teuthology.orchestra.run.smithi105.stdout: ms inject socket failures = 2500 2024-04-25T19:14:27.244 INFO:teuthology.orchestra.run.smithi105.stdout: osd blocked scrub grace period = 3600 2024-04-25T19:14:27.244 INFO:teuthology.orchestra.run.smithi105.stdout: osd failsafe full ratio = 0.95 2024-04-25T19:14:27.244 INFO:teuthology.orchestra.run.smithi105.stdout: osd max markdown count = 1000 2024-04-25T19:14:27.244 INFO:teuthology.orchestra.run.smithi105.stdout: osd objectstore = bluestore 2024-04-25T19:14:27.244 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2024-04-25T19:14:27.244 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.244 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr] 2024-04-25T19:14:27.244 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-04-25T19:14:27.244 INFO:teuthology.orchestra.run.smithi105.stdout: debug mgr = 20 2024-04-25T19:14:27.244 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2024-04-25T19:14:27.244 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2024-04-25T19:14:27.244 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2024-04-25T19:14:27.244 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2024-04-25T19:14:27.244 INFO:teuthology.orchestra.run.smithi105.stdout: mgr/telemetry/nag = false 2024-04-25T19:14:27.244 INFO:teuthology.orchestra.run.smithi105.stdout: debug client = 20 2024-04-25T19:14:27.245 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.245 INFO:teuthology.orchestra.run.smithi105.stdout:[mon] 2024-04-25T19:14:27.245 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-04-25T19:14:27.245 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2024-04-25T19:14:27.245 INFO:teuthology.orchestra.run.smithi105.stdout: debug paxos = 20 2024-04-25T19:14:27.245 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2024-04-25T19:14:27.245 INFO:teuthology.orchestra.run.smithi105.stdout: mon data avail warn = 5 2024-04-25T19:14:27.245 INFO:teuthology.orchestra.run.smithi105.stdout: mon mgr mkfs grace = 240 2024-04-25T19:14:27.245 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2024-04-25T19:14:27.245 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd reporter subtree level = osd 2024-04-25T19:14:27.245 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd prime pg temp = true 2024-04-25T19:14:27.245 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2024-04-25T19:14:27.245 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.245 INFO:teuthology.orchestra.run.smithi105.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T19:14:27.245 INFO:teuthology.orchestra.run.smithi105.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T19:14:27.246 INFO:teuthology.orchestra.run.smithi105.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T19:14:27.246 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.246 INFO:teuthology.orchestra.run.smithi105.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T19:14:27.246 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T19:14:27.246 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T19:14:27.246 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.246 INFO:teuthology.orchestra.run.smithi105.stdout: # 1m isn't quite enough 2024-04-25T19:14:27.246 INFO:teuthology.orchestra.run.smithi105.stdout: mon_down_mkfs_grace = 2m 2024-04-25T19:14:27.246 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.246 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_filestore_osds = false 2024-04-25T19:14:27.246 INFO:teuthology.orchestra.run.smithi105.stdout: mon down mkfs grace = 300 2024-04-25T19:14:27.246 INFO:teuthology.orchestra.run.smithi105.stdout: mon op complaint time = 120 2024-04-25T19:14:27.246 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-25T19:14:27.246 INFO:teuthology.orchestra.run.smithi105.stdout:[client] 2024-04-25T19:14:27.247 INFO:teuthology.orchestra.run.smithi105.stdout: rgw cache enabled = true 2024-04-25T19:14:27.247 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable ops log = true 2024-04-25T19:14:27.247 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable usage log = true 2024-04-25T19:14:27.247 INFO:teuthology.orchestra.run.smithi105.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T19:14:27.247 INFO:teuthology.orchestra.run.smithi105.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T19:14:27.247 INFO:teuthology.orchestra.run.smithi105.stdout: client mount timeout = 600 2024-04-25T19:14:27.247 INFO:teuthology.orchestra.run.smithi105.stdout: client_shutdown_timeout = 120 2024-04-25T19:14:27.247 INFO:teuthology.orchestra.run.smithi105.stdout: debug client = 20 2024-04-25T19:14:27.247 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-04-25T19:14:27.247 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2024-04-25T19:14:27.247 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2024-04-25T19:14:27.247 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.a] 2024-04-25T19:14:27.247 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.b] 2024-04-25T19:14:27.247 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.c] 2024-04-25T19:14:27.247 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.a] 2024-04-25T19:14:27.247 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.c] 2024-04-25T19:14:27.248 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.e] 2024-04-25T19:14:27.248 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.b] 2024-04-25T19:14:27.248 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.d] 2024-04-25T19:14:27.248 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.f] 2024-04-25T19:14:27.248 INFO:teuthology.orchestra.run.smithi105.stdout:[mds] 2024-04-25T19:14:27.248 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds = 20 2024-04-25T19:14:27.248 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds balancer = 20 2024-04-25T19:14:27.248 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-04-25T19:14:27.248 INFO:teuthology.orchestra.run.smithi105.stdout: mds bal fragment size max = 10000 2024-04-25T19:14:27.248 INFO:teuthology.orchestra.run.smithi105.stdout: mds bal merge size = 5 2024-04-25T19:14:27.248 INFO:teuthology.orchestra.run.smithi105.stdout: mds bal split bits = 3 2024-04-25T19:14:27.248 INFO:teuthology.orchestra.run.smithi105.stdout: mds bal split size = 100 2024-04-25T19:14:27.248 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug frag = True 2024-04-25T19:14:27.248 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug scatterstat = True 2024-04-25T19:14:27.248 INFO:teuthology.orchestra.run.smithi105.stdout: mds oft prefetch dirfrags = True 2024-04-25T19:14:27.249 INFO:teuthology.orchestra.run.smithi105.stdout: mds op complaint time = 180 2024-04-25T19:14:27.249 INFO:teuthology.orchestra.run.smithi105.stdout: mds verify scatter = True 2024-04-25T19:14:27.249 INFO:teuthology.orchestra.run.smithi105.stdout: mds_dir_prefetch = True 2024-04-25T19:14:27.249 INFO:teuthology.orchestra.run.smithi105.stdout: mds_sleep_rank_change = 5000000.0 2024-04-25T19:14:27.249 INFO:teuthology.orchestra.run.smithi105.stdout: mon client directed command retry = 5 2024-04-25T19:14:27.249 INFO:teuthology.orchestra.run.smithi105.stdout: ms inject delay max = 1 2024-04-25T19:14:27.249 INFO:teuthology.orchestra.run.smithi105.stdout: ms inject delay probability = 0.005 2024-04-25T19:14:27.249 INFO:teuthology.orchestra.run.smithi105.stdout: ms inject delay type = client mds osd 2024-04-25T19:14:27.249 INFO:teuthology.orchestra.run.smithi105.stdout: ms inject socket failures = 2500 2024-04-25T19:14:27.249 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2024-04-25T19:14:27.249 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2024-04-25T19:14:27.249 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2024-04-25T19:14:27.250 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T19:14:27.250 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T19:14:27.336 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T19:14:27.336 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:14:27.336 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T19:14:27.350 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:14:27.351 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T19:14:27.405 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi076.front.sepia.ceph.com 2024-04-25T19:14:27.405 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:14:27.405 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T19:14:27.405 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T19:14:27.478 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:14:27.478 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:14:27.533 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi105.front.sepia.ceph.com 2024-04-25T19:14:27.533 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-25T19:14:27.533 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T19:14:27.533 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T19:14:27.566 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-25T19:14:27.566 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:14:27.621 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T19:14:27.621 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T19:14:27.622 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T19:14:27.679 INFO:teuthology.orchestra.run.smithi076.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T19:14:27.682 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T19:14:27.752 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T19:14:27.755 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T19:14:27.755 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T19:14:27.812 INFO:teuthology.orchestra.run.smithi076.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T19:14:27.815 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T19:14:27.838 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T19:14:27.935 INFO:teuthology.orchestra.run.smithi076.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T19:14:27.938 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T19:14:27.961 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-04-25T19:14:28.059 INFO:teuthology.orchestra.run.smithi076.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-04-25T19:14:28.062 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-04-25T19:14:28.085 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T19:14:28.143 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T19:14:28.146 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T19:14:28.169 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T19:14:28.266 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T19:14:28.269 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T19:14:28.292 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-04-25T19:14:28.389 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-04-25T19:14:28.392 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-04-25T19:14:28.415 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T19:14:28.415 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T19:14:28.467 INFO:teuthology.orchestra.run.smithi076.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T19:14:28.480 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T19:14:28.480 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi076.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi105.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T19:14:28.480 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T19:14:28.503 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T19:14:28.503 INFO:tasks.ceph:role: osd.0 2024-04-25T19:14:28.503 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi076.front.sepia.ceph.com 2024-04-25T19:14:28.504 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T19:14:28.574 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T19:14:28.574 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T19:14:28.574 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T19:14:28.574 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T19:14:28.574 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T19:14:28.574 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2024-04-25T19:14:28.574 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T19:14:28.574 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T19:14:28.574 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T19:14:28.574 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T19:14:29.075 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2024-04-25T19:14:29.077 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2024-04-25T19:14:29.077 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T19:14:29.190 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T19:14:29.221 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T19:14:29.290 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T19:14:29.290 INFO:tasks.ceph:role: osd.1 2024-04-25T19:14:29.290 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi076.front.sepia.ceph.com 2024-04-25T19:14:29.290 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T19:14:29.362 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T19:14:29.362 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T19:14:29.362 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T19:14:29.362 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T19:14:29.362 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T19:14:29.362 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2024-04-25T19:14:29.362 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T19:14:29.363 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T19:14:29.363 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T19:14:29.363 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T19:14:29.829 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2024-04-25T19:14:29.832 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2024-04-25T19:14:29.832 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T19:14:29.922 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T19:14:29.949 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T19:14:30.013 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T19:14:30.013 INFO:tasks.ceph:role: osd.2 2024-04-25T19:14:30.013 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi076.front.sepia.ceph.com 2024-04-25T19:14:30.013 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T19:14:30.086 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T19:14:30.087 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T19:14:30.087 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T19:14:30.087 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T19:14:30.087 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T19:14:30.087 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2024-04-25T19:14:30.087 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T19:14:30.087 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T19:14:30.087 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T19:14:30.087 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T19:14:30.579 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2024-04-25T19:14:30.581 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2024-04-25T19:14:30.581 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T19:14:30.713 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T19:14:30.740 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T19:14:30.804 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T19:14:30.804 INFO:tasks.ceph:role: osd.3 2024-04-25T19:14:30.804 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi076.front.sepia.ceph.com 2024-04-25T19:14:30.804 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T19:14:30.875 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T19:14:30.875 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T19:14:30.876 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T19:14:30.876 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T19:14:30.876 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T19:14:30.876 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2024-04-25T19:14:30.876 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T19:14:30.876 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T19:14:30.876 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T19:14:30.876 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T19:14:31.404 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2024-04-25T19:14:31.406 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2024-04-25T19:14:31.407 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T19:14:31.565 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T19:14:31.595 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:14:31.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:14:31.818+0000 7f615e29a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T19:14:31.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:14:31.818+0000 7f615e29a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T19:14:31.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:14:31.818+0000 7f615e29a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T19:14:31.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:14:31.818+0000 7f615e29a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T19:14:31.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:14:31.818+0000 7f615e29a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T19:14:31.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:14:31.819+0000 7f615e29a640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T19:14:34.683 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T19:14:34.709 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:14:34.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:14:34.806+0000 7fc8a5c10640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T19:14:34.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:14:34.806+0000 7fc8a5c10640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T19:14:34.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:14:34.806+0000 7fc8a5c10640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T19:14:34.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:14:34.806+0000 7fc8a5c10640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T19:14:34.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:14:34.806+0000 7fc8a5c10640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T19:14:34.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:14:34.807+0000 7fc8a5c10640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T19:14:37.663 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T19:14:37.691 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:14:37.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:14:37.793+0000 7fb3c81a7640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T19:14:37.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:14:37.793+0000 7fb3c81a7640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T19:14:37.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:14:37.793+0000 7fb3c81a7640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T19:14:37.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:14:37.793+0000 7fb3c81a7640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T19:14:37.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:14:37.793+0000 7fb3c81a7640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T19:14:37.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:14:37.794+0000 7fb3c81a7640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T19:14:40.657 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T19:14:40.683 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:14:40.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:14:40.775+0000 7fd28e63a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T19:14:40.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:14:40.776+0000 7fd28e63a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T19:14:40.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:14:40.776+0000 7fd28e63a640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T19:14:40.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:14:40.776+0000 7fd28e63a640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T19:14:40.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:14:40.776+0000 7fd28e63a640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T19:14:40.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:14:40.776+0000 7fd28e63a640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T19:14:43.629 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T19:14:43.657 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T19:14:43.685 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T19:14:43.685 INFO:tasks.ceph:role: osd.4 2024-04-25T19:14:43.686 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi105.front.sepia.ceph.com 2024-04-25T19:14:43.686 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T19:14:43.760 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T19:14:43.760 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T19:14:43.760 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T19:14:43.761 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T19:14:43.761 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T19:14:43.761 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-04-25T19:14:43.761 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T19:14:43.761 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T19:14:43.761 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T19:14:43.761 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T19:14:44.263 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-04-25T19:14:44.265 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-04-25T19:14:44.265 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T19:14:44.405 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T19:14:44.437 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T19:14:44.500 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T19:14:44.500 INFO:tasks.ceph:role: osd.5 2024-04-25T19:14:44.500 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi105.front.sepia.ceph.com 2024-04-25T19:14:44.500 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T19:14:44.570 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T19:14:44.570 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T19:14:44.570 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T19:14:44.571 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T19:14:44.571 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T19:14:44.571 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-04-25T19:14:44.571 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T19:14:44.571 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T19:14:44.571 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T19:14:44.571 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T19:14:45.026 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-04-25T19:14:45.028 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-04-25T19:14:45.028 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T19:14:45.115 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T19:14:45.143 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T19:14:45.206 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T19:14:45.207 INFO:tasks.ceph:role: osd.6 2024-04-25T19:14:45.207 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi105.front.sepia.ceph.com 2024-04-25T19:14:45.207 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T19:14:45.277 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T19:14:45.277 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T19:14:45.277 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T19:14:45.278 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T19:14:45.278 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T19:14:45.278 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-04-25T19:14:45.278 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T19:14:45.278 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T19:14:45.278 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T19:14:45.278 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T19:14:45.771 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-04-25T19:14:45.772 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-04-25T19:14:45.773 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T19:14:45.923 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T19:14:45.953 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T19:14:46.017 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T19:14:46.018 INFO:tasks.ceph:role: osd.7 2024-04-25T19:14:46.018 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi105.front.sepia.ceph.com 2024-04-25T19:14:46.018 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T19:14:46.090 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T19:14:46.090 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T19:14:46.090 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T19:14:46.091 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T19:14:46.091 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T19:14:46.091 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-04-25T19:14:46.091 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T19:14:46.091 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T19:14:46.091 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T19:14:46.091 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T19:14:46.599 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-04-25T19:14:46.602 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-04-25T19:14:46.602 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T19:14:46.748 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T19:14:46.781 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:14:46.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:14:46.882+0000 7f0b9ee25640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T19:14:46.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:14:46.883+0000 7f0b9ee25640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T19:14:46.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:14:46.883+0000 7f0b9ee25640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T19:14:46.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:14:46.883+0000 7f0b9ee25640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T19:14:46.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:14:46.883+0000 7f0b9ee25640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T19:14:46.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:14:46.883+0000 7f0b9ee25640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T19:14:49.733 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T19:14:49.761 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:14:49.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:14:49.858+0000 7f9e5e1dc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T19:14:49.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:14:49.859+0000 7f9e5e1dc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T19:14:49.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:14:49.859+0000 7f9e5e1dc640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T19:14:49.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:14:49.859+0000 7f9e5e1dc640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T19:14:49.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:14:49.859+0000 7f9e5e1dc640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T19:14:49.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:14:49.859+0000 7f9e5e1dc640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T19:14:52.705 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T19:14:52.730 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:14:52.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:14:52.825+0000 7f9f8b7b1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T19:14:52.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:14:52.825+0000 7f9f8b7b1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T19:14:52.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:14:52.825+0000 7f9f8b7b1640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T19:14:52.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:14:52.825+0000 7f9f8b7b1640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T19:14:52.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:14:52.825+0000 7f9f8b7b1640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T19:14:52.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:14:52.825+0000 7f9f8b7b1640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T19:14:55.214 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:14:55.216 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:14:55.686 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T19:14:55.710 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:14:55.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:14:55.801+0000 7ff194a15640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T19:14:55.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:14:55.801+0000 7ff194a15640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T19:14:55.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:14:55.801+0000 7ff194a15640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T19:14:55.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:14:55.801+0000 7ff194a15640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T19:14:55.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:14:55.801+0000 7ff194a15640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T19:14:55.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:14:55.802+0000 7ff194a15640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T19:14:58.651 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T19:14:58.678 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T19:14:58.678 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:14:58.678 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T19:14:58.706 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:14:58.706 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T19:14:58.774 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:14:58.775 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T19:14:58.842 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:14:58.842 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-04-25T19:14:58.910 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:14:58.910 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T19:14:58.977 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:14:58.977 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T19:14:59.043 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:14:59.043 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T19:14:59.108 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:14:59.109 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T19:14:59.173 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-25T19:14:59.173 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T19:14:59.201 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-25T19:14:59.201 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T19:14:59.268 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-25T19:14:59.268 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T19:14:59.335 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-25T19:14:59.335 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-04-25T19:14:59.400 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-25T19:14:59.400 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T19:14:59.464 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-25T19:14:59.464 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T19:14:59.529 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-25T19:14:59.529 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T19:14:59.592 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-25T19:14:59.593 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T19:14:59.656 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:14:59.656 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T19:14:59.674 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T19:14:59.674 DEBUG:teuthology.orchestra.run.smithi076:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T19:14:59.716 DEBUG:teuthology.orchestra.run.smithi105:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T19:14:59.738 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr.y] 2024-04-25T19:14:59.738 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCTqypmqqR0KBAAhTGFfl2NHov2dXJTk0mczA== 2024-04-25T19:14:59.739 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.a] 2024-04-25T19:14:59.739 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCTqypmattoMBAABxn5ulo6mM+HDjH7w9U43A== 2024-04-25T19:14:59.739 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.c] 2024-04-25T19:14:59.739 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCTqypmkBO5NxAA8y5SEq3e3cbfu6sj12JRsg== 2024-04-25T19:14:59.739 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.e] 2024-04-25T19:14:59.739 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCUqypmjad6AxAA75Dl3nrx60VuHXcTKPpooQ== 2024-04-25T19:14:59.739 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.0] 2024-04-25T19:14:59.739 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCXqypmLIbVMBAA3l1+r2Npv7n+fklf8XOilQ== 2024-04-25T19:14:59.739 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.1] 2024-04-25T19:14:59.739 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCaqypm2IkdMBAAZ3OpqoRuNVLjIZ/nzHwxpg== 2024-04-25T19:14:59.739 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.2] 2024-04-25T19:14:59.739 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCdqypmcL1XLxAAfhEAPois2h9RsZNuVIeBGQ== 2024-04-25T19:14:59.739 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.3] 2024-04-25T19:14:59.739 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCgqypmUzlZLhAAElaeEN7ExufuLQcnNmceMw== 2024-04-25T19:14:59.739 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr.x] 2024-04-25T19:14:59.740 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCTqypmhhXLLBAABI5b++Qkx9+qQiaoTXw/bQ== 2024-04-25T19:14:59.740 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.b] 2024-04-25T19:14:59.740 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCUqypmcyaICBAAaPdfCxcI1N48YJgD1Sr+mg== 2024-04-25T19:14:59.740 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.d] 2024-04-25T19:14:59.740 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCUqypmfKXaDxAACna5YJUC0j2+AXk7hppifA== 2024-04-25T19:14:59.740 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.f] 2024-04-25T19:14:59.740 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCUqypmXycwFxAAZU1mgVMCPoJhQEtV95HEuw== 2024-04-25T19:14:59.740 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.4] 2024-04-25T19:14:59.740 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCmqypmquWuNBAA+pIRnOAsKAtjStAVSEhOJg== 2024-04-25T19:14:59.740 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.5] 2024-04-25T19:14:59.740 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCpqypm5mlAMxAAoG0vvT3TUyJ9w2G3l5Uweg== 2024-04-25T19:14:59.740 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.6] 2024-04-25T19:14:59.740 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCsqypmmOo8MRAA2OzFqhuwouvKFOoaJNm1yQ== 2024-04-25T19:14:59.740 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.7] 2024-04-25T19:14:59.740 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCvqypm/HfSLxAAY01oET1Jr2/CuKrWgDE0Lw== 2024-04-25T19:14:59.740 INFO:teuthology.orchestra.run.smithi076.stdout:[client.0] 2024-04-25T19:14:59.741 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCUqypm2LjQGxAAa2pr+9G1TGiPRGinysvu/A== 2024-04-25T19:14:59.742 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr.y] 2024-04-25T19:14:59.742 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCTqypmqqR0KBAAhTGFfl2NHov2dXJTk0mczA== 2024-04-25T19:14:59.742 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.a] 2024-04-25T19:14:59.742 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCTqypmattoMBAABxn5ulo6mM+HDjH7w9U43A== 2024-04-25T19:14:59.742 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.c] 2024-04-25T19:14:59.742 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCTqypmkBO5NxAA8y5SEq3e3cbfu6sj12JRsg== 2024-04-25T19:14:59.742 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.e] 2024-04-25T19:14:59.742 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCUqypmjad6AxAA75Dl3nrx60VuHXcTKPpooQ== 2024-04-25T19:14:59.743 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.0] 2024-04-25T19:14:59.743 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCXqypmLIbVMBAA3l1+r2Npv7n+fklf8XOilQ== 2024-04-25T19:14:59.743 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.1] 2024-04-25T19:14:59.743 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCaqypm2IkdMBAAZ3OpqoRuNVLjIZ/nzHwxpg== 2024-04-25T19:14:59.743 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.2] 2024-04-25T19:14:59.743 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCdqypmcL1XLxAAfhEAPois2h9RsZNuVIeBGQ== 2024-04-25T19:14:59.743 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.3] 2024-04-25T19:14:59.743 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCgqypmUzlZLhAAElaeEN7ExufuLQcnNmceMw== 2024-04-25T19:14:59.743 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr.x] 2024-04-25T19:14:59.743 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCTqypmhhXLLBAABI5b++Qkx9+qQiaoTXw/bQ== 2024-04-25T19:14:59.743 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.b] 2024-04-25T19:14:59.743 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCUqypmcyaICBAAaPdfCxcI1N48YJgD1Sr+mg== 2024-04-25T19:14:59.743 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.d] 2024-04-25T19:14:59.743 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCUqypmfKXaDxAACna5YJUC0j2+AXk7hppifA== 2024-04-25T19:14:59.743 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.f] 2024-04-25T19:14:59.743 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCUqypmXycwFxAAZU1mgVMCPoJhQEtV95HEuw== 2024-04-25T19:14:59.744 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.4] 2024-04-25T19:14:59.744 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCmqypmquWuNBAA+pIRnOAsKAtjStAVSEhOJg== 2024-04-25T19:14:59.744 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.5] 2024-04-25T19:14:59.744 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCpqypm5mlAMxAAoG0vvT3TUyJ9w2G3l5Uweg== 2024-04-25T19:14:59.744 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.6] 2024-04-25T19:14:59.744 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCsqypmmOo8MRAA2OzFqhuwouvKFOoaJNm1yQ== 2024-04-25T19:14:59.744 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.7] 2024-04-25T19:14:59.744 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCvqypm/HfSLxAAY01oET1Jr2/CuKrWgDE0Lw== 2024-04-25T19:14:59.744 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2024-04-25T19:14:59.744 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQCUqypm2LjQGxAAa2pr+9G1TGiPRGinysvu/A== 2024-04-25T19:14:59.744 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T19:14:59.785 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T19:14:59.846 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T19:14:59.848 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T19:14:59.908 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T19:14:59.910 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T19:14:59.972 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T19:14:59.974 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T19:15:00.035 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:15:00.038 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:15:00.096 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:15:00.098 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:15:00.157 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:15:00.160 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:15:00.218 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:15:00.220 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:15:00.279 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T19:15:00.281 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T19:15:00.340 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T19:15:00.342 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T19:15:00.403 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T19:15:00.406 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T19:15:00.467 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T19:15:00.469 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T19:15:00.530 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:15:00.532 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:15:00.590 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:15:00.593 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:15:00.651 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:15:00.653 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:15:00.712 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:15:00.714 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T19:15:00.773 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T19:15:00.775 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T19:15:00.836 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T19:15:00.837 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T19:15:00.860 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T19:15:00.965 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T19:15:00.988 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T19:15:01.012 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T19:15:01.117 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T19:15:01.141 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T19:15:01.204 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T19:15:01.310 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T19:15:01.334 DEBUG:teuthology.orchestra.run.smithi076:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:15:01.336 DEBUG:teuthology.orchestra.run.smithi105:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T19:15:01.388 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T19:15:01.388 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T19:15:01.389 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T19:15:01.393 INFO:tasks.ceph.mon.a:Started 2024-04-25T19:15:01.393 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T19:15:01.393 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T19:15:01.431 INFO:tasks.ceph.mon.b:Started 2024-04-25T19:15:01.431 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T19:15:01.431 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T19:15:01.433 INFO:tasks.ceph.mon.c:Started 2024-04-25T19:15:01.433 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T19:15:01.434 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T19:15:01.434 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T19:15:01.436 INFO:tasks.ceph.mgr.y:Started 2024-04-25T19:15:01.436 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T19:15:01.436 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T19:15:01.438 INFO:tasks.ceph.mgr.x:Started 2024-04-25T19:15:01.439 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T19:15:01.439 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T19:15:01.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.547+0000 7f1f2c582640 1 Processor -- start 2024-04-25T19:15:01.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.547+0000 7f1f2c582640 1 -- start start 2024-04-25T19:15:01.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.548+0000 7f1f2c582640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1f2415a9f0 0x7f1f240c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:01.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.548+0000 7f1f2c582640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1f2415df30 0x7f1f24074130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:01.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.548+0000 7f1f2c582640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f2415ae00 con 0x7f1f24154ee0 2024-04-25T19:15:01.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.548+0000 7f1f2c582640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f1f24068bf0 con 0x7f1f2415df30 2024-04-25T19:15:01.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.548+0000 7f1f2c582640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f1f240c8e60 con 0x7f1f2415a9f0 2024-04-25T19:15:01.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.548+0000 7f1f2ab1f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1f2415df30 0x7f1f24074130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:01.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.548+0000 7f1f2ab1f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1f2415df30 0x7f1f24074130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43728/0 (socket says 172.21.15.76:43728) 2024-04-25T19:15:01.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.548+0000 7f1f2a31e640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f1f24154ee0 0x7f1f240684e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45442/0 (socket says 172.21.15.76:45442) 2024-04-25T19:15:01.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.548+0000 7f1f2ab1f640 1 -- 172.21.15.76:0/2570883198 learned_addr learned my addr 172.21.15.76:0/2570883198 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:01.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.548+0000 7f1f2ab1f640 1 -- 172.21.15.76:0/2570883198 >> v2:172.21.15.76:3300/0 conn(0x7f1f2415df30 msgr2=0x7f1f24074130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T19:15:01.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.548+0000 7f1f2ab1f640 1 -- 172.21.15.76:0/2570883198 >> v2:172.21.15.76:3300/0 conn(0x7f1f2415df30 msgr2=0x7f1f24074130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:15:01.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.548+0000 7f1f2ab1f640 1 --2- 172.21.15.76:0/2570883198 >> v2:172.21.15.76:3300/0 conn(0x7f1f2415df30 0x7f1f24074130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:15:01.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.549+0000 7f1f29b1d640 1 --2- 172.21.15.76:0/2570883198 >> v2:172.21.15.105:3300/0 conn(0x7f1f2415a9f0 0x7f1f240c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:01.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.549+0000 7f1f2ab1f640 1 --2- 172.21.15.76:0/2570883198 >> v2:172.21.15.76:3300/0 conn(0x7f1f2415df30 0x7f1f24074130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:15:01.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.549+0000 7f1f29b1d640 1 -- 172.21.15.76:0/2570883198 >> v2:172.21.15.105:3300/0 conn(0x7f1f2415a9f0 msgr2=0x7f1f240c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T19:15:01.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.549+0000 7f1f29b1d640 1 -- 172.21.15.76:0/2570883198 >> v2:172.21.15.105:3300/0 conn(0x7f1f2415a9f0 msgr2=0x7f1f240c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:15:01.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.549+0000 7f1f29b1d640 1 --2- 172.21.15.76:0/2570883198 >> v2:172.21.15.105:3300/0 conn(0x7f1f2415a9f0 0x7f1f240c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:15:01.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.549+0000 7f1f29b1d640 1 --2- 172.21.15.76:0/2570883198 >> v2:172.21.15.105:3300/0 conn(0x7f1f2415a9f0 0x7f1f240c98b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:15:01.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.705+0000 7f1f2931c640 1 -- 172.21.15.76:0/2570883198 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1379410826 0 0) 0x7f1f20001110 con 0x7f1f24154ee0 2024-04-25T19:15:01.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.705+0000 7f1f2931c640 1 -- 172.21.15.76:0/2570883198 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f10003520 con 0x7f1f24154ee0 2024-04-25T19:15:01.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.705+0000 7f1f2931c640 1 -- 172.21.15.76:0/2570883198 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2131089354 0 0) 0x7f1f10003520 con 0x7f1f24154ee0 2024-04-25T19:15:01.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.705+0000 7f1f2931c640 1 -- 172.21.15.76:0/2570883198 >> v2:172.21.15.105:3300/0 conn(0x7f1f2415a9f0 msgr2=0x7f1f240c98b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:15:01.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.705+0000 7f1f2931c640 1 --2- 172.21.15.76:0/2570883198 >> v2:172.21.15.105:3300/0 conn(0x7f1f2415a9f0 0x7f1f240c98b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:01.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.705+0000 7f1f2931c640 1 -- 172.21.15.76:0/2570883198 >> v2:172.21.15.76:3300/0 conn(0x7f1f2415df30 msgr2=0x7f1f24074130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:15:01.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.705+0000 7f1f2931c640 1 --2- 172.21.15.76:0/2570883198 >> v2:172.21.15.76:3300/0 conn(0x7f1f2415df30 0x7f1f24074130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:01.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.705+0000 7f1f2931c640 1 -- 172.21.15.76:0/2570883198 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f2415e6c0 con 0x7f1f24154ee0 2024-04-25T19:15:01.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.705+0000 7f1f2931c640 1 -- 172.21.15.76:0/2570883198 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f1f20001840 con 0x7f1f24154ee0 2024-04-25T19:15:01.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.705+0000 7f1f2931c640 1 -- 172.21.15.76:0/2570883198 >> v1:172.21.15.76:6789/0 conn(0x7f1f24154ee0 legacy=0x7f1f240684e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:01.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.705+0000 7f1f2931c640 1 --2- 172.21.15.76:0/2570883198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f100039b0 0x7f1f10003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:01.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.705+0000 7f1f2931c640 1 --2- 172.21.15.76:0/2570883198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f10004800 0x7f1f10004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:01.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.705+0000 7f1f2931c640 1 --2- 172.21.15.76:0/2570883198 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f100051a0 0x7f1f1000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:01.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.705+0000 7f1f2931c640 1 -- 172.21.15.76:0/2570883198 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1f24068bf0 con 0x7f1f100039b0 2024-04-25T19:15:01.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.705+0000 7f1f2931c640 1 -- 172.21.15.76:0/2570883198 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1f240c8e60 con 0x7f1f10004800 2024-04-25T19:15:01.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.705+0000 7f1f2931c640 1 -- 172.21.15.76:0/2570883198 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f1f1000c1a0 con 0x7f1f100051a0 2024-04-25T19:15:01.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.705+0000 7f1f2a31e640 1 --2- 172.21.15.76:0/2570883198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f100039b0 0x7f1f10003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:01.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.705+0000 7f1f29b1d640 1 --2- 172.21.15.76:0/2570883198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f10004800 0x7f1f10004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:01.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.705+0000 7f1f2ab1f640 1 --2- 172.21.15.76:0/2570883198 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f100051a0 0x7f1f1000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:01.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.706+0000 7f1f2a31e640 1 -- 172.21.15.76:0/2570883198 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f100051a0 msgr2=0x7f1f1000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:01.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.706+0000 7f1f2a31e640 1 --2- 172.21.15.76:0/2570883198 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f100051a0 0x7f1f1000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:01.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.706+0000 7f1f2a31e640 1 -- 172.21.15.76:0/2570883198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f10004800 msgr2=0x7f1f10004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:01.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.706+0000 7f1f2a31e640 1 --2- 172.21.15.76:0/2570883198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f10004800 0x7f1f10004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:01.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.706+0000 7f1f2a31e640 1 -- 172.21.15.76:0/2570883198 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f10003520 con 0x7f1f100039b0 2024-04-25T19:15:01.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.706+0000 7f1f2ab1f640 1 -- 172.21.15.76:0/2570883198 reap_dead start 2024-04-25T19:15:01.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.706+0000 7f1f2a31e640 1 --2- 172.21.15.76:0/2570883198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f100039b0 0x7f1f10003e20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000c2f0 tx=0x7f1f10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc2a1a1defb6c9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:01.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.706+0000 7f1f2931c640 1 -- 172.21.15.76:0/2570883198 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f20036070 con 0x7f1f100039b0 2024-04-25T19:15:01.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.706+0000 7f1f2931c640 1 -- 172.21.15.76:0/2570883198 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f20005a10 con 0x7f1f100039b0 2024-04-25T19:15:01.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.706+0000 7f1f2931c640 1 -- 172.21.15.76:0/2570883198 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f20007e60 con 0x7f1f100039b0 2024-04-25T19:15:01.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.707+0000 7f1f2c582640 1 -- 172.21.15.76:0/2570883198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f100039b0 msgr2=0x7f1f10003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:01.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.707+0000 7f1f2c582640 1 --2- 172.21.15.76:0/2570883198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f100039b0 0x7f1f10003e20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000c2f0 tx=0x7f1f10005830 comp rx=0 tx=0).stop 2024-04-25T19:15:01.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.707+0000 7f1f2c582640 1 -- 172.21.15.76:0/2570883198 shutdown_connections 2024-04-25T19:15:01.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.707+0000 7f1f2c582640 1 --2- 172.21.15.76:0/2570883198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f100039b0 0x7f1f10003e20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:01.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.707+0000 7f1f2c582640 1 -- 172.21.15.76:0/2570883198 >> 172.21.15.76:0/2570883198 conn(0x7f1f2406c5c0 msgr2=0x7f1f2406c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:01.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.707+0000 7f1f2c582640 1 -- 172.21.15.76:0/2570883198 shutdown_connections 2024-04-25T19:15:01.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.707+0000 7f1f2c582640 1 -- 172.21.15.76:0/2570883198 wait complete. 2024-04-25T19:15:01.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.709+0000 7f1f2c582640 1 Processor -- start 2024-04-25T19:15:01.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.709+0000 7f1f2c582640 1 -- start start 2024-04-25T19:15:01.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.710+0000 7f1f2c582640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f100039b0 0x7f1f24150e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:01.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.710+0000 7f1f2c582640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f24152160 0x7f1f240ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:01.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.710+0000 7f1f2c582640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f241526b0 0x7f1f2414acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:01.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.710+0000 7f1f2c582640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1f240c90b0 con 0x7f1f241526b0 2024-04-25T19:15:01.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.710+0000 7f1f2c582640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2415ec80 con 0x7f1f24152160 2024-04-25T19:15:01.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.710+0000 7f1f2c582640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f1f240c9390 con 0x7f1f100039b0 2024-04-25T19:15:01.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.710+0000 7f1f2ab1f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f241526b0 0x7f1f2414acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:01.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.710+0000 7f1f2ab1f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f241526b0 0x7f1f2414acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43748/0 (socket says 172.21.15.76:43748) 2024-04-25T19:15:01.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.710+0000 7f1f2ab1f640 1 -- 172.21.15.76:0/1897884713 learned_addr learned my addr 172.21.15.76:0/1897884713 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:01.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.711+0000 7f1f2a31e640 1 --2- 172.21.15.76:0/1897884713 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f100039b0 0x7f1f24150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:01.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.711+0000 7f1f29b1d640 1 --2- 172.21.15.76:0/1897884713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f24152160 0x7f1f240ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:01.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.711+0000 7f1f2ab1f640 1 -- 172.21.15.76:0/1897884713 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f100039b0 msgr2=0x7f1f24150e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:01.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.711+0000 7f1f2ab1f640 1 --2- 172.21.15.76:0/1897884713 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f100039b0 0x7f1f24150e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:01.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.711+0000 7f1f2ab1f640 1 -- 172.21.15.76:0/1897884713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f24152160 msgr2=0x7f1f240ca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:01.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.711+0000 7f1f2ab1f640 1 --2- 172.21.15.76:0/1897884713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f24152160 0x7f1f240ca310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:01.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.711+0000 7f1f2ab1f640 1 -- 172.21.15.76:0/1897884713 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f24151390 con 0x7f1f241526b0 2024-04-25T19:15:01.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.711+0000 7f1f2a31e640 1 --2- 172.21.15.76:0/1897884713 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f100039b0 0x7f1f24150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:01.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.711+0000 7f1f29b1d640 1 --2- 172.21.15.76:0/1897884713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f24152160 0x7f1f240ca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:01.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.711+0000 7f1f2ab1f640 1 --2- 172.21.15.76:0/1897884713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f241526b0 0x7f1f2414acd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0096d0 tx=0x7f1f1c007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:01.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.711+0000 7f1f1affd640 1 -- 172.21.15.76:0/1897884713 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c01b070 con 0x7f1f241526b0 2024-04-25T19:15:01.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.711+0000 7f1f1affd640 1 -- 172.21.15.76:0/1897884713 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f1c00fd60 con 0x7f1f241526b0 2024-04-25T19:15:01.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.711+0000 7f1f2c582640 1 -- 172.21.15.76:0/1897884713 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f24152bb0 con 0x7f1f241526b0 2024-04-25T19:15:01.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.711+0000 7f1f1affd640 1 -- 172.21.15.76:0/1897884713 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c003bd0 con 0x7f1f241526b0 2024-04-25T19:15:01.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.711+0000 7f1f2c582640 1 -- 172.21.15.76:0/1897884713 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f24157e20 con 0x7f1f241526b0 2024-04-25T19:15:01.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.711+0000 7f1f1affd640 1 -- 172.21.15.76:0/1897884713 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1f1c00a030 con 0x7f1f241526b0 2024-04-25T19:15:01.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.711+0000 7f1f1affd640 1 -- 172.21.15.76:0/1897884713 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1f1c009d50 con 0x7f1f241526b0 2024-04-25T19:15:01.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.712+0000 7f1f2c582640 1 -- 172.21.15.76:0/1897884713 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f2406d6a0 con 0x7f1f241526b0 2024-04-25T19:15:01.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.714+0000 7f1f1affd640 1 -- 172.21.15.76:0/1897884713 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f1f1c024020 con 0x7f1f241526b0 2024-04-25T19:15:01.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.780+0000 7f1f2c582640 1 -- 172.21.15.76:0/1897884713 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f1f24073e00 con 0x7f1f241526b0 2024-04-25T19:15:01.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.782+0000 7f1f1affd640 1 -- 172.21.15.76:0/1897884713 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f1f1c003d70 con 0x7f1f241526b0 2024-04-25T19:15:01.784 INFO:teuthology.orchestra.run.smithi076.stderr:adjusted tunables profile to default 2024-04-25T19:15:01.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.783+0000 7f1f2c582640 1 -- 172.21.15.76:0/1897884713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f241526b0 msgr2=0x7f1f2414acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:01.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.783+0000 7f1f2c582640 1 --2- 172.21.15.76:0/1897884713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f241526b0 0x7f1f2414acd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0096d0 tx=0x7f1f1c007f40 comp rx=0 tx=0).stop 2024-04-25T19:15:01.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.783+0000 7f1f2c582640 1 -- 172.21.15.76:0/1897884713 shutdown_connections 2024-04-25T19:15:01.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.783+0000 7f1f2c582640 1 --2- 172.21.15.76:0/1897884713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f241526b0 0x7f1f2414acd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:01.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.783+0000 7f1f2c582640 1 --2- 172.21.15.76:0/1897884713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f24152160 0x7f1f240ca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:01.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.783+0000 7f1f2c582640 1 --2- 172.21.15.76:0/1897884713 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1f100039b0 0x7f1f24150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:01.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.783+0000 7f1f2c582640 1 -- 172.21.15.76:0/1897884713 >> 172.21.15.76:0/1897884713 conn(0x7f1f2406c5c0 msgr2=0x7f1f24073160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:01.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.783+0000 7f1f2c582640 1 -- 172.21.15.76:0/1897884713 shutdown_connections 2024-04-25T19:15:01.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:01.783+0000 7f1f2c582640 1 -- 172.21.15.76:0/1897884713 wait complete. 2024-04-25T19:15:01.796 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T19:15:01.796 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T19:15:01.796 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:15:01.797 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T19:15:01.820 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:15:01.820 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T19:15:01.829 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:01.827+0000 7fbf6c5ee040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T19:15:01.842 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-04-25T19:15:01.841+0000 7f653fe2f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T19:15:01.866 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:15:01.866 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T19:15:01.929 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:15:01.929 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T19:15:01.968 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:01.966+0000 7fbf6c5ee040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T19:15:01.992 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-04-25T19:15:01.990+0000 7f653fe2f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T19:15:01.993 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-25T19:15:01.993 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T19:15:02.017 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-25T19:15:02.017 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T19:15:02.023 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:02.021+0000 7fbf6c5ee040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T19:15:02.045 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-25T19:15:02.045 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T19:15:02.045 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-04-25T19:15:02.044+0000 7f653fe2f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T19:15:02.108 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-04-25T19:15:02.108 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T19:15:02.171 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new 72a16c02-48f9-45aa-9447-fcf755ebf936 0 2024-04-25T19:15:02.266 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:02.264+0000 7fbf6c5ee040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T19:15:02.283 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-04-25T19:15:02.282+0000 7f653fe2f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T19:15:02.315 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:02.313+0000 7fbf6c5ee040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T19:15:02.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.318+0000 7f8c864aa640 1 Processor -- start 2024-04-25T19:15:02.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.318+0000 7f8c864aa640 1 -- start start 2024-04-25T19:15:02.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.319+0000 7f8c864aa640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8c80073940 0x7f8c80074710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:02.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.319+0000 7f8c864aa640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8c80154960 0x7f8c800cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:02.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.319+0000 7f8c864aa640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c800cb920 0x7f8c800cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:02.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.319+0000 7f8c864aa640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f8c80074c50 con 0x7f8c80154960 2024-04-25T19:15:02.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.319+0000 7f8c864aa640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f8c8006cf90 con 0x7f8c80073940 2024-04-25T19:15:02.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.319+0000 7f8c864aa640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8c8006d420 con 0x7f8c800cb920 2024-04-25T19:15:02.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.319+0000 7f8c7ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8c80073940 0x7f8c80074710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:02.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.319+0000 7f8c84a47640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c800cb920 0x7f8c800cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:02.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.319+0000 7f8c7ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8c80073940 0x7f8c80074710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44744/0 (socket says 172.21.15.105:44744) 2024-04-25T19:15:02.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.319+0000 7f8c7ffff640 1 -- 172.21.15.105:0/688188926 learned_addr learned my addr 172.21.15.105:0/688188926 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-25T19:15:02.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.320+0000 7f8c7f7fe640 1 --2- 172.21.15.105:0/688188926 >> v2:172.21.15.76:3300/0 conn(0x7f8c80154960 0x7f8c800cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:02.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.320+0000 7f8c7ffff640 1 -- 172.21.15.105:0/688188926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c800cb920 msgr2=0x7f8c800cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:02.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.320+0000 7f8c7ffff640 1 --2- 172.21.15.105:0/688188926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c800cb920 0x7f8c800cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.320+0000 7f8c7ffff640 1 -- 172.21.15.105:0/688188926 >> v2:172.21.15.76:3300/0 conn(0x7f8c80154960 msgr2=0x7f8c800cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:02.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.320+0000 7f8c7ffff640 1 --2- 172.21.15.105:0/688188926 >> v2:172.21.15.76:3300/0 conn(0x7f8c80154960 0x7f8c800cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.320+0000 7f8c7ffff640 1 -- 172.21.15.105:0/688188926 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8015b400 con 0x7f8c80073940 2024-04-25T19:15:02.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.320+0000 7f8c7f7fe640 1 --2- 172.21.15.105:0/688188926 >> v2:172.21.15.76:3300/0 conn(0x7f8c80154960 0x7f8c800cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:02.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.320+0000 7f8c84a47640 1 --2- 172.21.15.105:0/688188926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c800cb920 0x7f8c800cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:02.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.320+0000 7f8c7ffff640 1 --2- 172.21.15.105:0/688188926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c80073940 0x7f8c80074710 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8c800c9280 tx=0x7f8c7402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4970164d95f8a91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:02.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.321+0000 7f8c7effd640 1 -- 172.21.15.105:0/688188926 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74037070 con 0x7f8c80073940 2024-04-25T19:15:02.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.321+0000 7f8c7effd640 1 -- 172.21.15.105:0/688188926 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c74032440 con 0x7f8c80073940 2024-04-25T19:15:02.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.321+0000 7f8c7effd640 1 -- 172.21.15.105:0/688188926 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74032810 con 0x7f8c80073940 2024-04-25T19:15:02.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.321+0000 7f8c864aa640 1 -- 172.21.15.105:0/688188926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c80073940 msgr2=0x7f8c80074710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:02.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.321+0000 7f8c864aa640 1 --2- 172.21.15.105:0/688188926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c80073940 0x7f8c80074710 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8c800c9280 tx=0x7f8c7402a270 comp rx=0 tx=0).stop 2024-04-25T19:15:02.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.321+0000 7f8c864aa640 1 -- 172.21.15.105:0/688188926 shutdown_connections 2024-04-25T19:15:02.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.321+0000 7f8c864aa640 1 --2- 172.21.15.105:0/688188926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c800cb920 0x7f8c800cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.321+0000 7f8c864aa640 1 --2- 172.21.15.105:0/688188926 >> v2:172.21.15.76:3300/0 conn(0x7f8c80154960 0x7f8c800cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.321+0000 7f8c864aa640 1 --2- 172.21.15.105:0/688188926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c80073940 0x7f8c80074710 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.321+0000 7f8c864aa640 1 -- 172.21.15.105:0/688188926 >> 172.21.15.105:0/688188926 conn(0x7f8c80071a30 msgr2=0x7f8c80071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:02.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.321+0000 7f8c864aa640 1 -- 172.21.15.105:0/688188926 shutdown_connections 2024-04-25T19:15:02.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.322+0000 7f8c864aa640 1 -- 172.21.15.105:0/688188926 wait complete. 2024-04-25T19:15:02.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.323+0000 7f8c864aa640 1 Processor -- start 2024-04-25T19:15:02.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.323+0000 7f8c864aa640 1 -- start start 2024-04-25T19:15:02.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.325+0000 7f8c864aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c80073940 0x7f8c80155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:02.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.325+0000 7f8c864aa640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c800cb920 0x7f8c801f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:02.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.325+0000 7f8c864aa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c80154960 0x7f8c800ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:02.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.325+0000 7f8c864aa640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8006c940 con 0x7f8c80154960 2024-04-25T19:15:02.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.325+0000 7f8c864aa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8006c660 con 0x7f8c80073940 2024-04-25T19:15:02.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.325+0000 7f8c864aa640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8c8006cc20 con 0x7f8c800cb920 2024-04-25T19:15:02.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.325+0000 7f8c7ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c80073940 0x7f8c80155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:02.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.325+0000 7f8c7f7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c800cb920 0x7f8c801f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:02.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.325+0000 7f8c7ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c80073940 0x7f8c80155050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44746/0 (socket says 172.21.15.105:44746) 2024-04-25T19:15:02.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.325+0000 7f8c7ffff640 1 -- 172.21.15.105:0/3495355952 learned_addr learned my addr 172.21.15.105:0/3495355952 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-25T19:15:02.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.325+0000 7f8c84a47640 1 --2- 172.21.15.105:0/3495355952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c80154960 0x7f8c800ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:02.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.325+0000 7f8c7ffff640 1 -- 172.21.15.105:0/3495355952 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c800cb920 msgr2=0x7f8c801f2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:02.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.325+0000 7f8c7ffff640 1 --2- 172.21.15.105:0/3495355952 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c800cb920 0x7f8c801f2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.325+0000 7f8c7ffff640 1 -- 172.21.15.105:0/3495355952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c80154960 msgr2=0x7f8c800ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:02.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.325+0000 7f8c7ffff640 1 --2- 172.21.15.105:0/3495355952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c80154960 0x7f8c800ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.325+0000 7f8c7ffff640 1 -- 172.21.15.105:0/3495355952 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c801f3210 con 0x7f8c80073940 2024-04-25T19:15:02.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.325+0000 7f8c7ffff640 1 --2- 172.21.15.105:0/3495355952 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c80073940 0x7f8c80155050 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8c74000c00 tx=0x7f8c740046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:02.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.325+0000 7f8c7cff9640 1 -- 172.21.15.105:0/3495355952 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74037040 con 0x7f8c80073940 2024-04-25T19:15:02.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.325+0000 7f8c7cff9640 1 -- 172.21.15.105:0/3495355952 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c74042070 con 0x7f8c80073940 2024-04-25T19:15:02.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.325+0000 7f8c864aa640 1 -- 172.21.15.105:0/3495355952 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c801f5c80 con 0x7f8c80073940 2024-04-25T19:15:02.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.325+0000 7f8c7cff9640 1 -- 172.21.15.105:0/3495355952 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c740308a0 con 0x7f8c80073940 2024-04-25T19:15:02.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.325+0000 7f8c864aa640 1 -- 172.21.15.105:0/3495355952 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c801f6500 con 0x7f8c80073940 2024-04-25T19:15:02.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.326+0000 7f8c7cff9640 1 -- 172.21.15.105:0/3495355952 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8c74030a40 con 0x7f8c80073940 2024-04-25T19:15:02.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.326+0000 7f8c7cff9640 1 -- 172.21.15.105:0/3495355952 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f8c74047af0 con 0x7f8c80073940 2024-04-25T19:15:02.330 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-04-25T19:15:02.328+0000 7f653fe2f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T19:15:02.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.327+0000 7f8c864aa640 1 -- 172.21.15.105:0/3495355952 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c8006a390 con 0x7f8c80073940 2024-04-25T19:15:02.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.329+0000 7f8c7cff9640 1 -- 172.21.15.105:0/3495355952 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8c74047460 con 0x7f8c80073940 2024-04-25T19:15:02.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.399+0000 7f8c864aa640 1 -- 172.21.15.105:0/3495355952 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd new", "uuid": "72a16c02-48f9-45aa-9447-fcf755ebf936", "id": 0} v 0) -- 0x7f8c80152a00 con 0x7f8c80073940 2024-04-25T19:15:02.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.402+0000 7f8c7cff9640 1 -- 172.21.15.105:0/3495355952 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "72a16c02-48f9-45aa-9447-fcf755ebf936", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f8c74047640 con 0x7f8c80073940 2024-04-25T19:15:02.404 INFO:teuthology.orchestra.run.smithi105.stdout:0 2024-04-25T19:15:02.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.403+0000 7f8c864aa640 1 -- 172.21.15.105:0/3495355952 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c80073940 msgr2=0x7f8c80155050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:02.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.403+0000 7f8c864aa640 1 --2- 172.21.15.105:0/3495355952 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c80073940 0x7f8c80155050 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8c74000c00 tx=0x7f8c740046c0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.403+0000 7f8c864aa640 1 -- 172.21.15.105:0/3495355952 shutdown_connections 2024-04-25T19:15:02.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.403+0000 7f8c864aa640 1 --2- 172.21.15.105:0/3495355952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c80154960 0x7f8c800ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.403+0000 7f8c864aa640 1 --2- 172.21.15.105:0/3495355952 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c800cb920 0x7f8c801f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.403+0000 7f8c864aa640 1 --2- 172.21.15.105:0/3495355952 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c80073940 0x7f8c80155050 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.403+0000 7f8c864aa640 1 -- 172.21.15.105:0/3495355952 >> 172.21.15.105:0/3495355952 conn(0x7f8c80071a30 msgr2=0x7f8c80153f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:02.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.403+0000 7f8c864aa640 1 -- 172.21.15.105:0/3495355952 shutdown_connections 2024-04-25T19:15:02.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.403+0000 7f8c864aa640 1 -- 172.21.15.105:0/3495355952 wait complete. 2024-04-25T19:15:02.416 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new 818d50f0-76c6-45f8-bf79-5e304f14c5cf 1 2024-04-25T19:15:02.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.521+0000 7fb219d4d640 1 Processor -- start 2024-04-25T19:15:02.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.521+0000 7fb219d4d640 1 -- start start 2024-04-25T19:15:02.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.523+0000 7fb219d4d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2140cefe0 0x7fb2140cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:02.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.523+0000 7fb219d4d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb21406d420 con 0x7fb214074710 2024-04-25T19:15:02.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.523+0000 7fb219d4d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb21415b400 con 0x7fb214073940 2024-04-25T19:15:02.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.523+0000 7fb219d4d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb21406cf90 con 0x7fb2140cefe0 2024-04-25T19:15:02.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.523+0000 7fb2137fe640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb214073940 0x7fb214154960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:52020/0 (socket says 172.21.15.105:52020) 2024-04-25T19:15:02.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.523+0000 7fb2137fe640 1 -- 172.21.15.105:0/3205593495 learned_addr learned my addr 172.21.15.105:0/3205593495 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-25T19:15:02.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.523+0000 7fb213fff640 1 --2- 172.21.15.105:0/3205593495 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2140cefe0 0x7fb2140cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:02.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.523+0000 7fb2127fc640 1 -- 172.21.15.105:0/3205593495 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2428515897 0 0) 0x7fb21415b400 con 0x7fb214073940 2024-04-25T19:15:02.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.523+0000 7fb2127fc640 1 -- 172.21.15.105:0/3205593495 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1fc003520 con 0x7fb214073940 2024-04-25T19:15:02.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.523+0000 7fb213fff640 1 -- 172.21.15.105:0/3205593495 >> v1:172.21.15.105:6789/0 conn(0x7fb214073940 legacy=0x7fb214154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:02.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.523+0000 7fb213fff640 1 -- 172.21.15.105:0/3205593495 >> v1:172.21.15.76:6789/0 conn(0x7fb214074710 legacy=0x7fb2140cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:02.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.523+0000 7fb2127fc640 1 -- 172.21.15.105:0/3205593495 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3172691910 0 0) 0x7fb21406d420 con 0x7fb214074710 2024-04-25T19:15:02.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.523+0000 7fb213fff640 1 -- 172.21.15.105:0/3205593495 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb21415b6e0 con 0x7fb2140cefe0 2024-04-25T19:15:02.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.524+0000 7fb213fff640 1 --2- 172.21.15.105:0/3205593495 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2140cefe0 0x7fb2140cf3f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb20c009dc0 tx=0x7fb20c02a760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3413dd633863ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:02.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.524+0000 7fb2127fc640 1 -- 172.21.15.105:0/3205593495 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c007740 con 0x7fb2140cefe0 2024-04-25T19:15:02.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.525+0000 7fb2127fc640 1 -- 172.21.15.105:0/3205593495 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb20c0078e0 con 0x7fb2140cefe0 2024-04-25T19:15:02.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.525+0000 7fb2127fc640 1 -- 172.21.15.105:0/3205593495 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c007cb0 con 0x7fb2140cefe0 2024-04-25T19:15:02.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.525+0000 7fb219d4d640 1 -- 172.21.15.105:0/3205593495 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2140cefe0 msgr2=0x7fb2140cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:02.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.525+0000 7fb219d4d640 1 --2- 172.21.15.105:0/3205593495 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2140cefe0 0x7fb2140cf3f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb20c009dc0 tx=0x7fb20c02a760 comp rx=0 tx=0).stop 2024-04-25T19:15:02.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.525+0000 7fb219d4d640 1 -- 172.21.15.105:0/3205593495 shutdown_connections 2024-04-25T19:15:02.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.525+0000 7fb219d4d640 1 --2- 172.21.15.105:0/3205593495 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb2140cefe0 0x7fb2140cf3f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.525+0000 7fb219d4d640 1 -- 172.21.15.105:0/3205593495 >> 172.21.15.105:0/3205593495 conn(0x7fb214071a30 msgr2=0x7fb214071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:02.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.525+0000 7fb219d4d640 1 -- 172.21.15.105:0/3205593495 shutdown_connections 2024-04-25T19:15:02.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.525+0000 7fb219d4d640 1 -- 172.21.15.105:0/3205593495 wait complete. 2024-04-25T19:15:02.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.527+0000 7fb219d4d640 1 Processor -- start 2024-04-25T19:15:02.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.527+0000 7fb219d4d640 1 -- start start 2024-04-25T19:15:02.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.528+0000 7fb219d4d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb214073940 0x7fb214155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:02.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.528+0000 7fb219d4d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb214074710 0x7fb214157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:02.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.528+0000 7fb219d4d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2140cefe0 0x7fb2140cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:02.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.528+0000 7fb219d4d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb21406cc20 con 0x7fb214074710 2024-04-25T19:15:02.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.528+0000 7fb219d4d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb21406c940 con 0x7fb2140cefe0 2024-04-25T19:15:02.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.528+0000 7fb219d4d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb214073d50 con 0x7fb214073940 2024-04-25T19:15:02.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.528+0000 7fb2137fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb214073940 0x7fb214155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:02.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.529+0000 7fb2137fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb214073940 0x7fb214155690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:49698/0 (socket says 172.21.15.105:49698) 2024-04-25T19:15:02.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.529+0000 7fb213fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2140cefe0 0x7fb2140cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:02.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.529+0000 7fb2137fe640 1 -- 172.21.15.105:0/2800574431 learned_addr learned my addr 172.21.15.105:0/2800574431 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-25T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.529+0000 7fb212ffd640 1 --2- 172.21.15.105:0/2800574431 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb214074710 0x7fb214157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.529+0000 7fb2137fe640 1 -- 172.21.15.105:0/2800574431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2140cefe0 msgr2=0x7fb2140cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.529+0000 7fb2137fe640 1 --2- 172.21.15.105:0/2800574431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2140cefe0 0x7fb2140cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.529+0000 7fb2137fe640 1 -- 172.21.15.105:0/2800574431 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb214074710 msgr2=0x7fb214157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.529+0000 7fb2137fe640 1 --2- 172.21.15.105:0/2800574431 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb214074710 0x7fb214157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.529+0000 7fb2137fe640 1 -- 172.21.15.105:0/2800574431 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb214155bd0 con 0x7fb214073940 2024-04-25T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.529+0000 7fb213fff640 1 --2- 172.21.15.105:0/2800574431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2140cefe0 0x7fb2140cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.529+0000 7fb2137fe640 1 --2- 172.21.15.105:0/2800574431 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb214073940 0x7fb214155690 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb20800c390 tx=0x7fb20800c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.529+0000 7fb1fbfff640 1 -- 172.21.15.105:0/2800574431 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb208017070 con 0x7fb214073940 2024-04-25T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.529+0000 7fb1fbfff640 1 -- 172.21.15.105:0/2800574431 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb208005ce0 con 0x7fb214073940 2024-04-25T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.529+0000 7fb1fbfff640 1 -- 172.21.15.105:0/2800574431 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2080072f0 con 0x7fb214073940 2024-04-25T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.529+0000 7fb219d4d640 1 -- 172.21.15.105:0/2800574431 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2141e2e70 con 0x7fb214073940 2024-04-25T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.529+0000 7fb219d4d640 1 -- 172.21.15.105:0/2800574431 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2141e3430 con 0x7fb214073940 2024-04-25T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.529+0000 7fb1fbfff640 1 -- 172.21.15.105:0/2800574431 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb20801a070 con 0x7fb214073940 2024-04-25T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.530+0000 7fb1fbfff640 1 -- 172.21.15.105:0/2800574431 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fb2080127b0 con 0x7fb214073940 2024-04-25T19:15:02.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.530+0000 7fb219d4d640 1 -- 172.21.15.105:0/2800574431 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2140cf480 con 0x7fb214073940 2024-04-25T19:15:02.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.533+0000 7fb1fbfff640 1 -- 172.21.15.105:0/2800574431 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb2140cf480 con 0x7fb214073940 2024-04-25T19:15:02.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.605+0000 7fb219d4d640 1 -- 172.21.15.105:0/2800574431 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd new", "uuid": "818d50f0-76c6-45f8-bf79-5e304f14c5cf", "id": 1} v 0) -- 0x7fb214152e90 con 0x7fb214073940 2024-04-25T19:15:02.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.609+0000 7fb1fbfff640 1 -- 172.21.15.105:0/2800574431 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "818d50f0-76c6-45f8-bf79-5e304f14c5cf", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fb214152e90 con 0x7fb214073940 2024-04-25T19:15:02.648 INFO:teuthology.orchestra.run.smithi105.stdout:1 2024-04-25T19:15:02.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.609+0000 7fb219d4d640 1 -- 172.21.15.105:0/2800574431 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb214073940 msgr2=0x7fb214155690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:02.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.609+0000 7fb219d4d640 1 --2- 172.21.15.105:0/2800574431 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb214073940 0x7fb214155690 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb20800c390 tx=0x7fb20800c850 comp rx=0 tx=0).stop 2024-04-25T19:15:02.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.610+0000 7fb219d4d640 1 -- 172.21.15.105:0/2800574431 shutdown_connections 2024-04-25T19:15:02.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.610+0000 7fb219d4d640 1 --2- 172.21.15.105:0/2800574431 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2140cefe0 0x7fb2140cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.610+0000 7fb219d4d640 1 --2- 172.21.15.105:0/2800574431 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb214074710 0x7fb214157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.610+0000 7fb219d4d640 1 --2- 172.21.15.105:0/2800574431 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb214073940 0x7fb214155690 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.610+0000 7fb219d4d640 1 -- 172.21.15.105:0/2800574431 >> 172.21.15.105:0/2800574431 conn(0x7fb214071a30 msgr2=0x7fb214071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:02.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.610+0000 7fb219d4d640 1 -- 172.21.15.105:0/2800574431 shutdown_connections 2024-04-25T19:15:02.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.610+0000 7fb219d4d640 1 -- 172.21.15.105:0/2800574431 wait complete. 2024-04-25T19:15:02.651 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new 5714c8f5-ae23-473c-8cf1-db73811acded 2 2024-04-25T19:15:02.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.759+0000 7fb871a25640 1 Processor -- start 2024-04-25T19:15:02.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.760+0000 7fb871a25640 1 -- start start 2024-04-25T19:15:02.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.761+0000 7fb871a25640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb86c0cefe0 0x7fb86c0cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:02.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.761+0000 7fb871a25640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb86c154ce0 con 0x7fb86c1548d0 2024-04-25T19:15:02.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.761+0000 7fb871a25640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb86c15a810 con 0x7fb86c073940 2024-04-25T19:15:02.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.761+0000 7fb871a25640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb86c06cf00 con 0x7fb86c0cefe0 2024-04-25T19:15:02.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.761+0000 7fb86affd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb86c073940 0x7fb86c06c7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:52034/0 (socket says 172.21.15.105:52034) 2024-04-25T19:15:02.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.761+0000 7fb86affd640 1 -- 172.21.15.105:0/1981446560 learned_addr learned my addr 172.21.15.105:0/1981446560 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-04-25T19:15:02.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.761+0000 7fb86b7fe640 1 --2- 172.21.15.105:0/1981446560 >> v2:172.21.15.76:3300/0 conn(0x7fb86c0cefe0 0x7fb86c0cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:02.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.761+0000 7fb869ffb640 1 -- 172.21.15.105:0/1981446560 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2994841085 0 0) 0x7fb86c15a810 con 0x7fb86c073940 2024-04-25T19:15:02.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.761+0000 7fb869ffb640 1 -- 172.21.15.105:0/1981446560 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb854003520 con 0x7fb86c073940 2024-04-25T19:15:02.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.761+0000 7fb869ffb640 1 -- 172.21.15.105:0/1981446560 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3356234986 0 0) 0x7fb86c154ce0 con 0x7fb86c1548d0 2024-04-25T19:15:02.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.761+0000 7fb869ffb640 1 -- 172.21.15.105:0/1981446560 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb86c15a810 con 0x7fb86c1548d0 2024-04-25T19:15:02.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.761+0000 7fb86b7fe640 1 -- 172.21.15.105:0/1981446560 >> v1:172.21.15.105:6789/0 conn(0x7fb86c073940 legacy=0x7fb86c06c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:02.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.761+0000 7fb869ffb640 1 -- 172.21.15.105:0/1981446560 <== mon.1 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4154843026 0 0) 0x7fb854003520 con 0x7fb86c073940 2024-04-25T19:15:02.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.761+0000 7fb86b7fe640 1 -- 172.21.15.105:0/1981446560 >> v1:172.21.15.76:6789/0 conn(0x7fb86c1548d0 legacy=0x7fb86c15aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:02.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.762+0000 7fb86b7fe640 1 -- 172.21.15.105:0/1981446560 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86c06d470 con 0x7fb86c0cefe0 2024-04-25T19:15:02.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.762+0000 7fb86b7fe640 1 --2- 172.21.15.105:0/1981446560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb86c0cefe0 0x7fb86c0cf3f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb864009720 tx=0x7fb86402a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8819043a25fdb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:02.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.762+0000 7fb869ffb640 1 -- 172.21.15.105:0/1981446560 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864037070 con 0x7fb86c0cefe0 2024-04-25T19:15:02.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.762+0000 7fb869ffb640 1 -- 172.21.15.105:0/1981446560 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb86402ac60 con 0x7fb86c0cefe0 2024-04-25T19:15:02.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.762+0000 7fb869ffb640 1 -- 172.21.15.105:0/1981446560 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c0019f0 con 0x7fb86c0cefe0 2024-04-25T19:15:02.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.763+0000 7fb871a25640 1 -- 172.21.15.105:0/1981446560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb86c0cefe0 msgr2=0x7fb86c0cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:02.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.763+0000 7fb871a25640 1 --2- 172.21.15.105:0/1981446560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb86c0cefe0 0x7fb86c0cf3f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb864009720 tx=0x7fb86402a150 comp rx=0 tx=0).stop 2024-04-25T19:15:02.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.763+0000 7fb871a25640 1 -- 172.21.15.105:0/1981446560 shutdown_connections 2024-04-25T19:15:02.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.763+0000 7fb871a25640 1 --2- 172.21.15.105:0/1981446560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb86c0cefe0 0x7fb86c0cf3f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.763+0000 7fb871a25640 1 -- 172.21.15.105:0/1981446560 >> 172.21.15.105:0/1981446560 conn(0x7fb86c071a30 msgr2=0x7fb86c071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:02.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.763+0000 7fb871a25640 1 -- 172.21.15.105:0/1981446560 shutdown_connections 2024-04-25T19:15:02.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.763+0000 7fb871a25640 1 -- 172.21.15.105:0/1981446560 wait complete. 2024-04-25T19:15:02.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.765+0000 7fb871a25640 1 Processor -- start 2024-04-25T19:15:02.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.765+0000 7fb871a25640 1 -- start start 2024-04-25T19:15:02.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.766+0000 7fb871a25640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb86c073940 0x7fb86c1e4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:02.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.766+0000 7fb871a25640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb86c0cefe0 0x7fb86c15c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:02.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.766+0000 7fb871a25640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb86c1548d0 0x7fb86c1552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:02.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.766+0000 7fb871a25640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c0744a0 con 0x7fb86c073940 2024-04-25T19:15:02.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.766+0000 7fb871a25640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c15ab60 con 0x7fb86c0cefe0 2024-04-25T19:15:02.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.766+0000 7fb871a25640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb86c074780 con 0x7fb86c1548d0 2024-04-25T19:15:02.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.766+0000 7fb86a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb86c0cefe0 0x7fb86c15c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:02.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.766+0000 7fb86b7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb86c1548d0 0x7fb86c1552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:02.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.766+0000 7fb86a7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb86c0cefe0 0x7fb86c15c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44760/0 (socket says 172.21.15.105:44760) 2024-04-25T19:15:02.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.766+0000 7fb86a7fc640 1 -- 172.21.15.105:0/1123936319 learned_addr learned my addr 172.21.15.105:0/1123936319 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-25T19:15:02.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.766+0000 7fb86affd640 1 --2- 172.21.15.105:0/1123936319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb86c073940 0x7fb86c1e4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.767+0000 7fb86a7fc640 1 -- 172.21.15.105:0/1123936319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb86c1548d0 msgr2=0x7fb86c1552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.767+0000 7fb86a7fc640 1 --2- 172.21.15.105:0/1123936319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb86c1548d0 0x7fb86c1552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.767+0000 7fb86a7fc640 1 -- 172.21.15.105:0/1123936319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb86c073940 msgr2=0x7fb86c1e4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.767+0000 7fb86a7fc640 1 --2- 172.21.15.105:0/1123936319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb86c073940 0x7fb86c1e4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.767+0000 7fb86a7fc640 1 -- 172.21.15.105:0/1123936319 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86c15c560 con 0x7fb86c0cefe0 2024-04-25T19:15:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.767+0000 7fb86b7fe640 1 --2- 172.21.15.105:0/1123936319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb86c1548d0 0x7fb86c1552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.767+0000 7fb86a7fc640 1 --2- 172.21.15.105:0/1123936319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb86c0cefe0 0x7fb86c15c020 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb86000c430 tx=0x7fb86000c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.767+0000 7fb853fff640 1 -- 172.21.15.105:0/1123936319 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860017070 con 0x7fb86c0cefe0 2024-04-25T19:15:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.767+0000 7fb853fff640 1 -- 172.21.15.105:0/1123936319 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8600052f0 con 0x7fb86c0cefe0 2024-04-25T19:15:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.767+0000 7fb871a25640 1 -- 172.21.15.105:0/1123936319 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86c1f3dd0 con 0x7fb86c0cefe0 2024-04-25T19:15:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.767+0000 7fb853fff640 1 -- 172.21.15.105:0/1123936319 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860003830 con 0x7fb86c0cefe0 2024-04-25T19:15:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.767+0000 7fb871a25640 1 -- 172.21.15.105:0/1123936319 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb86c15da40 con 0x7fb86c0cefe0 2024-04-25T19:15:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.767+0000 7fb853fff640 1 -- 172.21.15.105:0/1123936319 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb86001a070 con 0x7fb86c0cefe0 2024-04-25T19:15:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.767+0000 7fb853fff640 1 -- 172.21.15.105:0/1123936319 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fb860011c20 con 0x7fb86c0cefe0 2024-04-25T19:15:02.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.768+0000 7fb871a25640 1 -- 172.21.15.105:0/1123936319 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb86c1f3b20 con 0x7fb86c0cefe0 2024-04-25T19:15:02.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.770+0000 7fb853fff640 1 -- 172.21.15.105:0/1123936319 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb860021070 con 0x7fb86c0cefe0 2024-04-25T19:15:02.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.843+0000 7fb871a25640 1 -- 172.21.15.105:0/1123936319 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5714c8f5-ae23-473c-8cf1-db73811acded", "id": 2} v 0) -- 0x7fb86c06a390 con 0x7fb86c0cefe0 2024-04-25T19:15:02.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.849+0000 7fb853fff640 1 -- 172.21.15.105:0/1123936319 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5714c8f5-ae23-473c-8cf1-db73811acded", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fb860011590 con 0x7fb86c0cefe0 2024-04-25T19:15:02.851 INFO:teuthology.orchestra.run.smithi105.stdout:2 2024-04-25T19:15:02.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.850+0000 7fb871a25640 1 -- 172.21.15.105:0/1123936319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb86c0cefe0 msgr2=0x7fb86c15c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:02.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.850+0000 7fb871a25640 1 --2- 172.21.15.105:0/1123936319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb86c0cefe0 0x7fb86c15c020 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb86000c430 tx=0x7fb86000c8f0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.850+0000 7fb871a25640 1 -- 172.21.15.105:0/1123936319 shutdown_connections 2024-04-25T19:15:02.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.850+0000 7fb871a25640 1 --2- 172.21.15.105:0/1123936319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb86c1548d0 0x7fb86c1552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.850+0000 7fb871a25640 1 --2- 172.21.15.105:0/1123936319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb86c0cefe0 0x7fb86c15c020 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.850+0000 7fb871a25640 1 --2- 172.21.15.105:0/1123936319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb86c073940 0x7fb86c1e4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.850+0000 7fb871a25640 1 -- 172.21.15.105:0/1123936319 >> 172.21.15.105:0/1123936319 conn(0x7fb86c071a30 msgr2=0x7fb86c158710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:02.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.850+0000 7fb871a25640 1 -- 172.21.15.105:0/1123936319 shutdown_connections 2024-04-25T19:15:02.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.850+0000 7fb871a25640 1 -- 172.21.15.105:0/1123936319 wait complete. 2024-04-25T19:15:02.867 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new 13204cdb-d8e6-4f97-bc6e-9561f442decf 3 2024-04-25T19:15:02.975 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.973+0000 7f5b5ed57640 1 Processor -- start 2024-04-25T19:15:02.976 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.973+0000 7f5b5ed57640 1 -- start start 2024-04-25T19:15:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.975+0000 7f5b5ed57640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5b58156f60 0x7f5b580cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.975+0000 7f5b5ed57640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5b580c95c0 0x7f5b580cf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.975+0000 7f5b5ed57640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b58073c30 con 0x7f5b580c9ea0 2024-04-25T19:15:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.975+0000 7f5b5ed57640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5b5815d730 con 0x7f5b580c95c0 2024-04-25T19:15:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.975+0000 7f5b5ed57640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f5b580746b0 con 0x7f5b58156f60 2024-04-25T19:15:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.975+0000 7f5b5caf3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5b58156f60 0x7f5b580cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:02.977 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.975+0000 7f5b5caf3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5b58156f60 0x7f5b580cefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44774/0 (socket says 172.21.15.105:44774) 2024-04-25T19:15:02.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.975+0000 7f5b5caf3640 1 -- 172.21.15.105:0/2766837728 learned_addr learned my addr 172.21.15.105:0/2766837728 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-25T19:15:02.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.975+0000 7f5b5d2f4640 1 --1- 172.21.15.105:0/2766837728 >> v1:172.21.15.105:6789/0 conn(0x7f5b580c9ea0 0x7f5b580c7790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:52044/0 (socket says 172.21.15.105:52044) 2024-04-25T19:15:02.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.975+0000 7f5b57fff640 1 --2- 172.21.15.105:0/2766837728 >> v2:172.21.15.76:3300/0 conn(0x7f5b580c95c0 0x7f5b580cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:02.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.975+0000 7f5b5caf3640 1 -- 172.21.15.105:0/2766837728 >> v2:172.21.15.76:3300/0 conn(0x7f5b580c95c0 msgr2=0x7f5b580cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:02.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.975+0000 7f5b5caf3640 1 --2- 172.21.15.105:0/2766837728 >> v2:172.21.15.76:3300/0 conn(0x7f5b580c95c0 0x7f5b580cf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.975+0000 7f5b5caf3640 1 -- 172.21.15.105:0/2766837728 >> v1:172.21.15.105:6789/0 conn(0x7f5b580c9ea0 legacy=0x7f5b580c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:02.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.975+0000 7f5b5caf3640 1 -- 172.21.15.105:0/2766837728 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b58073940 con 0x7f5b58156f60 2024-04-25T19:15:02.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.976+0000 7f5b5caf3640 1 --2- 172.21.15.105:0/2766837728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b58156f60 0x7f5b580cefe0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5b5806c720 tx=0x7f5b4c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b77d16040c74539 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:02.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.976+0000 7f5b577fe640 1 -- 172.21.15.105:0/2766837728 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c037070 con 0x7f5b58156f60 2024-04-25T19:15:02.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.976+0000 7f5b577fe640 1 -- 172.21.15.105:0/2766837728 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b4c032440 con 0x7f5b58156f60 2024-04-25T19:15:02.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.976+0000 7f5b577fe640 1 -- 172.21.15.105:0/2766837728 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c032810 con 0x7f5b58156f60 2024-04-25T19:15:02.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.977+0000 7f5b5ed57640 1 -- 172.21.15.105:0/2766837728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b58156f60 msgr2=0x7f5b580cefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:02.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.977+0000 7f5b5ed57640 1 --2- 172.21.15.105:0/2766837728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b58156f60 0x7f5b580cefe0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5b5806c720 tx=0x7f5b4c02a270 comp rx=0 tx=0).stop 2024-04-25T19:15:02.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.977+0000 7f5b5ed57640 1 -- 172.21.15.105:0/2766837728 shutdown_connections 2024-04-25T19:15:02.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.977+0000 7f5b5ed57640 1 --2- 172.21.15.105:0/2766837728 >> v2:172.21.15.76:3300/0 conn(0x7f5b580c95c0 0x7f5b580cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.977+0000 7f5b5ed57640 1 --2- 172.21.15.105:0/2766837728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b58156f60 0x7f5b580cefe0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.977+0000 7f5b5ed57640 1 -- 172.21.15.105:0/2766837728 >> 172.21.15.105:0/2766837728 conn(0x7f5b58071a30 msgr2=0x7f5b58071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:02.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.977+0000 7f5b5ed57640 1 -- 172.21.15.105:0/2766837728 shutdown_connections 2024-04-25T19:15:02.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.977+0000 7f5b5ed57640 1 -- 172.21.15.105:0/2766837728 wait complete. 2024-04-25T19:15:02.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.979+0000 7f5b5ed57640 1 Processor -- start 2024-04-25T19:15:02.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.979+0000 7f5b5ed57640 1 -- start start 2024-04-25T19:15:02.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.980+0000 7f5b5ed57640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b580c95c0 0x7f5b580cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:02.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.980+0000 7f5b5ed57640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b580c9ea0 0x7f5b580ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:02.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.980+0000 7f5b5ed57640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b58156f60 0x7f5b580cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:02.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.980+0000 7f5b5ed57640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5b580740e0 con 0x7f5b580c95c0 2024-04-25T19:15:02.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.980+0000 7f5b5ed57640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5815da10 con 0x7f5b58156f60 2024-04-25T19:15:02.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.980+0000 7f5b5ed57640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f5b580c7ea0 con 0x7f5b580c9ea0 2024-04-25T19:15:02.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.980+0000 7f5b5d2f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b58156f60 0x7f5b580cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:02.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.980+0000 7f5b57fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b580c9ea0 0x7f5b580ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:02.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.980+0000 7f5b5d2f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b58156f60 0x7f5b580cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44784/0 (socket says 172.21.15.105:44784) 2024-04-25T19:15:02.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.980+0000 7f5b5d2f4640 1 -- 172.21.15.105:0/2895536382 learned_addr learned my addr 172.21.15.105:0/2895536382 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-25T19:15:02.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.980+0000 7f5b5caf3640 1 --2- 172.21.15.105:0/2895536382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b580c95c0 0x7f5b580cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:02.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.980+0000 7f5b5d2f4640 1 -- 172.21.15.105:0/2895536382 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b580c9ea0 msgr2=0x7f5b580ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:02.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.980+0000 7f5b5d2f4640 1 --2- 172.21.15.105:0/2895536382 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b580c9ea0 0x7f5b580ca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.980+0000 7f5b5d2f4640 1 -- 172.21.15.105:0/2895536382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b580c95c0 msgr2=0x7f5b580cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:02.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.980+0000 7f5b5d2f4640 1 --2- 172.21.15.105:0/2895536382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b580c95c0 0x7f5b580cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:02.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.980+0000 7f5b5d2f4640 1 -- 172.21.15.105:0/2895536382 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b580ca9a0 con 0x7f5b58156f60 2024-04-25T19:15:02.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.980+0000 7f5b57fff640 1 --2- 172.21.15.105:0/2895536382 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b580c9ea0 0x7f5b580ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:02.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.981+0000 7f5b5d2f4640 1 --2- 172.21.15.105:0/2895536382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b58156f60 0x7f5b580cb3e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5b5000c6c0 tx=0x7f5b5000cb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:02.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.981+0000 7f5b557fa640 1 -- 172.21.15.105:0/2895536382 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b50018070 con 0x7f5b58156f60 2024-04-25T19:15:02.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.981+0000 7f5b557fa640 1 -- 172.21.15.105:0/2895536382 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b50004d60 con 0x7f5b58156f60 2024-04-25T19:15:02.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.981+0000 7f5b5ed57640 1 -- 172.21.15.105:0/2895536382 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b5815ab40 con 0x7f5b58156f60 2024-04-25T19:15:02.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.981+0000 7f5b557fa640 1 -- 172.21.15.105:0/2895536382 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5000a840 con 0x7f5b58156f60 2024-04-25T19:15:02.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.981+0000 7f5b5ed57640 1 -- 172.21.15.105:0/2895536382 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b580ce130 con 0x7f5b58156f60 2024-04-25T19:15:02.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.981+0000 7f5b557fa640 1 -- 172.21.15.105:0/2895536382 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5b5001b070 con 0x7f5b58156f60 2024-04-25T19:15:02.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.981+0000 7f5b557fa640 1 -- 172.21.15.105:0/2895536382 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f5b50007750 con 0x7f5b58156f60 2024-04-25T19:15:02.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.982+0000 7f5b5ed57640 1 -- 172.21.15.105:0/2895536382 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b5806a390 con 0x7f5b58156f60 2024-04-25T19:15:02.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:02.984+0000 7f5b557fa640 1 -- 172.21.15.105:0/2895536382 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5b50022070 con 0x7f5b58156f60 2024-04-25T19:15:03.032 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:03.030+0000 7fbf6c5ee040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T19:15:03.045 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-04-25T19:15:03.044+0000 7f653fe2f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T19:15:03.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.057+0000 7f5b5ed57640 1 -- 172.21.15.105:0/2895536382 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd new", "uuid": "13204cdb-d8e6-4f97-bc6e-9561f442decf", "id": 3} v 0) -- 0x7f5b581545b0 con 0x7f5b58156f60 2024-04-25T19:15:03.064 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.062+0000 7f5b557fa640 1 -- 172.21.15.105:0/2895536382 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "13204cdb-d8e6-4f97-bc6e-9561f442decf", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f5b50007070 con 0x7f5b58156f60 2024-04-25T19:15:03.064 INFO:teuthology.orchestra.run.smithi105.stdout:3 2024-04-25T19:15:03.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.063+0000 7f5b5ed57640 1 -- 172.21.15.105:0/2895536382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b58156f60 msgr2=0x7f5b580cb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:03.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.063+0000 7f5b5ed57640 1 --2- 172.21.15.105:0/2895536382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b58156f60 0x7f5b580cb3e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5b5000c6c0 tx=0x7f5b5000cb80 comp rx=0 tx=0).stop 2024-04-25T19:15:03.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.063+0000 7f5b5ed57640 1 -- 172.21.15.105:0/2895536382 shutdown_connections 2024-04-25T19:15:03.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.063+0000 7f5b5ed57640 1 --2- 172.21.15.105:0/2895536382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b58156f60 0x7f5b580cb3e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.063+0000 7f5b5ed57640 1 --2- 172.21.15.105:0/2895536382 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b580c9ea0 0x7f5b580ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.063+0000 7f5b5ed57640 1 --2- 172.21.15.105:0/2895536382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b580c95c0 0x7f5b580cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.063+0000 7f5b5ed57640 1 -- 172.21.15.105:0/2895536382 >> 172.21.15.105:0/2895536382 conn(0x7f5b58071a30 msgr2=0x7f5b58155780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:03.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.063+0000 7f5b5ed57640 1 -- 172.21.15.105:0/2895536382 shutdown_connections 2024-04-25T19:15:03.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.063+0000 7f5b5ed57640 1 -- 172.21.15.105:0/2895536382 wait complete. 2024-04-25T19:15:03.076 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new 5bea5166-1986-4d56-b5f5-ecc8cbe7f3ec 4 2024-04-25T19:15:03.125 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:03.123+0000 7fbf6c5ee040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T19:15:03.133 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-04-25T19:15:03.132+0000 7f653fe2f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T19:15:03.171 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:03.170+0000 7fbf6c5ee040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T19:15:03.175 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-04-25T19:15:03.173+0000 7f653fe2f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T19:15:03.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.182+0000 7f34b9a67640 1 Processor -- start 2024-04-25T19:15:03.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.182+0000 7f34b9a67640 1 -- start start 2024-04-25T19:15:03.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.183+0000 7f34b9a67640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34b40c60c0 0x7f34b414ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:03.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.183+0000 7f34b9a67640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f34b4159f40 0x7f34b40c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:03.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.183+0000 7f34b9a67640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f34b415a350 0x7f34b40c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:03.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.183+0000 7f34b9a67640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f34b40c83d0 con 0x7f34b415a350 2024-04-25T19:15:03.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.184+0000 7f34b9a67640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f34b4159760 con 0x7f34b4159f40 2024-04-25T19:15:03.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.184+0000 7f34b9a67640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f34b40c6ae0 con 0x7f34b40c60c0 2024-04-25T19:15:03.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.184+0000 7f34b37fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34b40c60c0 0x7f34b414ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:03.186 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.184+0000 7f34b2ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f34b4159f40 0x7f34b40c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:03.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.184+0000 7f34b2ffd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f34b4159f40 0x7f34b40c87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44790/0 (socket says 172.21.15.105:44790) 2024-04-25T19:15:03.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.184+0000 7f34b37fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34b40c60c0 0x7f34b414ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:49736/0 (socket says 172.21.15.105:49736) 2024-04-25T19:15:03.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.184+0000 7f34b2ffd640 1 -- 172.21.15.105:0/867753294 learned_addr learned my addr 172.21.15.105:0/867753294 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-25T19:15:03.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.184+0000 7f34b3fff640 1 --2- 172.21.15.105:0/867753294 >> v2:172.21.15.76:3300/0 conn(0x7f34b415a350 0x7f34b40c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:03.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.184+0000 7f34b2ffd640 1 -- 172.21.15.105:0/867753294 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34b40c60c0 msgr2=0x7f34b414ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:03.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.184+0000 7f34b2ffd640 1 --2- 172.21.15.105:0/867753294 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34b40c60c0 0x7f34b414ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.184+0000 7f34b2ffd640 1 -- 172.21.15.105:0/867753294 >> v2:172.21.15.76:3300/0 conn(0x7f34b415a350 msgr2=0x7f34b40c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:03.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.184+0000 7f34b2ffd640 1 --2- 172.21.15.105:0/867753294 >> v2:172.21.15.76:3300/0 conn(0x7f34b415a350 0x7f34b40c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.184+0000 7f34b2ffd640 1 -- 172.21.15.105:0/867753294 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b40c6dc0 con 0x7f34b4159f40 2024-04-25T19:15:03.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.184+0000 7f34b37fe640 1 --2- 172.21.15.105:0/867753294 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34b40c60c0 0x7f34b414ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:03.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.185+0000 7f34b2ffd640 1 --2- 172.21.15.105:0/867753294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34b4159f40 0x7f34b40c87b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009870 tx=0x7f34a402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9625bbd4b608b021 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:03.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.185+0000 7f34b27fc640 1 -- 172.21.15.105:0/867753294 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4037070 con 0x7f34b4159f40 2024-04-25T19:15:03.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.185+0000 7f34b27fc640 1 -- 172.21.15.105:0/867753294 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a4032440 con 0x7f34b4159f40 2024-04-25T19:15:03.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.185+0000 7f34b27fc640 1 -- 172.21.15.105:0/867753294 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4032810 con 0x7f34b4159f40 2024-04-25T19:15:03.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.186+0000 7f34b9a67640 1 -- 172.21.15.105:0/867753294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34b4159f40 msgr2=0x7f34b40c87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:03.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.186+0000 7f34b9a67640 1 --2- 172.21.15.105:0/867753294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34b4159f40 0x7f34b40c87b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009870 tx=0x7f34a402a180 comp rx=0 tx=0).stop 2024-04-25T19:15:03.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.186+0000 7f34b9a67640 1 -- 172.21.15.105:0/867753294 shutdown_connections 2024-04-25T19:15:03.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.186+0000 7f34b9a67640 1 --2- 172.21.15.105:0/867753294 >> v2:172.21.15.76:3300/0 conn(0x7f34b415a350 0x7f34b40c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.186+0000 7f34b9a67640 1 --2- 172.21.15.105:0/867753294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34b4159f40 0x7f34b40c87b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.186+0000 7f34b9a67640 1 --2- 172.21.15.105:0/867753294 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34b40c60c0 0x7f34b414ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.186+0000 7f34b9a67640 1 -- 172.21.15.105:0/867753294 >> 172.21.15.105:0/867753294 conn(0x7f34b4071a30 msgr2=0x7f34b4071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:03.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.186+0000 7f34b9a67640 1 -- 172.21.15.105:0/867753294 shutdown_connections 2024-04-25T19:15:03.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.186+0000 7f34b9a67640 1 -- 172.21.15.105:0/867753294 wait complete. 2024-04-25T19:15:03.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.187+0000 7f34b9a67640 1 Processor -- start 2024-04-25T19:15:03.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.188+0000 7f34b9a67640 1 -- start start 2024-04-25T19:15:03.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.189+0000 7f34b9a67640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34b40c60c0 0x7f34b41f4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:03.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.189+0000 7f34b9a67640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34b4159f40 0x7f34b4152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:03.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.189+0000 7f34b9a67640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34b415a350 0x7f34b414eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:03.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.189+0000 7f34b9a67640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f34b415ac50 con 0x7f34b40c60c0 2024-04-25T19:15:03.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.189+0000 7f34b9a67640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f34b4159a20 con 0x7f34b4159f40 2024-04-25T19:15:03.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.189+0000 7f34b9a67640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f34b40c6600 con 0x7f34b415a350 2024-04-25T19:15:03.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.189+0000 7f34b3fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34b415a350 0x7f34b414eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:03.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.189+0000 7f34b2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34b4159f40 0x7f34b4152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:03.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.189+0000 7f34b3fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34b415a350 0x7f34b414eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:49742/0 (socket says 172.21.15.105:49742) 2024-04-25T19:15:03.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.189+0000 7f34b3fff640 1 -- 172.21.15.105:0/2167743314 learned_addr learned my addr 172.21.15.105:0/2167743314 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-25T19:15:03.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.189+0000 7f34b2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34b4159f40 0x7f34b4152e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44794/0 (socket says 172.21.15.105:44794) 2024-04-25T19:15:03.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.189+0000 7f34b37fe640 1 --2- 172.21.15.105:0/2167743314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34b40c60c0 0x7f34b41f4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:03.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.189+0000 7f34b2ffd640 1 -- 172.21.15.105:0/2167743314 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34b415a350 msgr2=0x7f34b414eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:03.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.189+0000 7f34b2ffd640 1 --2- 172.21.15.105:0/2167743314 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34b415a350 0x7f34b414eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.189+0000 7f34b2ffd640 1 -- 172.21.15.105:0/2167743314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34b40c60c0 msgr2=0x7f34b41f4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:03.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.189+0000 7f34b2ffd640 1 --2- 172.21.15.105:0/2167743314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34b40c60c0 0x7f34b41f4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.189+0000 7f34b2ffd640 1 -- 172.21.15.105:0/2167743314 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b4152280 con 0x7f34b4159f40 2024-04-25T19:15:03.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.189+0000 7f34b3fff640 1 --2- 172.21.15.105:0/2167743314 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34b415a350 0x7f34b414eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:03.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.189+0000 7f34b37fe640 1 --2- 172.21.15.105:0/2167743314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34b40c60c0 0x7f34b41f4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:03.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.190+0000 7f34b2ffd640 1 --2- 172.21.15.105:0/2167743314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34b4159f40 0x7f34b4152e00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f34a4031000 tx=0x7f34a40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:03.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.190+0000 7f34abfff640 1 -- 172.21.15.105:0/2167743314 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4041020 con 0x7f34b4159f40 2024-04-25T19:15:03.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.190+0000 7f34abfff640 1 -- 172.21.15.105:0/2167743314 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a403d640 con 0x7f34b4159f40 2024-04-25T19:15:03.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.190+0000 7f34b9a67640 1 -- 172.21.15.105:0/2167743314 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b4153930 con 0x7f34b4159f40 2024-04-25T19:15:03.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.190+0000 7f34abfff640 1 -- 172.21.15.105:0/2167743314 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4037040 con 0x7f34b4159f40 2024-04-25T19:15:03.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.190+0000 7f34b9a67640 1 -- 172.21.15.105:0/2167743314 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b41f44c0 con 0x7f34b4159f40 2024-04-25T19:15:03.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.190+0000 7f34abfff640 1 -- 172.21.15.105:0/2167743314 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f34a4044020 con 0x7f34b4159f40 2024-04-25T19:15:03.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.190+0000 7f34abfff640 1 -- 172.21.15.105:0/2167743314 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f34a403b3f0 con 0x7f34b4159f40 2024-04-25T19:15:03.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.191+0000 7f34b9a67640 1 -- 172.21.15.105:0/2167743314 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b406a390 con 0x7f34b4159f40 2024-04-25T19:15:03.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.193+0000 7f34abfff640 1 -- 172.21.15.105:0/2167743314 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f34a404b030 con 0x7f34b4159f40 2024-04-25T19:15:03.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.265+0000 7f34b9a67640 1 -- 172.21.15.105:0/2167743314 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5bea5166-1986-4d56-b5f5-ecc8cbe7f3ec", "id": 4} v 0) -- 0x7f34b4163ec0 con 0x7f34b4159f40 2024-04-25T19:15:03.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.271+0000 7f34abfff640 1 -- 172.21.15.105:0/2167743314 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5bea5166-1986-4d56-b5f5-ecc8cbe7f3ec", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f34a4046400 con 0x7f34b4159f40 2024-04-25T19:15:03.273 INFO:teuthology.orchestra.run.smithi105.stdout:4 2024-04-25T19:15:03.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.271+0000 7f34b9a67640 1 -- 172.21.15.105:0/2167743314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34b4159f40 msgr2=0x7f34b4152e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:03.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.271+0000 7f34b9a67640 1 --2- 172.21.15.105:0/2167743314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34b4159f40 0x7f34b4152e00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f34a4031000 tx=0x7f34a40046c0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.271+0000 7f34b9a67640 1 -- 172.21.15.105:0/2167743314 shutdown_connections 2024-04-25T19:15:03.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.271+0000 7f34b9a67640 1 --2- 172.21.15.105:0/2167743314 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34b415a350 0x7f34b414eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.271+0000 7f34b9a67640 1 --2- 172.21.15.105:0/2167743314 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34b4159f40 0x7f34b4152e00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.271+0000 7f34b9a67640 1 --2- 172.21.15.105:0/2167743314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34b40c60c0 0x7f34b41f4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.272+0000 7f34b9a67640 1 -- 172.21.15.105:0/2167743314 >> 172.21.15.105:0/2167743314 conn(0x7f34b4071a30 msgr2=0x7f34b4158000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:03.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.272+0000 7f34b9a67640 1 -- 172.21.15.105:0/2167743314 shutdown_connections 2024-04-25T19:15:03.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.272+0000 7f34b9a67640 1 -- 172.21.15.105:0/2167743314 wait complete. 2024-04-25T19:15:03.285 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new 96f89dcc-270d-4421-b8e8-3b92ba5f9844 5 2024-04-25T19:15:03.342 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-04-25T19:15:03.340+0000 7f653fe2f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T19:15:03.347 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:03.345+0000 7fbf6c5ee040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T19:15:03.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.390+0000 7f34201a8640 1 Processor -- start 2024-04-25T19:15:03.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.390+0000 7f34201a8640 1 -- start start 2024-04-25T19:15:03.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.391+0000 7f34201a8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3418154ee0 0x7f341815a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:03.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.391+0000 7f34201a8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f34180c98b0 0x7f341815df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:03.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.391+0000 7f34201a8640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34180c9cc0 con 0x7f34180684e0 2024-04-25T19:15:03.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.391+0000 7f34201a8640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f34180c8e60 con 0x7f34180c98b0 2024-04-25T19:15:03.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.392+0000 7f34201a8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f341815e470 con 0x7f3418154ee0 2024-04-25T19:15:03.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.392+0000 7f341df44640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3418154ee0 0x7f341815a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:03.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.392+0000 7f341df44640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3418154ee0 0x7f341815a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:49758/0 (socket says 172.21.15.105:49758) 2024-04-25T19:15:03.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.392+0000 7f341df44640 1 -- 172.21.15.105:0/1648287002 learned_addr learned my addr 172.21.15.105:0/1648287002 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-25T19:15:03.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.392+0000 7f341d743640 1 --2- 172.21.15.105:0/1648287002 >> v2:172.21.15.76:3300/0 conn(0x7f34180c98b0 0x7f341815df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:03.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.392+0000 7f341df44640 1 -- 172.21.15.105:0/1648287002 >> v2:172.21.15.76:3300/0 conn(0x7f34180c98b0 msgr2=0x7f341815df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:03.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.392+0000 7f341df44640 1 --2- 172.21.15.105:0/1648287002 >> v2:172.21.15.76:3300/0 conn(0x7f34180c98b0 0x7f341815df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.392+0000 7f341df44640 1 -- 172.21.15.105:0/1648287002 >> v1:172.21.15.76:6789/0 conn(0x7f34180684e0 legacy=0x7f3418074130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:03.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.392+0000 7f341df44640 1 -- 172.21.15.105:0/1648287002 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f341815e6c0 con 0x7f3418154ee0 2024-04-25T19:15:03.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.393+0000 7f341df44640 1 --2- 172.21.15.105:0/1648287002 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3418154ee0 0x7f341815a9f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3408009870 tx=0x7f340802a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6d313177bcbe7ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:03.395 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.393+0000 7f341cf42640 1 -- 172.21.15.105:0/1648287002 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408037070 con 0x7f3418154ee0 2024-04-25T19:15:03.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.393+0000 7f341cf42640 1 -- 172.21.15.105:0/1648287002 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f340802ae50 con 0x7f3418154ee0 2024-04-25T19:15:03.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.393+0000 7f341cf42640 1 -- 172.21.15.105:0/1648287002 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408032860 con 0x7f3418154ee0 2024-04-25T19:15:03.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.394+0000 7f34201a8640 1 -- 172.21.15.105:0/1648287002 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3418154ee0 msgr2=0x7f341815a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.394+0000 7f34201a8640 1 --2- 172.21.15.105:0/1648287002 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3418154ee0 0x7f341815a9f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3408009870 tx=0x7f340802a300 comp rx=0 tx=0).stop 2024-04-25T19:15:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.394+0000 7f34201a8640 1 -- 172.21.15.105:0/1648287002 shutdown_connections 2024-04-25T19:15:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.394+0000 7f34201a8640 1 --2- 172.21.15.105:0/1648287002 >> v2:172.21.15.76:3300/0 conn(0x7f34180c98b0 0x7f341815df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.394+0000 7f34201a8640 1 --2- 172.21.15.105:0/1648287002 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3418154ee0 0x7f341815a9f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.394+0000 7f34201a8640 1 -- 172.21.15.105:0/1648287002 >> 172.21.15.105:0/1648287002 conn(0x7f341806c5c0 msgr2=0x7f341806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.394+0000 7f34201a8640 1 -- 172.21.15.105:0/1648287002 shutdown_connections 2024-04-25T19:15:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.394+0000 7f34201a8640 1 -- 172.21.15.105:0/1648287002 wait complete. 2024-04-25T19:15:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.395+0000 7f34201a8640 1 Processor -- start 2024-04-25T19:15:03.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.395+0000 7f34201a8640 1 -- start start 2024-04-25T19:15:03.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.397+0000 7f34201a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34180684e0 0x7f3418150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:03.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.397+0000 7f34201a8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34180c98b0 0x7f341814ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:03.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.397+0000 7f34201a8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3418154ee0 0x7f34181e41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:03.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.397+0000 7f34201a8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f34180c90b0 con 0x7f3418154ee0 2024-04-25T19:15:03.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.397+0000 7f34201a8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f341815ec80 con 0x7f34180684e0 2024-04-25T19:15:03.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.397+0000 7f34201a8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f34180c9390 con 0x7f34180c98b0 2024-04-25T19:15:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.397+0000 7f341d743640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34180c98b0 0x7f341814ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.397+0000 7f341df44640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34180684e0 0x7f3418150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.397+0000 7f341d743640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34180c98b0 0x7f341814ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:49762/0 (socket says 172.21.15.105:49762) 2024-04-25T19:15:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.397+0000 7f341d743640 1 -- 172.21.15.105:0/1465579096 learned_addr learned my addr 172.21.15.105:0/1465579096 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-25T19:15:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.397+0000 7f341e745640 1 --2- 172.21.15.105:0/1465579096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3418154ee0 0x7f34181e41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.397+0000 7f341d743640 1 -- 172.21.15.105:0/1465579096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34180684e0 msgr2=0x7f3418150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.397+0000 7f341d743640 1 --2- 172.21.15.105:0/1465579096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34180684e0 0x7f3418150dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.397+0000 7f341d743640 1 -- 172.21.15.105:0/1465579096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3418154ee0 msgr2=0x7f34181e41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.397+0000 7f341d743640 1 --2- 172.21.15.105:0/1465579096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3418154ee0 0x7f34181e41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.397+0000 7f341d743640 1 -- 172.21.15.105:0/1465579096 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34181f0af0 con 0x7f34180c98b0 2024-04-25T19:15:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.397+0000 7f341df44640 1 --2- 172.21.15.105:0/1465579096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34180684e0 0x7f3418150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.397+0000 7f341d743640 1 --2- 172.21.15.105:0/1465579096 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34180c98b0 0x7f341814ca10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f341400e8a0 tx=0x7f341400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.397+0000 7f340e7fc640 1 -- 172.21.15.105:0/1465579096 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3414017070 con 0x7f34180c98b0 2024-04-25T19:15:03.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.397+0000 7f340e7fc640 1 -- 172.21.15.105:0/1465579096 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f341400c940 con 0x7f34180c98b0 2024-04-25T19:15:03.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.397+0000 7f340e7fc640 1 -- 172.21.15.105:0/1465579096 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3414009e60 con 0x7f34180c98b0 2024-04-25T19:15:03.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.397+0000 7f34201a8640 1 -- 172.21.15.105:0/1465579096 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34181f0dd0 con 0x7f34180c98b0 2024-04-25T19:15:03.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.397+0000 7f34201a8640 1 -- 172.21.15.105:0/1465579096 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f34181f1390 con 0x7f34180c98b0 2024-04-25T19:15:03.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.398+0000 7f340e7fc640 1 -- 172.21.15.105:0/1465579096 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f341401a070 con 0x7f34180c98b0 2024-04-25T19:15:03.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.398+0000 7f340e7fc640 1 -- 172.21.15.105:0/1465579096 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f341401c9d0 con 0x7f34180c98b0 2024-04-25T19:15:03.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.398+0000 7f34201a8640 1 -- 172.21.15.105:0/1465579096 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f341806d6a0 con 0x7f34180c98b0 2024-04-25T19:15:03.402 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-04-25T19:15:03.398+0000 7f653fe2f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T19:15:03.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.400+0000 7f340e7fc640 1 -- 172.21.15.105:0/1465579096 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f341400cdb0 con 0x7f34180c98b0 2024-04-25T19:15:03.403 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:03.401+0000 7fbf6c5ee040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T19:15:03.453 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-04-25T19:15:03.451+0000 7f653fe2f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T19:15:03.457 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:03.455+0000 7fbf6c5ee040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T19:15:03.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.473+0000 7f34201a8640 1 -- 172.21.15.105:0/1465579096 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd new", "uuid": "96f89dcc-270d-4421-b8e8-3b92ba5f9844", "id": 5} v 0) -- 0x7f341806a390 con 0x7f34180c98b0 2024-04-25T19:15:03.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.479+0000 7f340e7fc640 1 -- 172.21.15.105:0/1465579096 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "96f89dcc-270d-4421-b8e8-3b92ba5f9844", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f341401c520 con 0x7f34180c98b0 2024-04-25T19:15:03.481 INFO:teuthology.orchestra.run.smithi105.stdout:5 2024-04-25T19:15:03.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.479+0000 7f34201a8640 1 -- 172.21.15.105:0/1465579096 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34180c98b0 msgr2=0x7f341814ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:03.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.479+0000 7f34201a8640 1 --2- 172.21.15.105:0/1465579096 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34180c98b0 0x7f341814ca10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f341400e8a0 tx=0x7f341400ed60 comp rx=0 tx=0).stop 2024-04-25T19:15:03.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.479+0000 7f34201a8640 1 -- 172.21.15.105:0/1465579096 shutdown_connections 2024-04-25T19:15:03.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.479+0000 7f34201a8640 1 --2- 172.21.15.105:0/1465579096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3418154ee0 0x7f34181e41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.479+0000 7f34201a8640 1 --2- 172.21.15.105:0/1465579096 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34180c98b0 0x7f341814ca10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.479+0000 7f34201a8640 1 --2- 172.21.15.105:0/1465579096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34180684e0 0x7f3418150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.479+0000 7f34201a8640 1 -- 172.21.15.105:0/1465579096 >> 172.21.15.105:0/1465579096 conn(0x7f341806c5c0 msgr2=0x7f34180730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:03.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.480+0000 7f34201a8640 1 -- 172.21.15.105:0/1465579096 shutdown_connections 2024-04-25T19:15:03.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.480+0000 7f34201a8640 1 -- 172.21.15.105:0/1465579096 wait complete. 2024-04-25T19:15:03.492 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new 93eb087a-d9ba-4096-b85d-b74d96e819a0 6 2024-04-25T19:15:03.564 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-04-25T19:15:03.563+0000 7f653fe2f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T19:15:03.577 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:03.575+0000 7fbf6c5ee040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T19:15:03.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.597+0000 7f2f2ebf3640 1 Processor -- start 2024-04-25T19:15:03.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.597+0000 7f2f2ebf3640 1 -- start start 2024-04-25T19:15:03.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.599+0000 7f2f2ebf3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2f28159840 0x7f2f28153b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:03.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.599+0000 7f2f2ebf3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2f2815d880 0x7f2f280c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:03.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.599+0000 7f2f2ebf3640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f280715d0 con 0x7f2f280c5b20 2024-04-25T19:15:03.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.599+0000 7f2f2ebf3640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f2f2815ce30 con 0x7f2f28159840 2024-04-25T19:15:03.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.599+0000 7f2f2ebf3640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f2f280718b0 con 0x7f2f2815d880 2024-04-25T19:15:03.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.599+0000 7f2f2d190640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2f2815d880 0x7f2f280c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:03.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.599+0000 7f2f2d190640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2f2815d880 0x7f2f280c88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44810/0 (socket says 172.21.15.105:44810) 2024-04-25T19:15:03.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.599+0000 7f2f2d190640 1 -- 172.21.15.105:0/3812584694 learned_addr learned my addr 172.21.15.105:0/3812584694 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-25T19:15:03.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.599+0000 7f2f2c98f640 1 --2- 172.21.15.105:0/3812584694 >> v2:172.21.15.76:3300/0 conn(0x7f2f28159840 0x7f2f28153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:03.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.599+0000 7f2f2d190640 1 -- 172.21.15.105:0/3812584694 >> v2:172.21.15.76:3300/0 conn(0x7f2f28159840 msgr2=0x7f2f28153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:03.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.599+0000 7f2f2d190640 1 --2- 172.21.15.105:0/3812584694 >> v2:172.21.15.76:3300/0 conn(0x7f2f28159840 0x7f2f28153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.599+0000 7f2f2d190640 1 -- 172.21.15.105:0/3812584694 >> v1:172.21.15.105:6789/0 conn(0x7f2f280c5b20 legacy=0x7f2f280c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:03.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.599+0000 7f2f2d190640 1 -- 172.21.15.105:0/3812584694 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f28071b90 con 0x7f2f2815d880 2024-04-25T19:15:03.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.600+0000 7f2f2d190640 1 --2- 172.21.15.105:0/3812584694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f2815d880 0x7f2f280c88f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2f280c8810 tx=0x7f2f2002a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc87ff8f5719c0ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:03.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.600+0000 7f2f277fe640 1 -- 172.21.15.105:0/3812584694 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f20036070 con 0x7f2f2815d880 2024-04-25T19:15:03.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.600+0000 7f2f277fe640 1 -- 172.21.15.105:0/3812584694 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f20031600 con 0x7f2f2815d880 2024-04-25T19:15:03.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.600+0000 7f2f277fe640 1 -- 172.21.15.105:0/3812584694 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f200319d0 con 0x7f2f2815d880 2024-04-25T19:15:03.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.601+0000 7f2f2ebf3640 1 -- 172.21.15.105:0/3812584694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f2815d880 msgr2=0x7f2f280c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:03.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.601+0000 7f2f2ebf3640 1 --2- 172.21.15.105:0/3812584694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f2815d880 0x7f2f280c88f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2f280c8810 tx=0x7f2f2002a730 comp rx=0 tx=0).stop 2024-04-25T19:15:03.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.601+0000 7f2f2ebf3640 1 -- 172.21.15.105:0/3812584694 shutdown_connections 2024-04-25T19:15:03.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.601+0000 7f2f2ebf3640 1 --2- 172.21.15.105:0/3812584694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f2815d880 0x7f2f280c88f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.601+0000 7f2f2ebf3640 1 --2- 172.21.15.105:0/3812584694 >> v2:172.21.15.76:3300/0 conn(0x7f2f28159840 0x7f2f28153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.601+0000 7f2f2ebf3640 1 -- 172.21.15.105:0/3812584694 >> 172.21.15.105:0/3812584694 conn(0x7f2f280cecc0 msgr2=0x7f2f280cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:03.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.601+0000 7f2f2ebf3640 1 -- 172.21.15.105:0/3812584694 shutdown_connections 2024-04-25T19:15:03.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.601+0000 7f2f2ebf3640 1 -- 172.21.15.105:0/3812584694 wait complete. 2024-04-25T19:15:03.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.602+0000 7f2f2ebf3640 1 Processor -- start 2024-04-25T19:15:03.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.603+0000 7f2f2ebf3640 1 -- start start 2024-04-25T19:15:03.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.604+0000 7f2f2ebf3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f280c5b20 0x7f2f281e4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:03.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.604+0000 7f2f2ebf3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f28159840 0x7f2f28152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:03.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.604+0000 7f2f2ebf3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f2815d880 0x7f2f280ce370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:03.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.604+0000 7f2f2ebf3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2815dc90 con 0x7f2f28159840 2024-04-25T19:15:03.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.604+0000 7f2f2ebf3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2f28071270 con 0x7f2f2815d880 2024-04-25T19:15:03.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.604+0000 7f2f2ebf3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f2f280c8e30 con 0x7f2f280c5b20 2024-04-25T19:15:03.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.604+0000 7f2f2d190640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f2815d880 0x7f2f280ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:03.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.604+0000 7f2f2c98f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f280c5b20 0x7f2f281e4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:03.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.604+0000 7f2f2d190640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f2815d880 0x7f2f280ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44820/0 (socket says 172.21.15.105:44820) 2024-04-25T19:15:03.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.604+0000 7f2f2c98f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f280c5b20 0x7f2f281e4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:49772/0 (socket says 172.21.15.105:49772) 2024-04-25T19:15:03.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.604+0000 7f2f2d190640 1 -- 172.21.15.105:0/1705238650 learned_addr learned my addr 172.21.15.105:0/1705238650 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-25T19:15:03.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.604+0000 7f2f27fff640 1 --2- 172.21.15.105:0/1705238650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f28159840 0x7f2f28152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:03.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.604+0000 7f2f2c98f640 1 -- 172.21.15.105:0/1705238650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f2815d880 msgr2=0x7f2f280ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:03.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.604+0000 7f2f2c98f640 1 --2- 172.21.15.105:0/1705238650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f2815d880 0x7f2f280ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.604+0000 7f2f2c98f640 1 -- 172.21.15.105:0/1705238650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f28159840 msgr2=0x7f2f28152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:03.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.604+0000 7f2f2c98f640 1 --2- 172.21.15.105:0/1705238650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f28159840 0x7f2f28152ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.604+0000 7f2f2c98f640 1 -- 172.21.15.105:0/1705238650 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2814c650 con 0x7f2f280c5b20 2024-04-25T19:15:03.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.604+0000 7f2f2d190640 1 --2- 172.21.15.105:0/1705238650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f2815d880 0x7f2f280ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:03.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.605+0000 7f2f2c98f640 1 --2- 172.21.15.105:0/1705238650 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f280c5b20 0x7f2f281e4160 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00e870 tx=0x7f2f1c00ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:03.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.605+0000 7f2f257fa640 1 -- 172.21.15.105:0/1705238650 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c017070 con 0x7f2f280c5b20 2024-04-25T19:15:03.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.605+0000 7f2f257fa640 1 -- 172.21.15.105:0/1705238650 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f1c00ee60 con 0x7f2f280c5b20 2024-04-25T19:15:03.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.605+0000 7f2f257fa640 1 -- 172.21.15.105:0/1705238650 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c00cb60 con 0x7f2f280c5b20 2024-04-25T19:15:03.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.605+0000 7f2f2ebf3640 1 -- 172.21.15.105:0/1705238650 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f2814e110 con 0x7f2f280c5b20 2024-04-25T19:15:03.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.605+0000 7f2f2ebf3640 1 -- 172.21.15.105:0/1705238650 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f281f48a0 con 0x7f2f280c5b20 2024-04-25T19:15:03.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.605+0000 7f2f257fa640 1 -- 172.21.15.105:0/1705238650 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2f1c01a070 con 0x7f2f280c5b20 2024-04-25T19:15:03.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.605+0000 7f2f257fa640 1 -- 172.21.15.105:0/1705238650 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f2f1c01ca40 con 0x7f2f280c5b20 2024-04-25T19:15:03.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.606+0000 7f2f2ebf3640 1 -- 172.21.15.105:0/1705238650 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f2806a390 con 0x7f2f280c5b20 2024-04-25T19:15:03.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.608+0000 7f2f257fa640 1 -- 172.21.15.105:0/1705238650 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f2f1c00cda0 con 0x7f2f280c5b20 2024-04-25T19:15:03.683 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.681+0000 7f2f2ebf3640 1 -- 172.21.15.105:0/1705238650 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd new", "uuid": "93eb087a-d9ba-4096-b85d-b74d96e819a0", "id": 6} v 0) -- 0x7f2f281533e0 con 0x7f2f280c5b20 2024-04-25T19:15:03.689 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.687+0000 7f2f257fa640 1 -- 172.21.15.105:0/1705238650 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "93eb087a-d9ba-4096-b85d-b74d96e819a0", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f2f1c01c450 con 0x7f2f280c5b20 2024-04-25T19:15:03.689 INFO:teuthology.orchestra.run.smithi105.stdout:6 2024-04-25T19:15:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.688+0000 7f2f2ebf3640 1 -- 172.21.15.105:0/1705238650 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f280c5b20 msgr2=0x7f2f281e4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.688+0000 7f2f2ebf3640 1 --2- 172.21.15.105:0/1705238650 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f280c5b20 0x7f2f281e4160 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00e870 tx=0x7f2f1c00ed30 comp rx=0 tx=0).stop 2024-04-25T19:15:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.689+0000 7f2f2ebf3640 1 -- 172.21.15.105:0/1705238650 shutdown_connections 2024-04-25T19:15:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.689+0000 7f2f2ebf3640 1 --2- 172.21.15.105:0/1705238650 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2f2815d880 0x7f2f280ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.689+0000 7f2f2ebf3640 1 --2- 172.21.15.105:0/1705238650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f28159840 0x7f2f28152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.689+0000 7f2f2ebf3640 1 --2- 172.21.15.105:0/1705238650 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2f280c5b20 0x7f2f281e4160 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.689+0000 7f2f2ebf3640 1 -- 172.21.15.105:0/1705238650 >> 172.21.15.105:0/1705238650 conn(0x7f2f280cecc0 msgr2=0x7f2f280c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.689+0000 7f2f2ebf3640 1 -- 172.21.15.105:0/1705238650 shutdown_connections 2024-04-25T19:15:03.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.689+0000 7f2f2ebf3640 1 -- 172.21.15.105:0/1705238650 wait complete. 2024-04-25T19:15:03.705 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new 2f80f790-ad2d-460e-a37d-47bf1771ef43 7 2024-04-25T19:15:03.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.813+0000 7f0212b5f640 1 Processor -- start 2024-04-25T19:15:03.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.813+0000 7f0212b5f640 1 -- start start 2024-04-25T19:15:03.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.814+0000 7f0212b5f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f020c0c60c0 0x7f020c14ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:03.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.814+0000 7f0212b5f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f020c15a350 0x7f020c1640e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:03.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.814+0000 7f0212b5f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f020c159760 con 0x7f020c159f40 2024-04-25T19:15:03.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.814+0000 7f0212b5f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f020c0c83d0 con 0x7f020c0c60c0 2024-04-25T19:15:03.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.814+0000 7f0212b5f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f020c0c6ae0 con 0x7f020c15a350 2024-04-25T19:15:03.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.814+0000 7f02110fc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f020c15a350 0x7f020c1640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:03.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.814+0000 7f02108fb640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f020c0c60c0 0x7f020c14ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:03.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.814+0000 7f02110fc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f020c15a350 0x7f020c1640e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:49782/0 (socket says 172.21.15.105:49782) 2024-04-25T19:15:03.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.814+0000 7f02110fc640 1 -- 172.21.15.105:0/1546543413 learned_addr learned my addr 172.21.15.105:0/1546543413 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-25T19:15:03.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.814+0000 7f02110fc640 1 -- 172.21.15.105:0/1546543413 >> v2:172.21.15.105:3300/0 conn(0x7f020c0c60c0 msgr2=0x7f020c14ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:03.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.814+0000 7f02110fc640 1 --2- 172.21.15.105:0/1546543413 >> v2:172.21.15.105:3300/0 conn(0x7f020c0c60c0 0x7f020c14ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.814+0000 7f02110fc640 1 -- 172.21.15.105:0/1546543413 >> v1:172.21.15.76:6789/0 conn(0x7f020c159f40 legacy=0x7f020c0c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:03.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.814+0000 7f02110fc640 1 -- 172.21.15.105:0/1546543413 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020c0c6dc0 con 0x7f020c15a350 2024-04-25T19:15:03.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.815+0000 7f02108fb640 1 --2- 172.21.15.105:0/1546543413 >> v2:172.21.15.105:3300/0 conn(0x7f020c0c60c0 0x7f020c14ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:03.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.815+0000 7f02110fc640 1 --2- 172.21.15.105:0/1546543413 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f020c15a350 0x7f020c1640e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0204009e50 tx=0x7f020402a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa760d884f109601 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:03.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.815+0000 7f020b7fe640 1 -- 172.21.15.105:0/1546543413 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0204036070 con 0x7f020c15a350 2024-04-25T19:15:03.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.815+0000 7f020b7fe640 1 -- 172.21.15.105:0/1546543413 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0204031600 con 0x7f020c15a350 2024-04-25T19:15:03.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.815+0000 7f020b7fe640 1 -- 172.21.15.105:0/1546543413 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02040319d0 con 0x7f020c15a350 2024-04-25T19:15:03.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.816+0000 7f0212b5f640 1 -- 172.21.15.105:0/1546543413 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f020c15a350 msgr2=0x7f020c1640e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:03.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.816+0000 7f0212b5f640 1 --2- 172.21.15.105:0/1546543413 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f020c15a350 0x7f020c1640e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0204009e50 tx=0x7f020402a680 comp rx=0 tx=0).stop 2024-04-25T19:15:03.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.816+0000 7f0212b5f640 1 -- 172.21.15.105:0/1546543413 shutdown_connections 2024-04-25T19:15:03.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.816+0000 7f0212b5f640 1 --2- 172.21.15.105:0/1546543413 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f020c15a350 0x7f020c1640e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.816+0000 7f0212b5f640 1 --2- 172.21.15.105:0/1546543413 >> v2:172.21.15.105:3300/0 conn(0x7f020c0c60c0 0x7f020c14ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.816+0000 7f0212b5f640 1 -- 172.21.15.105:0/1546543413 >> 172.21.15.105:0/1546543413 conn(0x7f020c071a30 msgr2=0x7f020c071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:03.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.816+0000 7f0212b5f640 1 -- 172.21.15.105:0/1546543413 shutdown_connections 2024-04-25T19:15:03.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.816+0000 7f0212b5f640 1 -- 172.21.15.105:0/1546543413 wait complete. 2024-04-25T19:15:03.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.818+0000 7f0212b5f640 1 Processor -- start 2024-04-25T19:15:03.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.818+0000 7f0212b5f640 1 -- start start 2024-04-25T19:15:03.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.819+0000 7f0212b5f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f020c0c60c0 0x7f020c0ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:03.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.819+0000 7f0212b5f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f020c159f40 0x7f020c14b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:03.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.819+0000 7f0212b5f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f020c15a350 0x7f020c0cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:03.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.819+0000 7f0212b5f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f020c15ac50 con 0x7f020c0c60c0 2024-04-25T19:15:03.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.819+0000 7f0212b5f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f020c159a20 con 0x7f020c15a350 2024-04-25T19:15:03.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.819+0000 7f0212b5f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f020c0c6600 con 0x7f020c159f40 2024-04-25T19:15:03.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.819+0000 7f02110fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f020c15a350 0x7f020c0cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:03.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.819+0000 7f020bfff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f020c159f40 0x7f020c14b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:03.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.819+0000 7f02110fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f020c15a350 0x7f020c0cd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:44834/0 (socket says 172.21.15.105:44834) 2024-04-25T19:15:03.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.819+0000 7f020bfff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f020c159f40 0x7f020c14b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.105:49798/0 (socket says 172.21.15.105:49798) 2024-04-25T19:15:03.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.819+0000 7f02110fc640 1 -- 172.21.15.105:0/1323844213 learned_addr learned my addr 172.21.15.105:0/1323844213 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-04-25T19:15:03.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.819+0000 7f02108fb640 1 --2- 172.21.15.105:0/1323844213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f020c0c60c0 0x7f020c0ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:03.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.820+0000 7f020bfff640 1 -- 172.21.15.105:0/1323844213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f020c15a350 msgr2=0x7f020c0cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:03.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.820+0000 7f020bfff640 1 --2- 172.21.15.105:0/1323844213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f020c15a350 0x7f020c0cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.820+0000 7f020bfff640 1 -- 172.21.15.105:0/1323844213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f020c0c60c0 msgr2=0x7f020c0ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:03.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.820+0000 7f020bfff640 1 --2- 172.21.15.105:0/1323844213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f020c0c60c0 0x7f020c0ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.820+0000 7f020bfff640 1 -- 172.21.15.105:0/1323844213 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020c0ca520 con 0x7f020c159f40 2024-04-25T19:15:03.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.820+0000 7f02110fc640 1 --2- 172.21.15.105:0/1323844213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f020c15a350 0x7f020c0cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:03.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.820+0000 7f020bfff640 1 --2- 172.21.15.105:0/1323844213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f020c159f40 0x7f020c14b840 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f01f800c390 tx=0x7f01f800c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:03.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.820+0000 7f02097fa640 1 -- 172.21.15.105:0/1323844213 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f80052d0 con 0x7f020c159f40 2024-04-25T19:15:03.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.820+0000 7f02097fa640 1 -- 172.21.15.105:0/1323844213 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01f8005490 con 0x7f020c159f40 2024-04-25T19:15:03.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.820+0000 7f02097fa640 1 -- 172.21.15.105:0/1323844213 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8007260 con 0x7f020c159f40 2024-04-25T19:15:03.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.820+0000 7f0212b5f640 1 -- 172.21.15.105:0/1323844213 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f020c1e4920 con 0x7f020c159f40 2024-04-25T19:15:03.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.820+0000 7f0212b5f640 1 -- 172.21.15.105:0/1323844213 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f020c1e4ff0 con 0x7f020c159f40 2024-04-25T19:15:03.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.820+0000 7f02097fa640 1 -- 172.21.15.105:0/1323844213 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f01f8007400 con 0x7f020c159f40 2024-04-25T19:15:03.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.820+0000 7f02097fa640 1 -- 172.21.15.105:0/1323844213 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f01f8011bf0 con 0x7f020c159f40 2024-04-25T19:15:03.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.821+0000 7f0212b5f640 1 -- 172.21.15.105:0/1323844213 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f020c06a390 con 0x7f020c159f40 2024-04-25T19:15:03.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.823+0000 7f02097fa640 1 -- 172.21.15.105:0/1323844213 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f01f80039b0 con 0x7f020c159f40 2024-04-25T19:15:03.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.896+0000 7f0212b5f640 1 -- 172.21.15.105:0/1323844213 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2f80f790-ad2d-460e-a37d-47bf1771ef43", "id": 7} v 0) -- 0x7f020c1e4d10 con 0x7f020c159f40 2024-04-25T19:15:03.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.902+0000 7f02097fa640 1 -- 172.21.15.105:0/1323844213 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2f80f790-ad2d-460e-a37d-47bf1771ef43", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f01f8003b90 con 0x7f020c159f40 2024-04-25T19:15:03.904 INFO:teuthology.orchestra.run.smithi105.stdout:7 2024-04-25T19:15:03.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.903+0000 7f0212b5f640 1 -- 172.21.15.105:0/1323844213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f020c159f40 msgr2=0x7f020c14b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:03.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.903+0000 7f0212b5f640 1 --2- 172.21.15.105:0/1323844213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f020c159f40 0x7f020c14b840 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f01f800c390 tx=0x7f01f800c850 comp rx=0 tx=0).stop 2024-04-25T19:15:03.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.903+0000 7f0212b5f640 1 -- 172.21.15.105:0/1323844213 shutdown_connections 2024-04-25T19:15:03.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.903+0000 7f0212b5f640 1 --2- 172.21.15.105:0/1323844213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f020c15a350 0x7f020c0cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.903+0000 7f0212b5f640 1 --2- 172.21.15.105:0/1323844213 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f020c159f40 0x7f020c14b840 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.903+0000 7f0212b5f640 1 --2- 172.21.15.105:0/1323844213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f020c0c60c0 0x7f020c0ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:03.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.903+0000 7f0212b5f640 1 -- 172.21.15.105:0/1323844213 >> 172.21.15.105:0/1323844213 conn(0x7f020c071a30 msgr2=0x7f020c157de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:03.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.903+0000 7f0212b5f640 1 -- 172.21.15.105:0/1323844213 shutdown_connections 2024-04-25T19:15:03.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-04-25T19:15:03.903+0000 7f0212b5f640 1 -- 172.21.15.105:0/1323844213 wait complete. 2024-04-25T19:15:03.916 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T19:15:03.916 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T19:15:03.918 INFO:tasks.ceph.osd.0:Started 2024-04-25T19:15:03.918 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T19:15:03.918 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T19:15:03.922 INFO:tasks.ceph.osd.1:Started 2024-04-25T19:15:03.922 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T19:15:03.922 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T19:15:03.924 INFO:tasks.ceph.osd.2:Started 2024-04-25T19:15:03.924 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T19:15:03.924 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T19:15:03.926 INFO:tasks.ceph.osd.3:Started 2024-04-25T19:15:03.926 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T19:15:03.927 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T19:15:03.929 INFO:tasks.ceph.osd.4:Started 2024-04-25T19:15:03.929 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T19:15:03.929 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T19:15:03.931 INFO:tasks.ceph.osd.5:Started 2024-04-25T19:15:03.932 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T19:15:03.932 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T19:15:03.934 INFO:tasks.ceph.osd.6:Started 2024-04-25T19:15:03.934 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T19:15:03.934 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T19:15:03.936 INFO:tasks.ceph.osd.7:Started 2024-04-25T19:15:03.936 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:15:03.963 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-04-25T19:15:03.962+0000 7f653fe2f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T19:15:03.990 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:03.988+0000 7fbf6c5ee040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T19:15:04.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.070+0000 7f05aef26640 1 Processor -- start 2024-04-25T19:15:04.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.070+0000 7f05aef26640 1 -- start start 2024-04-25T19:15:04.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.071+0000 7f05aef26640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05a80c8440 0x7f05a80c8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:04.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.071+0000 7f05aef26640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05a806c7b0 con 0x7f05a80714c0 2024-04-25T19:15:04.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.072+0000 7f05aef26640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05a80ca230 con 0x7f05a806d120 2024-04-25T19:15:04.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.072+0000 7f05aef26640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f05a80cb490 con 0x7f05a80c8440 2024-04-25T19:15:04.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.072+0000 7f05accc2640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f05a80714c0 0x7f05a8154a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45456/0 (socket says 172.21.15.76:45456) 2024-04-25T19:15:04.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.072+0000 7f05accc2640 1 -- 172.21.15.76:0/3092907907 learned_addr learned my addr 172.21.15.76:0/3092907907 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:04.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.072+0000 7f05ad4c3640 1 --2- 172.21.15.76:0/3092907907 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05a80c8440 0x7f05a80c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:04.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.072+0000 7f05a77fe640 1 -- 172.21.15.76:0/3092907907 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4159298992 0 0) 0x7f05a806c7b0 con 0x7f05a80714c0 2024-04-25T19:15:04.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.072+0000 7f05a77fe640 1 -- 172.21.15.76:0/3092907907 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0590003520 con 0x7f05a80714c0 2024-04-25T19:15:04.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.072+0000 7f05a77fe640 1 -- 172.21.15.76:0/3092907907 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1431131613 0 0) 0x7f05a80ca230 con 0x7f05a806d120 2024-04-25T19:15:04.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.072+0000 7f05a77fe640 1 -- 172.21.15.76:0/3092907907 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05a806c7b0 con 0x7f05a806d120 2024-04-25T19:15:04.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.072+0000 7f05ad4c3640 1 -- 172.21.15.76:0/3092907907 >> v1:172.21.15.105:6789/0 conn(0x7f05a806d120 legacy=0x7f05a815e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:04.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.072+0000 7f05ad4c3640 1 -- 172.21.15.76:0/3092907907 >> v1:172.21.15.76:6789/0 conn(0x7f05a80714c0 legacy=0x7f05a8154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:04.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.072+0000 7f05ad4c3640 1 -- 172.21.15.76:0/3092907907 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a80ca510 con 0x7f05a80c8440 2024-04-25T19:15:04.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.073+0000 7f05ad4c3640 1 --2- 172.21.15.76:0/3092907907 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05a80c8440 0x7f05a80c8850 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f05980097d0 tx=0x7f059802a2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15f3f09caca43c8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:04.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.073+0000 7f05a77fe640 1 -- 172.21.15.76:0/3092907907 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598037070 con 0x7f05a80c8440 2024-04-25T19:15:04.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.073+0000 7f05a77fe640 1 -- 172.21.15.76:0/3092907907 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0598032440 con 0x7f05a80c8440 2024-04-25T19:15:04.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.073+0000 7f05a77fe640 1 -- 172.21.15.76:0/3092907907 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05980327f0 con 0x7f05a80c8440 2024-04-25T19:15:04.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.074+0000 7f05aef26640 1 -- 172.21.15.76:0/3092907907 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05a80c8440 msgr2=0x7f05a80c8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:04.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.074+0000 7f05aef26640 1 --2- 172.21.15.76:0/3092907907 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05a80c8440 0x7f05a80c8850 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f05980097d0 tx=0x7f059802a2a0 comp rx=0 tx=0).stop 2024-04-25T19:15:04.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.074+0000 7f05aef26640 1 -- 172.21.15.76:0/3092907907 shutdown_connections 2024-04-25T19:15:04.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.074+0000 7f05aef26640 1 --2- 172.21.15.76:0/3092907907 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05a80c8440 0x7f05a80c8850 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:04.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.074+0000 7f05aef26640 1 -- 172.21.15.76:0/3092907907 >> 172.21.15.76:0/3092907907 conn(0x7f05a806d6d0 msgr2=0x7f05a806dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:04.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.074+0000 7f05aef26640 1 -- 172.21.15.76:0/3092907907 shutdown_connections 2024-04-25T19:15:04.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.074+0000 7f05aef26640 1 -- 172.21.15.76:0/3092907907 wait complete. 2024-04-25T19:15:04.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.075+0000 7f05aef26640 1 Processor -- start 2024-04-25T19:15:04.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.075+0000 7f05aef26640 1 -- start start 2024-04-25T19:15:04.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.077+0000 7f05aef26640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05a806d120 0x7f05a80ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:04.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.077+0000 7f05aef26640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05a80714c0 0x7f05a81f13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:04.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.077+0000 7f05aef26640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05a80c8440 0x7f05a81449d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:04.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.077+0000 7f05aef26640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f05a80724b0 con 0x7f05a806d120 2024-04-25T19:15:04.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.077+0000 7f05aef26640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f05a80ca7f0 con 0x7f05a80714c0 2024-04-25T19:15:04.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.077+0000 7f05aef26640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f05a815eb90 con 0x7f05a80c8440 2024-04-25T19:15:04.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.077+0000 7f05accc2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05a806d120 0x7f05a80ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:04.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.077+0000 7f05accc2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05a806d120 0x7f05a80ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43834/0 (socket says 172.21.15.76:43834) 2024-04-25T19:15:04.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.077+0000 7f05accc2640 1 -- 172.21.15.76:0/2741568512 learned_addr learned my addr 172.21.15.76:0/2741568512 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:04.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.077+0000 7f05ad4c3640 1 --2- 172.21.15.76:0/2741568512 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05a80c8440 0x7f05a81449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:04.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.077+0000 7f05a7fff640 1 --2- 172.21.15.76:0/2741568512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05a80714c0 0x7f05a81f13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:04.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.077+0000 7f05accc2640 1 -- 172.21.15.76:0/2741568512 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05a80c8440 msgr2=0x7f05a81449d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:04.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.077+0000 7f05accc2640 1 --2- 172.21.15.76:0/2741568512 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05a80c8440 0x7f05a81449d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:04.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.077+0000 7f05accc2640 1 -- 172.21.15.76:0/2741568512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05a80714c0 msgr2=0x7f05a81f13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:04.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.077+0000 7f05accc2640 1 --2- 172.21.15.76:0/2741568512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05a80714c0 0x7f05a81f13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:04.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.077+0000 7f05accc2640 1 -- 172.21.15.76:0/2741568512 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a815d9a0 con 0x7f05a806d120 2024-04-25T19:15:04.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.077+0000 7f05ad4c3640 1 --2- 172.21.15.76:0/2741568512 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05a80c8440 0x7f05a81449d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:04.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.077+0000 7f05accc2640 1 --2- 172.21.15.76:0/2741568512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05a806d120 0x7f05a80ccb30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f05a000cb50 tx=0x7f05a000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:04.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.078+0000 7f05a57fa640 1 -- 172.21.15.76:0/2741568512 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a0017070 con 0x7f05a806d120 2024-04-25T19:15:04.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.078+0000 7f05a57fa640 1 -- 172.21.15.76:0/2741568512 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05a000ad40 con 0x7f05a806d120 2024-04-25T19:15:04.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.078+0000 7f05aef26640 1 -- 172.21.15.76:0/2741568512 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a80ce730 con 0x7f05a806d120 2024-04-25T19:15:04.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.078+0000 7f05a57fa640 1 -- 172.21.15.76:0/2741568512 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a0012620 con 0x7f05a806d120 2024-04-25T19:15:04.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.078+0000 7f05aef26640 1 -- 172.21.15.76:0/2741568512 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a81f5410 con 0x7f05a806d120 2024-04-25T19:15:04.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.078+0000 7f05a57fa640 1 -- 172.21.15.76:0/2741568512 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f05a001a070 con 0x7f05a806d120 2024-04-25T19:15:04.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.078+0000 7f05a57fa640 1 -- 172.21.15.76:0/2741568512 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f05a0007740 con 0x7f05a806d120 2024-04-25T19:15:04.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.079+0000 7f05aef26640 1 -- 172.21.15.76:0/2741568512 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05a806a390 con 0x7f05a806d120 2024-04-25T19:15:04.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.081+0000 7f05a57fa640 1 -- 172.21.15.76:0/2741568512 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f05a0021070 con 0x7f05a806d120 2024-04-25T19:15:04.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.143+0000 7f05aef26640 1 -- 172.21.15.76:0/2741568512 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05a8158250 con 0x7f05a806d120 2024-04-25T19:15:04.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.144+0000 7f05a57fa640 1 -- 172.21.15.76:0/2741568512 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f05a815d9a0 con 0x7f05a806d120 2024-04-25T19:15:04.145 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:04.146 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":10,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","created":"2024-04-25T19:15:01.693137+0000","modified":"2024-04-25T19:15:03.898065+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T19:15:03.898065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"72a16c02-48f9-45aa-9447-fcf755ebf936","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"818d50f0-76c6-45f8-bf79-5e304f14c5cf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5714c8f5-ae23-473c-8cf1-db73811acded","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"13204cdb-d8e6-4f97-bc6e-9561f442decf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5bea5166-1986-4d56-b5f5-ecc8cbe7f3ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"96f89dcc-270d-4421-b8e8-3b92ba5f9844","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"93eb087a-d9ba-4096-b85d-b74d96e819a0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2f80f790-ad2d-460e-a37d-47bf1771ef43","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:15:04.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.144+0000 7f05aef26640 1 -- 172.21.15.76:0/2741568512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05a806d120 msgr2=0x7f05a80ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:04.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.144+0000 7f05aef26640 1 --2- 172.21.15.76:0/2741568512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05a806d120 0x7f05a80ccb30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f05a000cb50 tx=0x7f05a000a430 comp rx=0 tx=0).stop 2024-04-25T19:15:04.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.145+0000 7f05aef26640 1 -- 172.21.15.76:0/2741568512 shutdown_connections 2024-04-25T19:15:04.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.145+0000 7f05aef26640 1 --2- 172.21.15.76:0/2741568512 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05a80c8440 0x7f05a81449d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:04.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.145+0000 7f05aef26640 1 --2- 172.21.15.76:0/2741568512 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05a80714c0 0x7f05a81f13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:04.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.145+0000 7f05aef26640 1 --2- 172.21.15.76:0/2741568512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05a806d120 0x7f05a80ccb30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:04.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.145+0000 7f05aef26640 1 -- 172.21.15.76:0/2741568512 >> 172.21.15.76:0/2741568512 conn(0x7f05a806d6d0 msgr2=0x7f05a8153060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:04.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.145+0000 7f05aef26640 1 -- 172.21.15.76:0/2741568512 shutdown_connections 2024-04-25T19:15:04.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:04.145+0000 7f05aef26640 1 -- 172.21.15.76:0/2741568512 wait complete. 2024-04-25T19:15:04.157 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T19:15:04.157 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T19:15:04.258 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-04-25T19:15:04.257+0000 7f653fe2f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T19:15:04.270 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:04.268+0000 7fbf6c5ee040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T19:15:04.311 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-04-25T19:15:04.310+0000 7f653fe2f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T19:15:04.324 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:04.322+0000 7fbf6c5ee040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T19:15:04.429 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-04-25T19:15:04.428+0000 7f653fe2f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T19:15:04.442 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:04.440+0000 7fbf6c5ee040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T19:15:04.552 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-04-25T19:15:04.550+0000 7fda27a1c640 -1 Falling back to public interface 2024-04-25T19:15:04.557 INFO:tasks.ceph.osd.4.smithi105.stderr:2024-04-25T19:15:04.555+0000 7fb9adb1a640 -1 Falling back to public interface 2024-04-25T19:15:04.561 INFO:tasks.ceph.osd.6.smithi105.stderr:2024-04-25T19:15:04.559+0000 7f9f740ee640 -1 Falling back to public interface 2024-04-25T19:15:04.563 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-04-25T19:15:04.561+0000 7f1e774e1640 -1 Falling back to public interface 2024-04-25T19:15:04.566 INFO:tasks.ceph.osd.7.smithi105.stderr:2024-04-25T19:15:04.565+0000 7fc86aa30640 -1 Falling back to public interface 2024-04-25T19:15:04.567 INFO:tasks.ceph.osd.5.smithi105.stderr:2024-04-25T19:15:04.565+0000 7fd6d0ba1640 -1 Falling back to public interface 2024-04-25T19:15:04.568 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-04-25T19:15:04.566+0000 7f76ce3da640 -1 Falling back to public interface 2024-04-25T19:15:04.574 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-04-25T19:15:04.572+0000 7fc0abe21640 -1 Falling back to public interface 2024-04-25T19:15:04.712 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-04-25T19:15:04.711+0000 7f653fe2f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T19:15:04.722 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:04.720+0000 7fbf6c5ee040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T19:15:04.760 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-04-25T19:15:04.758+0000 7f653fe2f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T19:15:04.769 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:04.767+0000 7fbf6c5ee040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T19:15:04.918 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-04-25T19:15:04.917+0000 7f653fe2f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T19:15:04.926 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:04.924+0000 7fbf6c5ee040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T19:15:04.961 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T19:15:05.059 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:05.057+0000 7fbf6c5ee040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T19:15:05.060 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-04-25T19:15:05.058+0000 7f653fe2f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T19:15:05.063 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.061+0000 7f7207692640 1 Processor -- start 2024-04-25T19:15:05.063 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.061+0000 7f7207692640 1 -- start start 2024-04-25T19:15:05.065 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.062+0000 7f7207692640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7200153660 0x7f7200160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:05.065 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.062+0000 7f7207692640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f72000c95c0 0x7f72000c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:05.065 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.062+0000 7f7207692640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7200073d30 con 0x7f720015d4c0 2024-04-25T19:15:05.065 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.062+0000 7f7207692640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f72000745a0 con 0x7f72000c95c0 2024-04-25T19:15:05.065 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.062+0000 7f7207692640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f720015d9b0 con 0x7f7200153660 2024-04-25T19:15:05.065 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.062+0000 7f7205c2f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f72000c95c0 0x7f72000c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:05.065 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.063+0000 7f720542e640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f720015d4c0 0x7f72000cf080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45470/0 (socket says 172.21.15.76:45470) 2024-04-25T19:15:05.065 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.063+0000 7f720542e640 1 -- 172.21.15.76:0/2282363329 learned_addr learned my addr 172.21.15.76:0/2282363329 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:05.065 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.063+0000 7f7205c2f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f72000c95c0 0x7f72000c9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43846/0 (socket says 172.21.15.76:43846) 2024-04-25T19:15:05.065 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.063+0000 7f7204c2d640 1 --2- 172.21.15.76:0/2282363329 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7200153660 0x7f7200160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:05.065 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.063+0000 7f7205c2f640 1 -- 172.21.15.76:0/2282363329 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7200153660 msgr2=0x7f7200160bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:05.065 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.063+0000 7f7205c2f640 1 --2- 172.21.15.76:0/2282363329 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7200153660 0x7f7200160bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.065 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.063+0000 7f7205c2f640 1 -- 172.21.15.76:0/2282363329 >> v1:172.21.15.76:6789/0 conn(0x7f720015d4c0 legacy=0x7f72000cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:05.065 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.063+0000 7f71f7fff640 1 -- 172.21.15.76:0/2282363329 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2414003105 0 0) 0x7f7200073d30 con 0x7f720015d4c0 2024-04-25T19:15:05.066 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.063+0000 7f7205c2f640 1 -- 172.21.15.76:0/2282363329 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7200073a50 con 0x7f72000c95c0 2024-04-25T19:15:05.067 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.064+0000 7f7205c2f640 1 --2- 172.21.15.76:0/2282363329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72000c95c0 0x7f72000c9d50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f71f8009d90 tx=0x7f71f802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb90a8edde460018 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:05.067 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.064+0000 7f71f7fff640 1 -- 172.21.15.76:0/2282363329 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8007740 con 0x7f72000c95c0 2024-04-25T19:15:05.067 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.064+0000 7f71f7fff640 1 -- 172.21.15.76:0/2282363329 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71f80078e0 con 0x7f72000c95c0 2024-04-25T19:15:05.067 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.064+0000 7f71f7fff640 1 -- 172.21.15.76:0/2282363329 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8007cb0 con 0x7f72000c95c0 2024-04-25T19:15:05.067 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.064+0000 7f7207692640 1 -- 172.21.15.76:0/2282363329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72000c95c0 msgr2=0x7f72000c9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:05.067 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.064+0000 7f7207692640 1 --2- 172.21.15.76:0/2282363329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72000c95c0 0x7f72000c9d50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f71f8009d90 tx=0x7f71f802a6f0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.067 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.064+0000 7f7207692640 1 -- 172.21.15.76:0/2282363329 shutdown_connections 2024-04-25T19:15:05.067 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.064+0000 7f7207692640 1 --2- 172.21.15.76:0/2282363329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72000c95c0 0x7f72000c9d50 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.067 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.064+0000 7f7207692640 1 --2- 172.21.15.76:0/2282363329 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7200153660 0x7f7200160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.067 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.064+0000 7f7207692640 1 -- 172.21.15.76:0/2282363329 >> 172.21.15.76:0/2282363329 conn(0x7f720006daf0 msgr2=0x7f720006df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:05.068 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.065+0000 7f7207692640 1 -- 172.21.15.76:0/2282363329 shutdown_connections 2024-04-25T19:15:05.068 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.065+0000 7f7207692640 1 -- 172.21.15.76:0/2282363329 wait complete. 2024-04-25T19:15:05.068 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.066+0000 7f7207692640 1 Processor -- start 2024-04-25T19:15:05.068 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.066+0000 7f7207692640 1 -- start start 2024-04-25T19:15:05.071 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.067+0000 7f7207692640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72000c95c0 0x7f72000c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:05.071 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.067+0000 7f7207692640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7200153660 0x7f72001f6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:05.071 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.067+0000 7f7207692640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f720015d4c0 0x7f72001e82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:05.071 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.068+0000 7f7207692640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7200160690 con 0x7f720015d4c0 2024-04-25T19:15:05.071 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.068+0000 7f7207692640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f72001603b0 con 0x7f72000c95c0 2024-04-25T19:15:05.071 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.068+0000 7f7207692640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f7200074000 con 0x7f7200153660 2024-04-25T19:15:05.071 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.068+0000 7f7205c2f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f720015d4c0 0x7f72001e82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:05.071 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.068+0000 7f7205c2f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f720015d4c0 0x7f72001e82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43854/0 (socket says 172.21.15.76:43854) 2024-04-25T19:15:05.072 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.068+0000 7f7205c2f640 1 -- 172.21.15.76:0/2713377975 learned_addr learned my addr 172.21.15.76:0/2713377975 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:05.072 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.068+0000 7f720542e640 1 --2- 172.21.15.76:0/2713377975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72000c95c0 0x7f72000c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:05.072 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.068+0000 7f7204c2d640 1 --2- 172.21.15.76:0/2713377975 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7200153660 0x7f72001f6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:05.072 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.068+0000 7f7205c2f640 1 -- 172.21.15.76:0/2713377975 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7200153660 msgr2=0x7f72001f6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:05.072 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.068+0000 7f7205c2f640 1 --2- 172.21.15.76:0/2713377975 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7200153660 0x7f72001f6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.072 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.068+0000 7f7205c2f640 1 -- 172.21.15.76:0/2713377975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72000c95c0 msgr2=0x7f72000c5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:05.072 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.068+0000 7f7205c2f640 1 --2- 172.21.15.76:0/2713377975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72000c95c0 0x7f72000c5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.072 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.068+0000 7f7205c2f640 1 -- 172.21.15.76:0/2713377975 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72001f6c60 con 0x7f720015d4c0 2024-04-25T19:15:05.072 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.068+0000 7f7205c2f640 1 --2- 172.21.15.76:0/2713377975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f720015d4c0 0x7f72001e82c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f71f8009d90 tx=0x7f71f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:05.072 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.068+0000 7f71f5ffb640 1 -- 172.21.15.76:0/2713377975 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8040030 con 0x7f720015d4c0 2024-04-25T19:15:05.072 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.068+0000 7f71f5ffb640 1 -- 172.21.15.76:0/2713377975 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71f80319c0 con 0x7f720015d4c0 2024-04-25T19:15:05.072 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.068+0000 7f71f5ffb640 1 -- 172.21.15.76:0/2713377975 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8009450 con 0x7f720015d4c0 2024-04-25T19:15:05.072 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.068+0000 7f7207692640 1 -- 172.21.15.76:0/2713377975 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72001f2b30 con 0x7f720015d4c0 2024-04-25T19:15:05.072 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.069+0000 7f7207692640 1 -- 172.21.15.76:0/2713377975 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7200158c60 con 0x7f720015d4c0 2024-04-25T19:15:05.073 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.069+0000 7f71f5ffb640 1 -- 172.21.15.76:0/2713377975 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f71f80457f0 con 0x7f720015d4c0 2024-04-25T19:15:05.073 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.069+0000 7f71f5ffb640 1 -- 172.21.15.76:0/2713377975 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f71f803a3a0 con 0x7f720015d4c0 2024-04-25T19:15:05.073 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.069+0000 7f7207692640 1 -- 172.21.15.76:0/2713377975 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7200077180 con 0x7f720015d4c0 2024-04-25T19:15:05.075 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.073+0000 7f71f5ffb640 1 -- 172.21.15.76:0/2713377975 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f71f8007840 con 0x7f720015d4c0 2024-04-25T19:15:05.110 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:05.108+0000 7fbf6c5ee040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T19:15:05.113 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-04-25T19:15:05.112+0000 7f653fe2f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T19:15:05.136 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.135+0000 7f7207692640 1 -- 172.21.15.76:0/2713377975 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7200074310 con 0x7f720015d4c0 2024-04-25T19:15:05.137 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.135+0000 7f71f5ffb640 1 -- 172.21.15.76:0/2713377975 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f71f8031d90 con 0x7f720015d4c0 2024-04-25T19:15:05.137 INFO:teuthology.misc.health.smithi076.stdout: 2024-04-25T19:15:05.137 INFO:teuthology.misc.health.smithi076.stdout:{"epoch":10,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","created":"2024-04-25T19:15:01.693137+0000","modified":"2024-04-25T19:15:03.898065+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T19:15:03.898065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"72a16c02-48f9-45aa-9447-fcf755ebf936","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"818d50f0-76c6-45f8-bf79-5e304f14c5cf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5714c8f5-ae23-473c-8cf1-db73811acded","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"13204cdb-d8e6-4f97-bc6e-9561f442decf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5bea5166-1986-4d56-b5f5-ecc8cbe7f3ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"96f89dcc-270d-4421-b8e8-3b92ba5f9844","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"93eb087a-d9ba-4096-b85d-b74d96e819a0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2f80f790-ad2d-460e-a37d-47bf1771ef43","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:15:05.138 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.136+0000 7f7207692640 1 -- 172.21.15.76:0/2713377975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f720015d4c0 msgr2=0x7f72001e82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:05.138 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.136+0000 7f7207692640 1 --2- 172.21.15.76:0/2713377975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f720015d4c0 0x7f72001e82c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f71f8009d90 tx=0x7f71f8004a40 comp rx=0 tx=0).stop 2024-04-25T19:15:05.138 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.136+0000 7f7207692640 1 -- 172.21.15.76:0/2713377975 shutdown_connections 2024-04-25T19:15:05.138 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.136+0000 7f7207692640 1 --2- 172.21.15.76:0/2713377975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f720015d4c0 0x7f72001e82c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.138 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.136+0000 7f7207692640 1 --2- 172.21.15.76:0/2713377975 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7200153660 0x7f72001f6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.138 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.136+0000 7f7207692640 1 --2- 172.21.15.76:0/2713377975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72000c95c0 0x7f72000c5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:05.138 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.136+0000 7f7207692640 1 -- 172.21.15.76:0/2713377975 >> 172.21.15.76:0/2713377975 conn(0x7f720006daf0 msgr2=0x7f72000c8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:05.138 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.136+0000 7f7207692640 1 -- 172.21.15.76:0/2713377975 shutdown_connections 2024-04-25T19:15:05.139 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:05.136+0000 7f7207692640 1 -- 172.21.15.76:0/2713377975 wait complete. 2024-04-25T19:15:05.148 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T19:15:05.181 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:05.179+0000 7fbf6c5ee040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T19:15:05.186 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-04-25T19:15:05.185+0000 7f653fe2f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T19:15:05.226 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:05.224+0000 7fbf6c5ee040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T19:15:05.232 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-04-25T19:15:05.231+0000 7f653fe2f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T19:15:05.700 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:05.698+0000 7fbf6c5ee040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T19:15:05.708 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-04-25T19:15:05.707+0000 7f653fe2f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T19:15:05.816 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:05.814+0000 7fbf6c5ee040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T19:15:05.825 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-04-25T19:15:05.824+0000 7f653fe2f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T19:15:06.832 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:06.830+0000 7fbf0d07a640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T19:15:06.832 INFO:tasks.ceph.mgr.x.smithi105.stderr:2024-04-25T19:15:06.830+0000 7fbf0d07a640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T19:15:08.018 INFO:tasks.ceph.osd.4.smithi105.stderr:2024-04-25T19:15:08.016+0000 7fb9adb1a640 -1 osd.4 0 log_to_monitors true 2024-04-25T19:15:08.043 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-04-25T19:15:08.042+0000 7fda27a1c640 -1 osd.1 0 log_to_monitors true 2024-04-25T19:15:08.087 INFO:tasks.ceph.osd.6.smithi105.stderr:2024-04-25T19:15:08.085+0000 7f9f740ee640 -1 osd.6 0 log_to_monitors true 2024-04-25T19:15:08.095 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-04-25T19:15:08.094+0000 7f1e774e1640 -1 osd.2 0 log_to_monitors true 2024-04-25T19:15:08.096 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-04-25T19:15:08.095+0000 7fc0abe21640 -1 osd.3 0 log_to_monitors true 2024-04-25T19:15:08.100 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-04-25T19:15:08.099+0000 7f76ce3da640 -1 osd.0 0 log_to_monitors true 2024-04-25T19:15:08.115 INFO:tasks.ceph.osd.5.smithi105.stderr:2024-04-25T19:15:08.114+0000 7fd6d0ba1640 -1 osd.5 0 log_to_monitors true 2024-04-25T19:15:08.121 INFO:tasks.ceph.osd.7.smithi105.stderr:2024-04-25T19:15:08.119+0000 7fc86aa30640 -1 osd.7 0 log_to_monitors true 2024-04-25T19:15:10.887 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-04-25T19:15:10.885+0000 7f76ca370640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T19:15:10.900 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-04-25T19:15:10.898+0000 7fda233a0640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T19:15:10.900 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-04-25T19:15:10.899+0000 7f76c216b640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T19:15:10.906 INFO:tasks.ceph.osd.4.smithi105.stderr:2024-04-25T19:15:10.904+0000 7fb9aa2c4640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T19:15:10.909 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-04-25T19:15:10.907+0000 7f1e73477640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T19:15:10.909 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-04-25T19:15:10.908+0000 7fc0a7db7640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T19:15:10.911 INFO:tasks.ceph.osd.7.smithi105.stderr:2024-04-25T19:15:10.909+0000 7fc8669c6640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T19:15:10.912 INFO:tasks.ceph.osd.5.smithi105.stderr:2024-04-25T19:15:10.910+0000 7fd6ccb37640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T19:15:10.915 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-04-25T19:15:10.914+0000 7fda1b19b640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T19:15:10.919 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-04-25T19:15:10.918+0000 7fc09fbb2640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T19:15:10.921 INFO:tasks.ceph.osd.4.smithi105.stderr:2024-04-25T19:15:10.919+0000 7fb9a18ab640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T19:15:10.921 INFO:tasks.ceph.osd.7.smithi105.stderr:2024-04-25T19:15:10.919+0000 7fc85e7c1640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T19:15:10.922 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-04-25T19:15:10.921+0000 7f1e6b272640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T19:15:10.929 INFO:tasks.ceph.osd.5.smithi105.stderr:2024-04-25T19:15:10.928+0000 7fd6c4932640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T19:15:10.959 INFO:tasks.ceph.osd.6.smithi105.stderr:2024-04-25T19:15:10.958+0000 7f9f70084640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T19:15:10.969 INFO:tasks.ceph.osd.6.smithi105.stderr:2024-04-25T19:15:10.967+0000 7f9f67e7f640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T19:15:11.954 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T19:15:12.064 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.062+0000 7f7623182640 1 Processor -- start 2024-04-25T19:15:12.064 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.062+0000 7f7623182640 1 -- start start 2024-04-25T19:15:12.066 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.064+0000 7f7623182640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f761c06eaa0 0x7f761c156b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:12.066 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.064+0000 7f7623182640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f761c06d5a0 0x7f761c06d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:12.067 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.064+0000 7f7623182640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f761c0ce540 con 0x7f761c0c6ec0 2024-04-25T19:15:12.067 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.064+0000 7f7623182640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f761c0ce820 con 0x7f761c06d5a0 2024-04-25T19:15:12.067 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.064+0000 7f7623182640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f761c157070 con 0x7f761c06eaa0 2024-04-25T19:15:12.067 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.064+0000 7f762171f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f761c06d5a0 0x7f761c06d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:12.067 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.064+0000 7f762171f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f761c06d5a0 0x7f761c06d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60292/0 (socket says 172.21.15.76:60292) 2024-04-25T19:15:12.067 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.064+0000 7f762171f640 1 -- 172.21.15.76:0/367599095 learned_addr learned my addr 172.21.15.76:0/367599095 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:12.067 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.064+0000 7f7613fff640 1 --2- 172.21.15.76:0/367599095 >> v2:172.21.15.105:3300/0 conn(0x7f761c06eaa0 0x7f761c156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:12.067 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.064+0000 7f762171f640 1 -- 172.21.15.76:0/367599095 >> v2:172.21.15.105:3300/0 conn(0x7f761c06eaa0 msgr2=0x7f761c156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:12.067 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.064+0000 7f762171f640 1 --2- 172.21.15.76:0/367599095 >> v2:172.21.15.105:3300/0 conn(0x7f761c06eaa0 0x7f761c156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.067 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.064+0000 7f762171f640 1 -- 172.21.15.76:0/367599095 >> v1:172.21.15.105:6789/0 conn(0x7f761c0c6ec0 legacy=0x7f761c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:12.067 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.064+0000 7f762171f640 1 -- 172.21.15.76:0/367599095 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761c15d780 con 0x7f761c06d5a0 2024-04-25T19:15:12.067 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.065+0000 7f762171f640 1 --2- 172.21.15.76:0/367599095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f761c06d5a0 0x7f761c06d9b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f761c0d2be0 tx=0x7f761402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe26c6334ebb8a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:12.067 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.065+0000 7f76137fe640 1 -- 172.21.15.76:0/367599095 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7614036070 con 0x7f761c06d5a0 2024-04-25T19:15:12.068 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.065+0000 7f76137fe640 1 -- 172.21.15.76:0/367599095 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7614031600 con 0x7f761c06d5a0 2024-04-25T19:15:12.068 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.065+0000 7f76137fe640 1 -- 172.21.15.76:0/367599095 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76140319d0 con 0x7f761c06d5a0 2024-04-25T19:15:12.068 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.066+0000 7f7623182640 1 -- 172.21.15.76:0/367599095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f761c06d5a0 msgr2=0x7f761c06d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:12.069 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.066+0000 7f7623182640 1 --2- 172.21.15.76:0/367599095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f761c06d5a0 0x7f761c06d9b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f761c0d2be0 tx=0x7f761402a780 comp rx=0 tx=0).stop 2024-04-25T19:15:12.069 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.066+0000 7f7623182640 1 -- 172.21.15.76:0/367599095 shutdown_connections 2024-04-25T19:15:12.069 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.066+0000 7f7623182640 1 --2- 172.21.15.76:0/367599095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f761c06d5a0 0x7f761c06d9b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.069 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.066+0000 7f7623182640 1 --2- 172.21.15.76:0/367599095 >> v2:172.21.15.105:3300/0 conn(0x7f761c06eaa0 0x7f761c156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.069 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.066+0000 7f7623182640 1 -- 172.21.15.76:0/367599095 >> 172.21.15.76:0/367599095 conn(0x7f761c068ee0 msgr2=0x7f761c0d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:12.069 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.066+0000 7f7623182640 1 -- 172.21.15.76:0/367599095 shutdown_connections 2024-04-25T19:15:12.069 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.066+0000 7f7623182640 1 -- 172.21.15.76:0/367599095 wait complete. 2024-04-25T19:15:12.069 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.067+0000 7f7623182640 1 Processor -- start 2024-04-25T19:15:12.070 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.067+0000 7f7623182640 1 -- start start 2024-04-25T19:15:12.072 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.069+0000 7f7623182640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f761c06d5a0 0x7f761c153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:12.072 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.069+0000 7f7623182640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f761c06eaa0 0x7f761c15b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:12.072 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.069+0000 7f7623182640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f761c0c6ec0 0x7f761c15bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:12.072 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.069+0000 7f7623182640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f761c0ce250 con 0x7f761c06d5a0 2024-04-25T19:15:12.072 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.069+0000 7f7623182640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f761c0ccce0 con 0x7f761c06eaa0 2024-04-25T19:15:12.072 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.069+0000 7f7623182640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f761c15da60 con 0x7f761c0c6ec0 2024-04-25T19:15:12.073 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.069+0000 7f7620f1e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f761c06d5a0 0x7f761c153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:12.073 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.069+0000 7f7620f1e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f761c06d5a0 0x7f761c153230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60306/0 (socket says 172.21.15.76:60306) 2024-04-25T19:15:12.073 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.069+0000 7f7620f1e640 1 -- 172.21.15.76:0/80752080 learned_addr learned my addr 172.21.15.76:0/80752080 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:12.073 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.069+0000 7f7613fff640 1 --2- 172.21.15.76:0/80752080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f761c06eaa0 0x7f761c15b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:12.073 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.069+0000 7f762171f640 1 --2- 172.21.15.76:0/80752080 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f761c0c6ec0 0x7f761c15bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:12.073 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.069+0000 7f7620f1e640 1 -- 172.21.15.76:0/80752080 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f761c0c6ec0 msgr2=0x7f761c15bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:12.073 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.069+0000 7f7620f1e640 1 --2- 172.21.15.76:0/80752080 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f761c0c6ec0 0x7f761c15bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.073 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.069+0000 7f7620f1e640 1 -- 172.21.15.76:0/80752080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f761c06eaa0 msgr2=0x7f761c15b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:12.073 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.069+0000 7f7620f1e640 1 --2- 172.21.15.76:0/80752080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f761c06eaa0 0x7f761c15b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.073 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.069+0000 7f7620f1e640 1 -- 172.21.15.76:0/80752080 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761c153770 con 0x7f761c06d5a0 2024-04-25T19:15:12.073 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.070+0000 7f7620f1e640 1 --2- 172.21.15.76:0/80752080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f761c06d5a0 0x7f761c153230 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f761800b420 tx=0x7f761800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:12.073 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.070+0000 7f76117fa640 1 -- 172.21.15.76:0/80752080 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618011830 con 0x7f761c06d5a0 2024-04-25T19:15:12.073 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.070+0000 7f76117fa640 1 -- 172.21.15.76:0/80752080 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7618016070 con 0x7f761c06d5a0 2024-04-25T19:15:12.073 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.070+0000 7f7623182640 1 -- 172.21.15.76:0/80752080 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f761c1521d0 con 0x7f761c06d5a0 2024-04-25T19:15:12.073 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.070+0000 7f76117fa640 1 -- 172.21.15.76:0/80752080 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618011c10 con 0x7f761c06d5a0 2024-04-25T19:15:12.074 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.070+0000 7f7623182640 1 -- 172.21.15.76:0/80752080 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f761c153980 con 0x7f761c06d5a0 2024-04-25T19:15:12.074 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.071+0000 7f7623182640 1 -- 172.21.15.76:0/80752080 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f761c077180 con 0x7f761c06d5a0 2024-04-25T19:15:12.074 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.071+0000 7f76117fa640 1 -- 172.21.15.76:0/80752080 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 4) ==== 96726+0+0 (secure 0 0 0) 0x7f761801b460 con 0x7f761c06d5a0 2024-04-25T19:15:12.075 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.072+0000 7f76117fa640 1 --2- 172.21.15.76:0/80752080 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f7608074420 0x7f7608076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:12.075 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.072+0000 7f76117fa640 1 -- 172.21.15.76:0/80752080 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7f7618094600 con 0x7f761c06d5a0 2024-04-25T19:15:12.075 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.072+0000 7f7613fff640 1 --2- 172.21.15.76:0/80752080 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f7608074420 0x7f7608076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:12.075 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.073+0000 7f7613fff640 1 --2- 172.21.15.76:0/80752080 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f7608074420 0x7f7608076930 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f761c1f04a0 tx=0x7f760c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:12.078 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.077+0000 7f76117fa640 1 -- 172.21.15.76:0/80752080 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7618020020 con 0x7f761c06d5a0 2024-04-25T19:15:12.256 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.253+0000 7f7623182640 1 -- 172.21.15.76:0/80752080 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f761c15cbd0 con 0x7f761c06d5a0 2024-04-25T19:15:12.257 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.254+0000 7f76117fa640 1 -- 172.21.15.76:0/80752080 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10909 (secure 0 0 0) 0x7f761805f3d0 con 0x7f761c06d5a0 2024-04-25T19:15:12.257 INFO:teuthology.misc.health.smithi076.stdout: 2024-04-25T19:15:12.257 INFO:teuthology.misc.health.smithi076.stdout:{"epoch":13,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","created":"2024-04-25T19:15:01.693137+0000","modified":"2024-04-25T19:15:11.891681+0000","last_up_change":"2024-04-25T19:15:11.891681+0000","last_in_change":"2024-04-25T19:15:03.898065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"72a16c02-48f9-45aa-9447-fcf755ebf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409574998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409574998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409574998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409574998}]},"public_addr":"172.21.15.76:6817/1409574998","cluster_addr":"172.21.15.76:6819/1409574998","heartbeat_back_addr":"172.21.15.76:6823/1409574998","heartbeat_front_addr":"172.21.15.76:6821/1409574998","state":["exists","up"]},{"osd":1,"uuid":"818d50f0-76c6-45f8-bf79-5e304f14c5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6801","nonce":1302663821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6803","nonce":1302663821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6807","nonce":1302663821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6805","nonce":1302663821}]},"public_addr":"172.21.15.76:6801/1302663821","cluster_addr":"172.21.15.76:6803/1302663821","heartbeat_back_addr":"172.21.15.76:6807/1302663821","heartbeat_front_addr":"172.21.15.76:6805/1302663821","state":["exists","up"]},{"osd":2,"uuid":"5714c8f5-ae23-473c-8cf1-db73811acded","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6809","nonce":1992938181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6811","nonce":1992938181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6815","nonce":1992938181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6813","nonce":1992938181}]},"public_addr":"172.21.15.76:6809/1992938181","cluster_addr":"172.21.15.76:6811/1992938181","heartbeat_back_addr":"172.21.15.76:6815/1992938181","heartbeat_front_addr":"172.21.15.76:6813/1992938181","state":["exists","up"]},{"osd":3,"uuid":"13204cdb-d8e6-4f97-bc6e-9561f442decf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6825","nonce":3418257163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6827","nonce":3418257163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6831","nonce":3418257163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6829","nonce":3418257163}]},"public_addr":"172.21.15.76:6825/3418257163","cluster_addr":"172.21.15.76:6827/3418257163","heartbeat_back_addr":"172.21.15.76:6831/3418257163","heartbeat_front_addr":"172.21.15.76:6829/3418257163","state":["exists","up"]},{"osd":4,"uuid":"5bea5166-1986-4d56-b5f5-ecc8cbe7f3ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6801","nonce":2125002468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6803","nonce":2125002468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6807","nonce":2125002468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6805","nonce":2125002468}]},"public_addr":"172.21.15.105:6801/2125002468","cluster_addr":"172.21.15.105:6803/2125002468","heartbeat_back_addr":"172.21.15.105:6807/2125002468","heartbeat_front_addr":"172.21.15.105:6805/2125002468","state":["exists","up"]},{"osd":5,"uuid":"96f89dcc-270d-4421-b8e8-3b92ba5f9844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6819","nonce":3413103121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6821","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6823","nonce":3413103121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6831","nonce":3413103121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6828","nonce":3413103121}]},"public_addr":"172.21.15.105:6819/3413103121","cluster_addr":"172.21.15.105:6823/3413103121","heartbeat_back_addr":"172.21.15.105:6831/3413103121","heartbeat_front_addr":"172.21.15.105:6828/3413103121","state":["exists","up"]},{"osd":6,"uuid":"93eb087a-d9ba-4096-b85d-b74d96e819a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6809","nonce":2721832954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6811","nonce":2721832954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6815","nonce":2721832954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6813","nonce":2721832954}]},"public_addr":"172.21.15.105:6809/2721832954","cluster_addr":"172.21.15.105:6811/2721832954","heartbeat_back_addr":"172.21.15.105:6815/2721832954","heartbeat_front_addr":"172.21.15.105:6813/2721832954","state":["exists","up"]},{"osd":7,"uuid":"2f80f790-ad2d-460e-a37d-47bf1771ef43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6818","nonce":548350430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6822","nonce":548350430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6829","nonce":548350430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6825","nonce":548350430}]},"public_addr":"172.21.15.105:6818/548350430","cluster_addr":"172.21.15.105:6822/548350430","heartbeat_back_addr":"172.21.15.105:6829/548350430","heartbeat_front_addr":"172.21.15.105:6825/548350430","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:15:12.259 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.256+0000 7f7623182640 1 -- 172.21.15.76:0/80752080 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f7608074420 msgr2=0x7f7608076930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:12.259 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.256+0000 7f7623182640 1 --2- 172.21.15.76:0/80752080 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f7608074420 0x7f7608076930 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f761c1f04a0 tx=0x7f760c008040 comp rx=0 tx=0).stop 2024-04-25T19:15:12.259 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.256+0000 7f7623182640 1 -- 172.21.15.76:0/80752080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f761c06d5a0 msgr2=0x7f761c153230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:12.259 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.256+0000 7f7623182640 1 --2- 172.21.15.76:0/80752080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f761c06d5a0 0x7f761c153230 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f761800b420 tx=0x7f761800b8e0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.259 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.256+0000 7f7623182640 1 -- 172.21.15.76:0/80752080 shutdown_connections 2024-04-25T19:15:12.259 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.256+0000 7f7623182640 1 --2- 172.21.15.76:0/80752080 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f7608074420 0x7f7608076930 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.259 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.256+0000 7f7623182640 1 --2- 172.21.15.76:0/80752080 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f761c0c6ec0 0x7f761c15bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.259 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.256+0000 7f7623182640 1 --2- 172.21.15.76:0/80752080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f761c06eaa0 0x7f761c15b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.259 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.256+0000 7f7623182640 1 --2- 172.21.15.76:0/80752080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f761c06d5a0 0x7f761c153230 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.260 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.256+0000 7f7623182640 1 -- 172.21.15.76:0/80752080 >> 172.21.15.76:0/80752080 conn(0x7f761c068ee0 msgr2=0x7f761c0cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:12.260 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.256+0000 7f7623182640 1 -- 172.21.15.76:0/80752080 shutdown_connections 2024-04-25T19:15:12.260 INFO:teuthology.misc.health.smithi076.stderr:2024-04-25T19:15:12.256+0000 7f7623182640 1 -- 172.21.15.76:0/80752080 wait complete. 2024-04-25T19:15:12.269 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T19:15:12.269 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T19:15:12.269 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T19:15:12.269 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T19:15:12.271 INFO:tasks.ceph.mds.a:Started 2024-04-25T19:15:12.271 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T19:15:12.272 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T19:15:12.273 INFO:tasks.ceph.mds.c:Started 2024-04-25T19:15:12.273 INFO:tasks.ceph.mds.e:Restarting daemon 2024-04-25T19:15:12.274 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-04-25T19:15:12.275 INFO:tasks.ceph.mds.e:Started 2024-04-25T19:15:12.275 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T19:15:12.276 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T19:15:12.278 INFO:tasks.ceph.mds.b:Started 2024-04-25T19:15:12.278 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T19:15:12.279 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T19:15:12.281 INFO:tasks.ceph.mds.d:Started 2024-04-25T19:15:12.281 INFO:tasks.ceph.mds.f:Restarting daemon 2024-04-25T19:15:12.281 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-04-25T19:15:12.283 INFO:tasks.ceph.mds.f:Started 2024-04-25T19:15:12.284 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T19:15:12.284 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:15:12.359 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2024-04-25T19:15:12.364 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2024-04-25T19:15:12.365 INFO:tasks.ceph.mds.e.smithi076.stdout:starting mds.e at 2024-04-25T19:15:12.377 INFO:tasks.ceph.mds.b.smithi105.stdout:starting mds.b at 2024-04-25T19:15:12.378 INFO:tasks.ceph.mds.f.smithi105.stdout:starting mds.f at 2024-04-25T19:15:12.378 INFO:tasks.ceph.mds.d.smithi105.stdout:starting mds.d at 2024-04-25T19:15:12.379 INFO:tasks.ceph.mds.f.smithi105.stderr:2024-04-25T19:15:12.377+0000 7f9f417bb640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6837/0: (98) Address already in use 2024-04-25T19:15:12.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.402+0000 7fb9b4e61640 1 Processor -- start 2024-04-25T19:15:12.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.402+0000 7fb9b4e61640 1 -- start start 2024-04-25T19:15:12.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.403+0000 7fb9b4e61640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb9b0072b60 0x7fb9b0072f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:12.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.403+0000 7fb9b4e61640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9b015e740 0x7fb9b015eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:12.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.403+0000 7fb9b4e61640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9b015af40 con 0x7fb9b0154d60 2024-04-25T19:15:12.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.403+0000 7fb9b4e61640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb9b015bac0 con 0x7fb9b0072b60 2024-04-25T19:15:12.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.403+0000 7fb9b4e61640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb9b015e2b0 con 0x7fb9b015e740 2024-04-25T19:15:12.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.403+0000 7fb9a7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb9b0072b60 0x7fb9b0072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:12.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.403+0000 7fb9a7fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb9b0072b60 0x7fb9b0072f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60358/0 (socket says 172.21.15.76:60358) 2024-04-25T19:15:12.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.403+0000 7fb9a7fff640 1 -- 172.21.15.76:0/2711036684 learned_addr learned my addr 172.21.15.76:0/2711036684 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:12.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.404+0000 7fb9aed9d640 1 --2- 172.21.15.76:0/2711036684 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9b015e740 0x7fb9b015eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:12.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.404+0000 7fb9a7fff640 1 -- 172.21.15.76:0/2711036684 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9b015e740 msgr2=0x7fb9b015eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:12.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.404+0000 7fb9a7fff640 1 --2- 172.21.15.76:0/2711036684 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9b015e740 0x7fb9b015eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.404+0000 7fb9a7fff640 1 -- 172.21.15.76:0/2711036684 >> v1:172.21.15.105:6789/0 conn(0x7fb9b0154d60 legacy=0x7fb9b00ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:12.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.404+0000 7fb9add9b640 1 -- 172.21.15.76:0/2711036684 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1849115582 0 0) 0x7fb9b015af40 con 0x7fb9b0154d60 2024-04-25T19:15:12.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.404+0000 7fb9a7fff640 1 -- 172.21.15.76:0/2711036684 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b015b220 con 0x7fb9b0072b60 2024-04-25T19:15:12.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.404+0000 7fb9a7fff640 1 --2- 172.21.15.76:0/2711036684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9b0072b60 0x7fb9b0072f70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb998009870 tx=0x7fb99802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00e704b87a67e3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:12.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.405+0000 7fb9add9b640 1 -- 172.21.15.76:0/2711036684 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998037070 con 0x7fb9b0072b60 2024-04-25T19:15:12.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.405+0000 7fb9add9b640 1 -- 172.21.15.76:0/2711036684 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb998032440 con 0x7fb9b0072b60 2024-04-25T19:15:12.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.405+0000 7fb9add9b640 1 -- 172.21.15.76:0/2711036684 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998032810 con 0x7fb9b0072b60 2024-04-25T19:15:12.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.405+0000 7fb9b4e61640 1 -- 172.21.15.76:0/2711036684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9b0072b60 msgr2=0x7fb9b0072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:12.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.405+0000 7fb9b4e61640 1 --2- 172.21.15.76:0/2711036684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9b0072b60 0x7fb9b0072f70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb998009870 tx=0x7fb99802a1c0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.405+0000 7fb9b4e61640 1 -- 172.21.15.76:0/2711036684 shutdown_connections 2024-04-25T19:15:12.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.405+0000 7fb9b4e61640 1 --2- 172.21.15.76:0/2711036684 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9b015e740 0x7fb9b015eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.405+0000 7fb9b4e61640 1 --2- 172.21.15.76:0/2711036684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9b0072b60 0x7fb9b0072f70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.405+0000 7fb9b4e61640 1 -- 172.21.15.76:0/2711036684 >> 172.21.15.76:0/2711036684 conn(0x7fb9b0070b00 msgr2=0x7fb9b0070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:12.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.405+0000 7fb9b4e61640 1 -- 172.21.15.76:0/2711036684 shutdown_connections 2024-04-25T19:15:12.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.405+0000 7fb9b4e61640 1 -- 172.21.15.76:0/2711036684 wait complete. 2024-04-25T19:15:12.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.407+0000 7fb9b4e61640 1 Processor -- start 2024-04-25T19:15:12.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.408+0000 7fb9b4e61640 1 -- start start 2024-04-25T19:15:12.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.409+0000 7fb9b4e61640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9b0072b60 0x7fb9b015c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:12.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.409+0000 7fb9b4e61640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9b0154d60 0x7fb9b01f1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:12.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.409+0000 7fb9b4e61640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9b015e740 0x7fb9b0149da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:12.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.409+0000 7fb9b4e61640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b0155170 con 0x7fb9b0154d60 2024-04-25T19:15:12.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.409+0000 7fb9b4e61640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b015b7e0 con 0x7fb9b0072b60 2024-04-25T19:15:12.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.409+0000 7fb9a7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9b0154d60 0x7fb9b01f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:12.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.409+0000 7fb9a7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9b0154d60 0x7fb9b01f1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60364/0 (socket says 172.21.15.76:60364) 2024-04-25T19:15:12.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.409+0000 7fb9a7fff640 1 -- 172.21.15.76:0/1781463348 learned_addr learned my addr 172.21.15.76:0/1781463348 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:12.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.409+0000 7fb9b4e61640 1 -- 172.21.15.76:0/1781463348 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb9b00cb0e0 con 0x7fb9b015e740 2024-04-25T19:15:12.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.409+0000 7fb9ae59c640 1 --2- 172.21.15.76:0/1781463348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9b0072b60 0x7fb9b015c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:12.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.409+0000 7fb9aed9d640 1 --2- 172.21.15.76:0/1781463348 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9b015e740 0x7fb9b0149da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:12.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.409+0000 7fb9a7fff640 1 -- 172.21.15.76:0/1781463348 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9b015e740 msgr2=0x7fb9b0149da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:12.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.409+0000 7fb9a7fff640 1 --2- 172.21.15.76:0/1781463348 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9b015e740 0x7fb9b0149da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.409+0000 7fb9a7fff640 1 -- 172.21.15.76:0/1781463348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9b0072b60 msgr2=0x7fb9b015c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:12.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.409+0000 7fb9a7fff640 1 --2- 172.21.15.76:0/1781463348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9b0072b60 0x7fb9b015c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.409+0000 7fb9a7fff640 1 -- 172.21.15.76:0/1781463348 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b01f1c70 con 0x7fb9b0154d60 2024-04-25T19:15:12.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.410+0000 7fb9a7fff640 1 --2- 172.21.15.76:0/1781463348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9b0154d60 0x7fb9b01f1730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb998009870 tx=0x7fb9980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:12.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.410+0000 7fb9a6ffd640 1 -- 172.21.15.76:0/1781463348 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998041020 con 0x7fb9b0154d60 2024-04-25T19:15:12.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.410+0000 7fb9a6ffd640 1 -- 172.21.15.76:0/1781463348 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb998030640 con 0x7fb9b0154d60 2024-04-25T19:15:12.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.410+0000 7fb9b4e61640 1 -- 172.21.15.76:0/1781463348 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b014adf0 con 0x7fb9b0154d60 2024-04-25T19:15:12.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.410+0000 7fb9a6ffd640 1 -- 172.21.15.76:0/1781463348 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998037040 con 0x7fb9b0154d60 2024-04-25T19:15:12.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.410+0000 7fb9b4e61640 1 -- 172.21.15.76:0/1781463348 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b01f0f30 con 0x7fb9b0154d60 2024-04-25T19:15:12.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.411+0000 7fb9b4e61640 1 -- 172.21.15.76:0/1781463348 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9b006a390 con 0x7fb9b0154d60 2024-04-25T19:15:12.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.411+0000 7fb9a6ffd640 1 -- 172.21.15.76:0/1781463348 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 4) ==== 96726+0+0 (secure 0 0 0) 0x7fb998044020 con 0x7fb9b0154d60 2024-04-25T19:15:12.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.412+0000 7fb9a6ffd640 1 --2- 172.21.15.76:0/1781463348 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb990074420 0x7fb990076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:12.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.412+0000 7fb9a6ffd640 1 -- 172.21.15.76:0/1781463348 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7fb9980b5340 con 0x7fb9b0154d60 2024-04-25T19:15:12.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.412+0000 7fb9ae59c640 1 --2- 172.21.15.76:0/1781463348 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb990074420 0x7fb990076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:12.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.413+0000 7fb9ae59c640 1 --2- 172.21.15.76:0/1781463348 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb990074420 0x7fb990076930 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0000f60 tx=0x7fb9a0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:12.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.417+0000 7fb9a6ffd640 1 -- 172.21.15.76:0/1781463348 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb99804b030 con 0x7fb9b0154d60 2024-04-25T19:15:12.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.594+0000 7fb9b4e61640 1 -- 172.21.15.76:0/1781463348 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9b00ca470 con 0x7fb9b0154d60 2024-04-25T19:15:12.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.594+0000 7fb9a6ffd640 1 -- 172.21.15.76:0/1781463348 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10909 (secure 0 0 0) 0x7fb99807fa30 con 0x7fb9b0154d60 2024-04-25T19:15:12.597 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:12.597 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":13,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","created":"2024-04-25T19:15:01.693137+0000","modified":"2024-04-25T19:15:11.891681+0000","last_up_change":"2024-04-25T19:15:11.891681+0000","last_in_change":"2024-04-25T19:15:03.898065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"72a16c02-48f9-45aa-9447-fcf755ebf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409574998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409574998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409574998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409574998}]},"public_addr":"172.21.15.76:6817/1409574998","cluster_addr":"172.21.15.76:6819/1409574998","heartbeat_back_addr":"172.21.15.76:6823/1409574998","heartbeat_front_addr":"172.21.15.76:6821/1409574998","state":["exists","up"]},{"osd":1,"uuid":"818d50f0-76c6-45f8-bf79-5e304f14c5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6801","nonce":1302663821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6803","nonce":1302663821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6807","nonce":1302663821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6805","nonce":1302663821}]},"public_addr":"172.21.15.76:6801/1302663821","cluster_addr":"172.21.15.76:6803/1302663821","heartbeat_back_addr":"172.21.15.76:6807/1302663821","heartbeat_front_addr":"172.21.15.76:6805/1302663821","state":["exists","up"]},{"osd":2,"uuid":"5714c8f5-ae23-473c-8cf1-db73811acded","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6809","nonce":1992938181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6811","nonce":1992938181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6815","nonce":1992938181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6813","nonce":1992938181}]},"public_addr":"172.21.15.76:6809/1992938181","cluster_addr":"172.21.15.76:6811/1992938181","heartbeat_back_addr":"172.21.15.76:6815/1992938181","heartbeat_front_addr":"172.21.15.76:6813/1992938181","state":["exists","up"]},{"osd":3,"uuid":"13204cdb-d8e6-4f97-bc6e-9561f442decf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6825","nonce":3418257163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6827","nonce":3418257163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6831","nonce":3418257163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6829","nonce":3418257163}]},"public_addr":"172.21.15.76:6825/3418257163","cluster_addr":"172.21.15.76:6827/3418257163","heartbeat_back_addr":"172.21.15.76:6831/3418257163","heartbeat_front_addr":"172.21.15.76:6829/3418257163","state":["exists","up"]},{"osd":4,"uuid":"5bea5166-1986-4d56-b5f5-ecc8cbe7f3ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6801","nonce":2125002468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6803","nonce":2125002468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6807","nonce":2125002468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6805","nonce":2125002468}]},"public_addr":"172.21.15.105:6801/2125002468","cluster_addr":"172.21.15.105:6803/2125002468","heartbeat_back_addr":"172.21.15.105:6807/2125002468","heartbeat_front_addr":"172.21.15.105:6805/2125002468","state":["exists","up"]},{"osd":5,"uuid":"96f89dcc-270d-4421-b8e8-3b92ba5f9844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6819","nonce":3413103121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6821","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6823","nonce":3413103121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6831","nonce":3413103121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6828","nonce":3413103121}]},"public_addr":"172.21.15.105:6819/3413103121","cluster_addr":"172.21.15.105:6823/3413103121","heartbeat_back_addr":"172.21.15.105:6831/3413103121","heartbeat_front_addr":"172.21.15.105:6828/3413103121","state":["exists","up"]},{"osd":6,"uuid":"93eb087a-d9ba-4096-b85d-b74d96e819a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6809","nonce":2721832954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6811","nonce":2721832954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6815","nonce":2721832954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6813","nonce":2721832954}]},"public_addr":"172.21.15.105:6809/2721832954","cluster_addr":"172.21.15.105:6811/2721832954","heartbeat_back_addr":"172.21.15.105:6815/2721832954","heartbeat_front_addr":"172.21.15.105:6813/2721832954","state":["exists","up"]},{"osd":7,"uuid":"2f80f790-ad2d-460e-a37d-47bf1771ef43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6818","nonce":548350430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6822","nonce":548350430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6829","nonce":548350430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6825","nonce":548350430}]},"public_addr":"172.21.15.105:6818/548350430","cluster_addr":"172.21.15.105:6822/548350430","heartbeat_back_addr":"172.21.15.105:6829/548350430","heartbeat_front_addr":"172.21.15.105:6825/548350430","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:15:12.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.596+0000 7fb9b4e61640 1 -- 172.21.15.76:0/1781463348 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb990074420 msgr2=0x7fb990076930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:12.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.596+0000 7fb9b4e61640 1 --2- 172.21.15.76:0/1781463348 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb990074420 0x7fb990076930 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0000f60 tx=0x7fb9a0009040 comp rx=0 tx=0).stop 2024-04-25T19:15:12.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.596+0000 7fb9b4e61640 1 -- 172.21.15.76:0/1781463348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9b0154d60 msgr2=0x7fb9b01f1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:12.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.596+0000 7fb9b4e61640 1 --2- 172.21.15.76:0/1781463348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9b0154d60 0x7fb9b01f1730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb998009870 tx=0x7fb9980046c0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.596+0000 7fb9b4e61640 1 -- 172.21.15.76:0/1781463348 shutdown_connections 2024-04-25T19:15:12.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.596+0000 7fb9b4e61640 1 --2- 172.21.15.76:0/1781463348 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb990074420 0x7fb990076930 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.596+0000 7fb9b4e61640 1 --2- 172.21.15.76:0/1781463348 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9b015e740 0x7fb9b0149da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.596+0000 7fb9b4e61640 1 --2- 172.21.15.76:0/1781463348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9b0154d60 0x7fb9b01f1730 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.596+0000 7fb9b4e61640 1 --2- 172.21.15.76:0/1781463348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb9b0072b60 0x7fb9b015c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.596+0000 7fb9b4e61640 1 -- 172.21.15.76:0/1781463348 >> 172.21.15.76:0/1781463348 conn(0x7fb9b0070b00 msgr2=0x7fb9b0152560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:12.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.596+0000 7fb9b4e61640 1 -- 172.21.15.76:0/1781463348 shutdown_connections 2024-04-25T19:15:12.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.596+0000 7fb9b4e61640 1 -- 172.21.15.76:0/1781463348 wait complete. 2024-04-25T19:15:12.610 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-04-25T19:15:12.611 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:15:12.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.717+0000 7f5e45f46640 1 Processor -- start 2024-04-25T19:15:12.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.717+0000 7f5e45f46640 1 -- start start 2024-04-25T19:15:12.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.718+0000 7f5e45f46640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e4006fd50 0x7f5e40153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:12.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.718+0000 7f5e45f46640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5e40074130 0x7f5e40074540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:12.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.718+0000 7f5e45f46640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e400b4630 con 0x7f5e4015d370 2024-04-25T19:15:12.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.718+0000 7f5e45f46640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f5e40068520 con 0x7f5e40074130 2024-04-25T19:15:12.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.718+0000 7f5e45f46640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f5e400c3fa0 con 0x7f5e4006fd50 2024-04-25T19:15:12.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.718+0000 7f5e3ffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f5e4015d370 0x7f5e4015d780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52788/0 (socket says 172.21.15.76:52788) 2024-04-25T19:15:12.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.718+0000 7f5e3ffff640 1 -- 172.21.15.76:0/4170575563 learned_addr learned my addr 172.21.15.76:0/4170575563 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:12.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.718+0000 7f5e3effd640 1 --2- 172.21.15.76:0/4170575563 >> v2:172.21.15.105:3300/0 conn(0x7f5e40074130 0x7f5e40074540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:12.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.718+0000 7f5e3f7fe640 1 --2- 172.21.15.76:0/4170575563 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e4006fd50 0x7f5e40153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:12.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.718+0000 7f5e3e7fc640 1 -- 172.21.15.76:0/4170575563 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1848599183 0 0) 0x7f5e400b4630 con 0x7f5e4015d370 2024-04-25T19:15:12.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.719+0000 7f5e3e7fc640 1 -- 172.21.15.76:0/4170575563 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e28003520 con 0x7f5e4015d370 2024-04-25T19:15:12.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.719+0000 7f5e3effd640 1 -- 172.21.15.76:0/4170575563 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e4006fd50 msgr2=0x7f5e40153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:12.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.719+0000 7f5e3effd640 1 --2- 172.21.15.76:0/4170575563 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e4006fd50 0x7f5e40153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.719+0000 7f5e3effd640 1 -- 172.21.15.76:0/4170575563 >> v1:172.21.15.76:6789/0 conn(0x7f5e4015d370 legacy=0x7f5e4015d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:12.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.719+0000 7f5e3effd640 1 -- 172.21.15.76:0/4170575563 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e400c4280 con 0x7f5e40074130 2024-04-25T19:15:12.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.719+0000 7f5e3effd640 1 --2- 172.21.15.76:0/4170575563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e40074130 0x7f5e40074540 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5e340097a0 tx=0x7f5e3402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7741dbf72cd67d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:12.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.720+0000 7f5e3e7fc640 1 -- 172.21.15.76:0/4170575563 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e34037070 con 0x7f5e40074130 2024-04-25T19:15:12.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.720+0000 7f5e3e7fc640 1 -- 172.21.15.76:0/4170575563 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e34032440 con 0x7f5e40074130 2024-04-25T19:15:12.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.720+0000 7f5e3e7fc640 1 -- 172.21.15.76:0/4170575563 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e340327f0 con 0x7f5e40074130 2024-04-25T19:15:12.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.720+0000 7f5e45f46640 1 -- 172.21.15.76:0/4170575563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e40074130 msgr2=0x7f5e40074540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:12.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.720+0000 7f5e45f46640 1 --2- 172.21.15.76:0/4170575563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e40074130 0x7f5e40074540 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5e340097a0 tx=0x7f5e3402a180 comp rx=0 tx=0).stop 2024-04-25T19:15:12.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.720+0000 7f5e45f46640 1 -- 172.21.15.76:0/4170575563 shutdown_connections 2024-04-25T19:15:12.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.720+0000 7f5e45f46640 1 --2- 172.21.15.76:0/4170575563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e40074130 0x7f5e40074540 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.720+0000 7f5e45f46640 1 --2- 172.21.15.76:0/4170575563 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e4006fd50 0x7f5e40153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.720+0000 7f5e45f46640 1 -- 172.21.15.76:0/4170575563 >> 172.21.15.76:0/4170575563 conn(0x7f5e4006c5c0 msgr2=0x7f5e4006c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:12.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.720+0000 7f5e45f46640 1 -- 172.21.15.76:0/4170575563 shutdown_connections 2024-04-25T19:15:12.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.720+0000 7f5e45f46640 1 -- 172.21.15.76:0/4170575563 wait complete. 2024-04-25T19:15:12.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.722+0000 7f5e45f46640 1 Processor -- start 2024-04-25T19:15:12.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.722+0000 7f5e45f46640 1 -- start start 2024-04-25T19:15:12.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.723+0000 7f5e45f46640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e4006fd50 0x7f5e4014db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:12.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.723+0000 7f5e45f46640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e40074130 0x7f5e400cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:12.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.723+0000 7f5e45f46640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e4015d370 0x7f5e4014acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:12.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.723+0000 7f5e45f46640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5e400cacc0 con 0x7f5e40074130 2024-04-25T19:15:12.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.723+0000 7f5e45f46640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5e401541f0 con 0x7f5e4006fd50 2024-04-25T19:15:12.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.723+0000 7f5e45f46640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f5e40068a50 con 0x7f5e4015d370 2024-04-25T19:15:12.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.724+0000 7f5e3effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e40074130 0x7f5e400cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:12.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.724+0000 7f5e3effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e40074130 0x7f5e400cd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60380/0 (socket says 172.21.15.76:60380) 2024-04-25T19:15:12.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.724+0000 7f5e3effd640 1 -- 172.21.15.76:0/4240216171 learned_addr learned my addr 172.21.15.76:0/4240216171 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:12.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.724+0000 7f5e3ffff640 1 --2- 172.21.15.76:0/4240216171 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e4015d370 0x7f5e4014acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:12.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.724+0000 7f5e3f7fe640 1 --2- 172.21.15.76:0/4240216171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e4006fd50 0x7f5e4014db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:12.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.724+0000 7f5e3effd640 1 -- 172.21.15.76:0/4240216171 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e4015d370 msgr2=0x7f5e4014acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:12.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.724+0000 7f5e3effd640 1 --2- 172.21.15.76:0/4240216171 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e4015d370 0x7f5e4014acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.724+0000 7f5e3effd640 1 -- 172.21.15.76:0/4240216171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e4006fd50 msgr2=0x7f5e4014db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:12.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.724+0000 7f5e3effd640 1 --2- 172.21.15.76:0/4240216171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e4006fd50 0x7f5e4014db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.724+0000 7f5e3effd640 1 -- 172.21.15.76:0/4240216171 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4014e780 con 0x7f5e40074130 2024-04-25T19:15:12.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.724+0000 7f5e3effd640 1 --2- 172.21.15.76:0/4240216171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e40074130 0x7f5e400cd1a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5e34035040 tx=0x7f5e340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:12.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.724+0000 7f5e27fff640 1 -- 172.21.15.76:0/4240216171 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e34037040 con 0x7f5e40074130 2024-04-25T19:15:12.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.724+0000 7f5e27fff640 1 -- 172.21.15.76:0/4240216171 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e34042070 con 0x7f5e40074130 2024-04-25T19:15:12.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.724+0000 7f5e45f46640 1 -- 172.21.15.76:0/4240216171 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e401f4040 con 0x7f5e40074130 2024-04-25T19:15:12.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.724+0000 7f5e27fff640 1 -- 172.21.15.76:0/4240216171 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3403b930 con 0x7f5e40074130 2024-04-25T19:15:12.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.724+0000 7f5e45f46640 1 -- 172.21.15.76:0/4240216171 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e40154720 con 0x7f5e40074130 2024-04-25T19:15:12.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.725+0000 7f5e45f46640 1 -- 172.21.15.76:0/4240216171 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e4006d6a0 con 0x7f5e40074130 2024-04-25T19:15:12.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.726+0000 7f5e27fff640 1 -- 172.21.15.76:0/4240216171 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 4) ==== 96726+0+0 (secure 0 0 0) 0x7f5e3403bad0 con 0x7f5e40074130 2024-04-25T19:15:12.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.726+0000 7f5e27fff640 1 --2- 172.21.15.76:0/4240216171 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5e2c074420 0x7f5e2c076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:12.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.726+0000 7f5e27fff640 1 -- 172.21.15.76:0/4240216171 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7f5e340b5300 con 0x7f5e40074130 2024-04-25T19:15:12.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.726+0000 7f5e3f7fe640 1 --2- 172.21.15.76:0/4240216171 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5e2c074420 0x7f5e2c076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:12.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.727+0000 7f5e3f7fe640 1 --2- 172.21.15.76:0/4240216171 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5e2c074420 0x7f5e2c076930 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009a30 tx=0x7f5e30008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:12.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.731+0000 7f5e27fff640 1 -- 172.21.15.76:0/4240216171 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5e340800d0 con 0x7f5e40074130 2024-04-25T19:15:12.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.948+0000 7f5e45f46640 1 -- 172.21.15.76:0/4240216171 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e4006a390 con 0x7f5e40074130 2024-04-25T19:15:12.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.948+0000 7f5e27fff640 1 -- 172.21.15.76:0/4240216171 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7f5e3407f9d0 con 0x7f5e40074130 2024-04-25T19:15:12.951 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:12.951 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2101494685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2101494685},{"type":"v1","addr":"172.21.15.76:6833","nonce":2101494685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T19:15:12.952 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 2 2024-04-25T19:15:12.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.950+0000 7f5e45f46640 1 -- 172.21.15.76:0/4240216171 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5e2c074420 msgr2=0x7f5e2c076930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:12.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.950+0000 7f5e45f46640 1 --2- 172.21.15.76:0/4240216171 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5e2c074420 0x7f5e2c076930 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009a30 tx=0x7f5e30008040 comp rx=0 tx=0).stop 2024-04-25T19:15:12.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.950+0000 7f5e45f46640 1 -- 172.21.15.76:0/4240216171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e40074130 msgr2=0x7f5e400cd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:12.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.950+0000 7f5e45f46640 1 --2- 172.21.15.76:0/4240216171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e40074130 0x7f5e400cd1a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5e34035040 tx=0x7f5e340046c0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.950+0000 7f5e45f46640 1 -- 172.21.15.76:0/4240216171 shutdown_connections 2024-04-25T19:15:12.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.950+0000 7f5e45f46640 1 --2- 172.21.15.76:0/4240216171 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5e2c074420 0x7f5e2c076930 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.950+0000 7f5e45f46640 1 --2- 172.21.15.76:0/4240216171 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5e4015d370 0x7f5e4014acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.950+0000 7f5e45f46640 1 --2- 172.21.15.76:0/4240216171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e40074130 0x7f5e400cd1a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.950+0000 7f5e45f46640 1 --2- 172.21.15.76:0/4240216171 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e4006fd50 0x7f5e4014db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:12.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.950+0000 7f5e45f46640 1 -- 172.21.15.76:0/4240216171 >> 172.21.15.76:0/4240216171 conn(0x7f5e4006c5c0 msgr2=0x7f5e40153310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:12.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.950+0000 7f5e45f46640 1 -- 172.21.15.76:0/4240216171 shutdown_connections 2024-04-25T19:15:12.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:12.950+0000 7f5e45f46640 1 -- 172.21.15.76:0/4240216171 wait complete. 2024-04-25T19:15:14.964 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:15:15.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.083+0000 7fa08fd4e640 1 Processor -- start 2024-04-25T19:15:15.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.084+0000 7fa08fd4e640 1 -- start start 2024-04-25T19:15:15.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.085+0000 7fa08fd4e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa088157200 0x7fa0880d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:15.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.085+0000 7fa08fd4e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa0880cfd50 0x7fa0880cee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:15.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.085+0000 7fa08fd4e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa08815dd00 con 0x7fa0880d0160 2024-04-25T19:15:15.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.085+0000 7fa08fd4e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa08815cea0 con 0x7fa0880cfd50 2024-04-25T19:15:15.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.085+0000 7fa08fd4e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa08815d180 con 0x7fa088157200 2024-04-25T19:15:15.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.085+0000 7fa08d2e9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa0880cfd50 0x7fa0880cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:15.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.085+0000 7fa08d2e9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa0880cfd50 0x7fa0880cee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60392/0 (socket says 172.21.15.76:60392) 2024-04-25T19:15:15.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.085+0000 7fa08d2e9640 1 -- 172.21.15.76:0/1502074920 learned_addr learned my addr 172.21.15.76:0/1502074920 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:15.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.085+0000 7fa08daea640 1 --2- 172.21.15.76:0/1502074920 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa088157200 0x7fa0880d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:15.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.085+0000 7fa08d2e9640 1 -- 172.21.15.76:0/1502074920 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa088157200 msgr2=0x7fa0880d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:15.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.085+0000 7fa08d2e9640 1 --2- 172.21.15.76:0/1502074920 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa088157200 0x7fa0880d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.085+0000 7fa08d2e9640 1 -- 172.21.15.76:0/1502074920 >> v1:172.21.15.105:6789/0 conn(0x7fa0880d0160 legacy=0x7fa0880ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:15.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.085+0000 7fa08d2e9640 1 -- 172.21.15.76:0/1502074920 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08815d460 con 0x7fa0880cfd50 2024-04-25T19:15:15.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.085+0000 7fa08daea640 1 --2- 172.21.15.76:0/1502074920 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa088157200 0x7fa0880d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:15.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.086+0000 7fa08d2e9640 1 --2- 172.21.15.76:0/1502074920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0880cfd50 0x7fa0880cee90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa078009870 tx=0x7fa07802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636a965347bd9b36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:15.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.086+0000 7fa08cae8640 1 -- 172.21.15.76:0/1502074920 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa078037070 con 0x7fa0880cfd50 2024-04-25T19:15:15.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.086+0000 7fa08cae8640 1 -- 172.21.15.76:0/1502074920 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa07802ae50 con 0x7fa0880cfd50 2024-04-25T19:15:15.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.086+0000 7fa08cae8640 1 -- 172.21.15.76:0/1502074920 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa078032860 con 0x7fa0880cfd50 2024-04-25T19:15:15.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.087+0000 7fa08fd4e640 1 -- 172.21.15.76:0/1502074920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0880cfd50 msgr2=0x7fa0880cee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:15.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.087+0000 7fa08fd4e640 1 --2- 172.21.15.76:0/1502074920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0880cfd50 0x7fa0880cee90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa078009870 tx=0x7fa07802a300 comp rx=0 tx=0).stop 2024-04-25T19:15:15.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.087+0000 7fa08fd4e640 1 -- 172.21.15.76:0/1502074920 shutdown_connections 2024-04-25T19:15:15.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.087+0000 7fa08fd4e640 1 --2- 172.21.15.76:0/1502074920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0880cfd50 0x7fa0880cee90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.087+0000 7fa08fd4e640 1 --2- 172.21.15.76:0/1502074920 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa088157200 0x7fa0880d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.087+0000 7fa08fd4e640 1 -- 172.21.15.76:0/1502074920 >> 172.21.15.76:0/1502074920 conn(0x7fa0880d2370 msgr2=0x7fa0880d2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:15.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.087+0000 7fa08fd4e640 1 -- 172.21.15.76:0/1502074920 shutdown_connections 2024-04-25T19:15:15.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.087+0000 7fa08fd4e640 1 -- 172.21.15.76:0/1502074920 wait complete. 2024-04-25T19:15:15.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.089+0000 7fa08fd4e640 1 Processor -- start 2024-04-25T19:15:15.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.089+0000 7fa08fd4e640 1 -- start start 2024-04-25T19:15:15.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.090+0000 7fa08fd4e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0880cfd50 0x7fa0880b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:15.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.090+0000 7fa08fd4e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0880d0160 0x7fa0881538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:15.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.090+0000 7fa08fd4e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa088157200 0x7fa088153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:15.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.090+0000 7fa08fd4e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa0880cd280 con 0x7fa0880cfd50 2024-04-25T19:15:15.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.090+0000 7fa08fd4e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa08815da20 con 0x7fa088157200 2024-04-25T19:15:15.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.090+0000 7fa08fd4e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa0880cd400 con 0x7fa0880d0160 2024-04-25T19:15:15.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.090+0000 7fa08daea640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0880cfd50 0x7fa0880b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:15.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.090+0000 7fa08daea640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0880cfd50 0x7fa0880b6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60396/0 (socket says 172.21.15.76:60396) 2024-04-25T19:15:15.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.090+0000 7fa08daea640 1 -- 172.21.15.76:0/3432626994 learned_addr learned my addr 172.21.15.76:0/3432626994 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:15.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.090+0000 7fa08d2e9640 1 --2- 172.21.15.76:0/3432626994 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0880d0160 0x7fa0881538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:15.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.090+0000 7fa08e2eb640 1 --2- 172.21.15.76:0/3432626994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa088157200 0x7fa088153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:15.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.090+0000 7fa08daea640 1 -- 172.21.15.76:0/3432626994 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0880d0160 msgr2=0x7fa0881538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:15.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.090+0000 7fa08daea640 1 --2- 172.21.15.76:0/3432626994 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0880d0160 0x7fa0881538f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.090+0000 7fa08daea640 1 -- 172.21.15.76:0/3432626994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa088157200 msgr2=0x7fa088153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:15.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.090+0000 7fa08daea640 1 --2- 172.21.15.76:0/3432626994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa088157200 0x7fa088153e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.090+0000 7fa08daea640 1 -- 172.21.15.76:0/3432626994 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa088154370 con 0x7fa0880cfd50 2024-04-25T19:15:15.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.090+0000 7fa08e2eb640 1 --2- 172.21.15.76:0/3432626994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa088157200 0x7fa088153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:15:15.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.091+0000 7fa08daea640 1 --2- 172.21.15.76:0/3432626994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0880cfd50 0x7fa0880b6c20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa08400d800 tx=0x7fa08400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:15.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.091+0000 7fa07e7fc640 1 -- 172.21.15.76:0/3432626994 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084004030 con 0x7fa0880cfd50 2024-04-25T19:15:15.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.091+0000 7fa07e7fc640 1 -- 172.21.15.76:0/3432626994 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0840041d0 con 0x7fa0880cfd50 2024-04-25T19:15:15.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.091+0000 7fa08fd4e640 1 -- 172.21.15.76:0/3432626994 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0880b7160 con 0x7fa0880cfd50 2024-04-25T19:15:15.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.091+0000 7fa07e7fc640 1 -- 172.21.15.76:0/3432626994 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0840096a0 con 0x7fa0880cfd50 2024-04-25T19:15:15.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.091+0000 7fa08fd4e640 1 -- 172.21.15.76:0/3432626994 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0880b5e60 con 0x7fa0880cfd50 2024-04-25T19:15:15.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.092+0000 7fa08fd4e640 1 -- 172.21.15.76:0/3432626994 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa08806a390 con 0x7fa0880cfd50 2024-04-25T19:15:15.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.092+0000 7fa07e7fc640 1 -- 172.21.15.76:0/3432626994 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 4) ==== 96726+0+0 (secure 0 0 0) 0x7fa084009840 con 0x7fa0880cfd50 2024-04-25T19:15:15.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.093+0000 7fa07e7fc640 1 --2- 172.21.15.76:0/3432626994 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa06c074420 0x7fa06c076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:15.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.093+0000 7fa07e7fc640 1 -- 172.21.15.76:0/3432626994 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 4846+0+0 (secure 0 0 0) 0x7fa0840950c0 con 0x7fa0880cfd50 2024-04-25T19:15:15.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.093+0000 7fa08d2e9640 1 --2- 172.21.15.76:0/3432626994 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa06c074420 0x7fa06c076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:15.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.094+0000 7fa08d2e9640 1 --2- 172.21.15.76:0/3432626994 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa06c074420 0x7fa06c076930 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa078034000 tx=0x7fa07802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:15.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.098+0000 7fa07e7fc640 1 -- 172.21.15.76:0/3432626994 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa08405fc90 con 0x7fa0880cfd50 2024-04-25T19:15:15.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.312+0000 7fa08fd4e640 1 -- 172.21.15.76:0/3432626994 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0880b7ed0 con 0x7fa0880cfd50 2024-04-25T19:15:15.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.313+0000 7fa07e7fc640 1 -- 172.21.15.76:0/3432626994 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+5521 (secure 0 0 0) 0x7fa08405f590 con 0x7fa0880cfd50 2024-04-25T19:15:15.316 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:15.316 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/3003966182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3003966182},{"type":"v1","addr":"172.21.15.76:6836","nonce":3003966182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/1809877606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1809877606},{"type":"v1","addr":"172.21.15.105:6836","nonce":1809877606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4238,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6838/2724142282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2724142282},{"type":"v1","addr":"172.21.15.105:6838","nonce":2724142282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6839/1798232582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6837","nonce":1798232582},{"type":"v1","addr":"172.21.15.105:6839","nonce":1798232582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/2101494685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2101494685},{"type":"v1","addr":"172.21.15.76:6833","nonce":2101494685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/4226277412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":4226277412},{"type":"v1","addr":"172.21.15.76:6837","nonce":4226277412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T19:15:15.316 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 3 2024-04-25T19:15:15.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.315+0000 7fa08fd4e640 1 -- 172.21.15.76:0/3432626994 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa06c074420 msgr2=0x7fa06c076930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:15.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.315+0000 7fa08fd4e640 1 --2- 172.21.15.76:0/3432626994 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa06c074420 0x7fa06c076930 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa078034000 tx=0x7fa07802e000 comp rx=0 tx=0).stop 2024-04-25T19:15:15.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.315+0000 7fa08fd4e640 1 -- 172.21.15.76:0/3432626994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0880cfd50 msgr2=0x7fa0880b6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:15.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.315+0000 7fa08fd4e640 1 --2- 172.21.15.76:0/3432626994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0880cfd50 0x7fa0880b6c20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa08400d800 tx=0x7fa08400dcc0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.315+0000 7fa08fd4e640 1 -- 172.21.15.76:0/3432626994 shutdown_connections 2024-04-25T19:15:15.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.315+0000 7fa08fd4e640 1 --2- 172.21.15.76:0/3432626994 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa06c074420 0x7fa06c076930 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.315+0000 7fa08fd4e640 1 --2- 172.21.15.76:0/3432626994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa088157200 0x7fa088153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.315+0000 7fa08fd4e640 1 --2- 172.21.15.76:0/3432626994 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa0880d0160 0x7fa0881538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.315+0000 7fa08fd4e640 1 --2- 172.21.15.76:0/3432626994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0880cfd50 0x7fa0880b6c20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.315+0000 7fa08fd4e640 1 -- 172.21.15.76:0/3432626994 >> 172.21.15.76:0/3432626994 conn(0x7fa0880d2370 msgr2=0x7fa08815b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:15.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.315+0000 7fa08fd4e640 1 -- 172.21.15.76:0/3432626994 shutdown_connections 2024-04-25T19:15:15.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.315+0000 7fa08fd4e640 1 -- 172.21.15.76:0/3432626994 wait complete. 2024-04-25T19:15:15.328 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:15:15.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.435+0000 7fe70afb2640 1 Processor -- start 2024-04-25T19:15:15.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.435+0000 7fe70afb2640 1 -- start start 2024-04-25T19:15:15.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.436+0000 7fe70afb2640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe70406de20 0x7fe70406d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:15.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.436+0000 7fe70afb2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe704156f10 0x7fe70415d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.436+0000 7fe70afb2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe70406e310 con 0x7fe7040cc9e0 2024-04-25T19:15:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.436+0000 7fe70afb2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe70415d2f0 con 0x7fe704156f10 2024-04-25T19:15:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.436+0000 7fe70afb2640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fe70415dcc0 con 0x7fe70406de20 2024-04-25T19:15:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.437+0000 7fe70954f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe704156f10 0x7fe70415d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.437+0000 7fe70954f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe704156f10 0x7fe70415d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60400/0 (socket says 172.21.15.76:60400) 2024-04-25T19:15:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.437+0000 7fe70954f640 1 -- 172.21.15.76:0/2864302645 learned_addr learned my addr 172.21.15.76:0/2864302645 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.437+0000 7fe708d4e640 1 --2- 172.21.15.76:0/2864302645 >> v2:172.21.15.105:3300/0 conn(0x7fe70406de20 0x7fe70406d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.437+0000 7fe70954f640 1 -- 172.21.15.76:0/2864302645 >> v2:172.21.15.105:3300/0 conn(0x7fe70406de20 msgr2=0x7fe70406d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.437+0000 7fe70954f640 1 --2- 172.21.15.76:0/2864302645 >> v2:172.21.15.105:3300/0 conn(0x7fe70406de20 0x7fe70406d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.437+0000 7fe70954f640 1 -- 172.21.15.76:0/2864302645 >> v1:172.21.15.76:6789/0 conn(0x7fe7040cc9e0 legacy=0x7fe7040cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:15.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.437+0000 7fe70954f640 1 -- 172.21.15.76:0/2864302645 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70415ccc0 con 0x7fe704156f10 2024-04-25T19:15:15.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.438+0000 7fe70954f640 1 --2- 172.21.15.76:0/2864302645 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe704156f10 0x7fe70415d780 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe7040694c0 tx=0x7fe6fc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f396a86a8636c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:15.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.438+0000 7fe7037fe640 1 -- 172.21.15.76:0/2864302645 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc036070 con 0x7fe704156f10 2024-04-25T19:15:15.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.438+0000 7fe7037fe640 1 -- 172.21.15.76:0/2864302645 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6fc031600 con 0x7fe704156f10 2024-04-25T19:15:15.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.438+0000 7fe7037fe640 1 -- 172.21.15.76:0/2864302645 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc0319d0 con 0x7fe704156f10 2024-04-25T19:15:15.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.438+0000 7fe70afb2640 1 -- 172.21.15.76:0/2864302645 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe704156f10 msgr2=0x7fe70415d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:15.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.438+0000 7fe70afb2640 1 --2- 172.21.15.76:0/2864302645 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe704156f10 0x7fe70415d780 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe7040694c0 tx=0x7fe6fc02a6a0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.438+0000 7fe70afb2640 1 -- 172.21.15.76:0/2864302645 shutdown_connections 2024-04-25T19:15:15.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.438+0000 7fe70afb2640 1 --2- 172.21.15.76:0/2864302645 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe704156f10 0x7fe70415d780 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.438+0000 7fe70afb2640 1 --2- 172.21.15.76:0/2864302645 >> v2:172.21.15.105:3300/0 conn(0x7fe70406de20 0x7fe70406d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.438+0000 7fe70afb2640 1 -- 172.21.15.76:0/2864302645 >> 172.21.15.76:0/2864302645 conn(0x7fe7040684e0 msgr2=0x7fe7040688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:15.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.438+0000 7fe70afb2640 1 -- 172.21.15.76:0/2864302645 shutdown_connections 2024-04-25T19:15:15.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.438+0000 7fe70afb2640 1 -- 172.21.15.76:0/2864302645 wait complete. 2024-04-25T19:15:15.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.440+0000 7fe70afb2640 1 Processor -- start 2024-04-25T19:15:15.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.440+0000 7fe70afb2640 1 -- start start 2024-04-25T19:15:15.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.441+0000 7fe70afb2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe70406de20 0x7fe70414c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:15.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.441+0000 7fe70afb2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7040cc9e0 0x7fe7041520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:15.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.441+0000 7fe70afb2640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe704156f10 0x7fe70415b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:15.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.441+0000 7fe70afb2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe70415cf80 con 0x7fe7040cc9e0 2024-04-25T19:15:15.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.441+0000 7fe70afb2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe7040cb980 con 0x7fe70406de20 2024-04-25T19:15:15.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.441+0000 7fe70afb2640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fe704157320 con 0x7fe704156f10 2024-04-25T19:15:15.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.442+0000 7fe703fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7040cc9e0 0x7fe7041520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:15.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.442+0000 7fe703fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7040cc9e0 0x7fe7041520a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60406/0 (socket says 172.21.15.76:60406) 2024-04-25T19:15:15.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.442+0000 7fe703fff640 1 -- 172.21.15.76:0/3018672559 learned_addr learned my addr 172.21.15.76:0/3018672559 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:15.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.442+0000 7fe708d4e640 1 --2- 172.21.15.76:0/3018672559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe70406de20 0x7fe70414c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:15.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.442+0000 7fe70954f640 1 --2- 172.21.15.76:0/3018672559 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe704156f10 0x7fe70415b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:15.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.442+0000 7fe703fff640 1 -- 172.21.15.76:0/3018672559 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe704156f10 msgr2=0x7fe70415b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:15.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.442+0000 7fe703fff640 1 --2- 172.21.15.76:0/3018672559 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe704156f10 0x7fe70415b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.442+0000 7fe703fff640 1 -- 172.21.15.76:0/3018672559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe70406de20 msgr2=0x7fe70414c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:15.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.442+0000 7fe703fff640 1 --2- 172.21.15.76:0/3018672559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe70406de20 0x7fe70414c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.442+0000 7fe703fff640 1 -- 172.21.15.76:0/3018672559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70414cd30 con 0x7fe7040cc9e0 2024-04-25T19:15:15.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.442+0000 7fe703fff640 1 --2- 172.21.15.76:0/3018672559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7040cc9e0 0x7fe7041520a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800c430 tx=0x7fe6f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:15.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.442+0000 7fe7017fa640 1 -- 172.21.15.76:0/3018672559 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8017070 con 0x7fe7040cc9e0 2024-04-25T19:15:15.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.442+0000 7fe7017fa640 1 -- 172.21.15.76:0/3018672559 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6f80052d0 con 0x7fe7040cc9e0 2024-04-25T19:15:15.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.442+0000 7fe70afb2640 1 -- 172.21.15.76:0/3018672559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70414ba70 con 0x7fe7040cc9e0 2024-04-25T19:15:15.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.442+0000 7fe7017fa640 1 -- 172.21.15.76:0/3018672559 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8003830 con 0x7fe7040cc9e0 2024-04-25T19:15:15.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.442+0000 7fe70afb2640 1 -- 172.21.15.76:0/3018672559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe70414bff0 con 0x7fe7040cc9e0 2024-04-25T19:15:15.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.443+0000 7fe70afb2640 1 -- 172.21.15.76:0/3018672559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe70406a390 con 0x7fe7040cc9e0 2024-04-25T19:15:15.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.444+0000 7fe7017fa640 1 -- 172.21.15.76:0/3018672559 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 4) ==== 96726+0+0 (secure 0 0 0) 0x7fe6f801a070 con 0x7fe7040cc9e0 2024-04-25T19:15:15.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.444+0000 7fe7017fa640 1 --2- 172.21.15.76:0/3018672559 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe6f0074420 0x7fe6f0076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:15.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.445+0000 7fe7017fa640 1 -- 172.21.15.76:0/3018672559 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe6f8094020 con 0x7fe7040cc9e0 2024-04-25T19:15:15.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.445+0000 7fe708d4e640 1 --2- 172.21.15.76:0/3018672559 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe6f0074420 0x7fe6f0076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:15.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.445+0000 7fe708d4e640 1 --2- 172.21.15.76:0/3018672559 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe6f0074420 0x7fe6f0076930 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009aa0 tx=0x7fe6f4009190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:15.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.449+0000 7fe7017fa640 1 -- 172.21.15.76:0/3018672559 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe6f805e620 con 0x7fe7040cc9e0 2024-04-25T19:15:15.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.630+0000 7fe70afb2640 1 -- 172.21.15.76:0/3018672559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7040cc0b0 con 0x7fe7040cc9e0 2024-04-25T19:15:15.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.631+0000 7fe7017fa640 1 -- 172.21.15.76:0/3018672559 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12897 (secure 0 0 0) 0x7fe6f8062920 con 0x7fe7040cc9e0 2024-04-25T19:15:15.634 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:15.634 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":16,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","created":"2024-04-25T19:15:01.693137+0000","modified":"2024-04-25T19:15:15.372133+0000","last_up_change":"2024-04-25T19:15:11.891681+0000","last_in_change":"2024-04-25T19:15:03.898065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:15:13.846514+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72a16c02-48f9-45aa-9447-fcf755ebf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409574998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409574998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409574998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409574998}]},"public_addr":"172.21.15.76:6817/1409574998","cluster_addr":"172.21.15.76:6819/1409574998","heartbeat_back_addr":"172.21.15.76:6823/1409574998","heartbeat_front_addr":"172.21.15.76:6821/1409574998","state":["exists","up"]},{"osd":1,"uuid":"818d50f0-76c6-45f8-bf79-5e304f14c5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6801","nonce":1302663821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6803","nonce":1302663821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6807","nonce":1302663821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6805","nonce":1302663821}]},"public_addr":"172.21.15.76:6801/1302663821","cluster_addr":"172.21.15.76:6803/1302663821","heartbeat_back_addr":"172.21.15.76:6807/1302663821","heartbeat_front_addr":"172.21.15.76:6805/1302663821","state":["exists","up"]},{"osd":2,"uuid":"5714c8f5-ae23-473c-8cf1-db73811acded","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6809","nonce":1992938181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6811","nonce":1992938181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6815","nonce":1992938181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6813","nonce":1992938181}]},"public_addr":"172.21.15.76:6809/1992938181","cluster_addr":"172.21.15.76:6811/1992938181","heartbeat_back_addr":"172.21.15.76:6815/1992938181","heartbeat_front_addr":"172.21.15.76:6813/1992938181","state":["exists","up"]},{"osd":3,"uuid":"13204cdb-d8e6-4f97-bc6e-9561f442decf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6825","nonce":3418257163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6827","nonce":3418257163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6831","nonce":3418257163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6829","nonce":3418257163}]},"public_addr":"172.21.15.76:6825/3418257163","cluster_addr":"172.21.15.76:6827/3418257163","heartbeat_back_addr":"172.21.15.76:6831/3418257163","heartbeat_front_addr":"172.21.15.76:6829/3418257163","state":["exists","up"]},{"osd":4,"uuid":"5bea5166-1986-4d56-b5f5-ecc8cbe7f3ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6801","nonce":2125002468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6803","nonce":2125002468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6807","nonce":2125002468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6805","nonce":2125002468}]},"public_addr":"172.21.15.105:6801/2125002468","cluster_addr":"172.21.15.105:6803/2125002468","heartbeat_back_addr":"172.21.15.105:6807/2125002468","heartbeat_front_addr":"172.21.15.105:6805/2125002468","state":["exists","up"]},{"osd":5,"uuid":"96f89dcc-270d-4421-b8e8-3b92ba5f9844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6819","nonce":3413103121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6821","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6823","nonce":3413103121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6831","nonce":3413103121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6828","nonce":3413103121}]},"public_addr":"172.21.15.105:6819/3413103121","cluster_addr":"172.21.15.105:6823/3413103121","heartbeat_back_addr":"172.21.15.105:6831/3413103121","heartbeat_front_addr":"172.21.15.105:6828/3413103121","state":["exists","up"]},{"osd":6,"uuid":"93eb087a-d9ba-4096-b85d-b74d96e819a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6809","nonce":2721832954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6811","nonce":2721832954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6815","nonce":2721832954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6813","nonce":2721832954}]},"public_addr":"172.21.15.105:6809/2721832954","cluster_addr":"172.21.15.105:6811/2721832954","heartbeat_back_addr":"172.21.15.105:6815/2721832954","heartbeat_front_addr":"172.21.15.105:6813/2721832954","state":["exists","up"]},{"osd":7,"uuid":"2f80f790-ad2d-460e-a37d-47bf1771ef43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6818","nonce":548350430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6822","nonce":548350430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6829","nonce":548350430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6825","nonce":548350430}]},"public_addr":"172.21.15.105:6818/548350430","cluster_addr":"172.21.15.105:6822/548350430","heartbeat_back_addr":"172.21.15.105:6829/548350430","heartbeat_front_addr":"172.21.15.105:6825/548350430","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.106901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.072819+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.126865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.053464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:08.982614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.104825+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.112714+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.107387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:15:15.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.633+0000 7fe70afb2640 1 -- 172.21.15.76:0/3018672559 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe6f0074420 msgr2=0x7fe6f0076930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:15.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.633+0000 7fe70afb2640 1 --2- 172.21.15.76:0/3018672559 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe6f0074420 0x7fe6f0076930 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009aa0 tx=0x7fe6f4009190 comp rx=0 tx=0).stop 2024-04-25T19:15:15.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.633+0000 7fe70afb2640 1 -- 172.21.15.76:0/3018672559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7040cc9e0 msgr2=0x7fe7041520a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:15.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.633+0000 7fe70afb2640 1 --2- 172.21.15.76:0/3018672559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7040cc9e0 0x7fe7041520a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800c430 tx=0x7fe6f800c8f0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.633+0000 7fe70afb2640 1 -- 172.21.15.76:0/3018672559 shutdown_connections 2024-04-25T19:15:15.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.633+0000 7fe70afb2640 1 --2- 172.21.15.76:0/3018672559 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe6f0074420 0x7fe6f0076930 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.633+0000 7fe70afb2640 1 --2- 172.21.15.76:0/3018672559 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe704156f10 0x7fe70415b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.633+0000 7fe70afb2640 1 --2- 172.21.15.76:0/3018672559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7040cc9e0 0x7fe7041520a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.633+0000 7fe70afb2640 1 --2- 172.21.15.76:0/3018672559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe70406de20 0x7fe70414c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.633+0000 7fe70afb2640 1 -- 172.21.15.76:0/3018672559 >> 172.21.15.76:0/3018672559 conn(0x7fe7040684e0 msgr2=0x7fe704075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:15.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.634+0000 7fe70afb2640 1 -- 172.21.15.76:0/3018672559 shutdown_connections 2024-04-25T19:15:15.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.634+0000 7fe70afb2640 1 -- 172.21.15.76:0/3018672559 wait complete. 2024-04-25T19:15:15.647 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T19:15:13.846514+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T19:15:15.647 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:15:15.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.759+0000 7efc7ef6c640 1 Processor -- start 2024-04-25T19:15:15.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.759+0000 7efc7ef6c640 1 -- start start 2024-04-25T19:15:15.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.761+0000 7efc7ef6c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efc78160260 0x7efc7806cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:15.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.761+0000 7efc7ef6c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc781609a0 con 0x7efc781570d0 2024-04-25T19:15:15.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.761+0000 7efc7ef6c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc78160c80 con 0x7efc7806d2b0 2024-04-25T19:15:15.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.761+0000 7efc7ef6c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7efc781574e0 con 0x7efc78160260 2024-04-25T19:15:15.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.761+0000 7efc7cd08640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7efc781570d0 0x7efc7806e040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52808/0 (socket says 172.21.15.76:52808) 2024-04-25T19:15:15.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.761+0000 7efc7cd08640 1 -- 172.21.15.76:0/192298958 learned_addr learned my addr 172.21.15.76:0/192298958 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:15.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.761+0000 7efc77fff640 1 --2- 172.21.15.76:0/192298958 >> v2:172.21.15.105:3300/0 conn(0x7efc78160260 0x7efc7806cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:15.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.761+0000 7efc777fe640 1 -- 172.21.15.76:0/192298958 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2185978470 0 0) 0x7efc781609a0 con 0x7efc781570d0 2024-04-25T19:15:15.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.761+0000 7efc777fe640 1 -- 172.21.15.76:0/192298958 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc68003520 con 0x7efc781570d0 2024-04-25T19:15:15.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.761+0000 7efc77fff640 1 -- 172.21.15.76:0/192298958 >> v1:172.21.15.105:6789/0 conn(0x7efc7806d2b0 legacy=0x7efc780cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:15.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.761+0000 7efc777fe640 1 -- 172.21.15.76:0/192298958 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3919422767 0 0) 0x7efc78160c80 con 0x7efc7806d2b0 2024-04-25T19:15:15.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.761+0000 7efc77fff640 1 -- 172.21.15.76:0/192298958 >> v1:172.21.15.76:6789/0 conn(0x7efc781570d0 legacy=0x7efc7806e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:15.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.761+0000 7efc77fff640 1 -- 172.21.15.76:0/192298958 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc78160f60 con 0x7efc78160260 2024-04-25T19:15:15.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.762+0000 7efc77fff640 1 --2- 172.21.15.76:0/192298958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc78160260 0x7efc7806cd70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efc640097a0 tx=0x7efc6402a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a41766616829f08b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:15.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.762+0000 7efc777fe640 1 -- 172.21.15.76:0/192298958 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc64037070 con 0x7efc78160260 2024-04-25T19:15:15.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.762+0000 7efc777fe640 1 -- 172.21.15.76:0/192298958 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc64032640 con 0x7efc78160260 2024-04-25T19:15:15.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.762+0000 7efc777fe640 1 -- 172.21.15.76:0/192298958 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc640329f0 con 0x7efc78160260 2024-04-25T19:15:15.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.763+0000 7efc7ef6c640 1 -- 172.21.15.76:0/192298958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc78160260 msgr2=0x7efc7806cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:15.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.763+0000 7efc7ef6c640 1 --2- 172.21.15.76:0/192298958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc78160260 0x7efc7806cd70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efc640097a0 tx=0x7efc6402a2c0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.763+0000 7efc7ef6c640 1 -- 172.21.15.76:0/192298958 shutdown_connections 2024-04-25T19:15:15.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.763+0000 7efc7ef6c640 1 --2- 172.21.15.76:0/192298958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc78160260 0x7efc7806cd70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.763+0000 7efc7ef6c640 1 -- 172.21.15.76:0/192298958 >> 172.21.15.76:0/192298958 conn(0x7efc780d22d0 msgr2=0x7efc780d26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:15.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.763+0000 7efc7ef6c640 1 -- 172.21.15.76:0/192298958 shutdown_connections 2024-04-25T19:15:15.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.763+0000 7efc7ef6c640 1 -- 172.21.15.76:0/192298958 wait complete. 2024-04-25T19:15:15.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.765+0000 7efc7ef6c640 1 Processor -- start 2024-04-25T19:15:15.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.765+0000 7efc7ef6c640 1 -- start start 2024-04-25T19:15:15.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.766+0000 7efc7ef6c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc7806d2b0 0x7efc781507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:15.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.766+0000 7efc7ef6c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc781570d0 0x7efc7815cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:15.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.766+0000 7efc7ef6c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc78160260 0x7efc7815d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:15.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.766+0000 7efc7ef6c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efc78160670 con 0x7efc7806d2b0 2024-04-25T19:15:15.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.766+0000 7efc7ef6c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efc7806db20 con 0x7efc781570d0 2024-04-25T19:15:15.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.766+0000 7efc7ef6c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7efc780ce380 con 0x7efc78160260 2024-04-25T19:15:15.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.766+0000 7efc7cd08640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc7806d2b0 0x7efc781507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:15.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.766+0000 7efc7cd08640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc7806d2b0 0x7efc781507a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60410/0 (socket says 172.21.15.76:60410) 2024-04-25T19:15:15.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.766+0000 7efc7cd08640 1 -- 172.21.15.76:0/3352878986 learned_addr learned my addr 172.21.15.76:0/3352878986 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:15.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.766+0000 7efc77fff640 1 --2- 172.21.15.76:0/3352878986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc781570d0 0x7efc7815cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:15.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.766+0000 7efc7d509640 1 --2- 172.21.15.76:0/3352878986 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc78160260 0x7efc7815d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:15.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.767+0000 7efc7cd08640 1 -- 172.21.15.76:0/3352878986 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc78160260 msgr2=0x7efc7815d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:15.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.767+0000 7efc7cd08640 1 --2- 172.21.15.76:0/3352878986 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc78160260 0x7efc7815d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.767+0000 7efc7cd08640 1 -- 172.21.15.76:0/3352878986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc781570d0 msgr2=0x7efc7815cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:15.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.767+0000 7efc7cd08640 1 --2- 172.21.15.76:0/3352878986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc781570d0 0x7efc7815cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.767+0000 7efc7cd08640 1 -- 172.21.15.76:0/3352878986 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc78150ce0 con 0x7efc7806d2b0 2024-04-25T19:15:15.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.767+0000 7efc7cd08640 1 --2- 172.21.15.76:0/3352878986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc7806d2b0 0x7efc781507a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00c390 tx=0x7efc6c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:15.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.767+0000 7efc757fa640 1 -- 172.21.15.76:0/3352878986 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c017070 con 0x7efc7806d2b0 2024-04-25T19:15:15.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.767+0000 7efc7ef6c640 1 -- 172.21.15.76:0/3352878986 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc781f3370 con 0x7efc7806d2b0 2024-04-25T19:15:15.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.767+0000 7efc757fa640 1 -- 172.21.15.76:0/3352878986 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc6c005ce0 con 0x7efc7806d2b0 2024-04-25T19:15:15.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.767+0000 7efc757fa640 1 -- 172.21.15.76:0/3352878986 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c0072f0 con 0x7efc7806d2b0 2024-04-25T19:15:15.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.768+0000 7efc7ef6c640 1 -- 172.21.15.76:0/3352878986 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc7815afc0 con 0x7efc7806d2b0 2024-04-25T19:15:15.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.769+0000 7efc7ef6c640 1 -- 172.21.15.76:0/3352878986 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc7806a390 con 0x7efc7806d2b0 2024-04-25T19:15:15.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.769+0000 7efc757fa640 1 -- 172.21.15.76:0/3352878986 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 4) ==== 96726+0+0 (secure 0 0 0) 0x7efc6c01a070 con 0x7efc7806d2b0 2024-04-25T19:15:15.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.770+0000 7efc757fa640 1 --2- 172.21.15.76:0/3352878986 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7efc5c074450 0x7efc5c076960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:15.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.770+0000 7efc757fa640 1 -- 172.21.15.76:0/3352878986 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7efc6c093ff0 con 0x7efc7806d2b0 2024-04-25T19:15:15.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.770+0000 7efc77fff640 1 --2- 172.21.15.76:0/3352878986 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7efc5c074450 0x7efc5c076960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:15.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.771+0000 7efc77fff640 1 --2- 172.21.15.76:0/3352878986 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7efc5c074450 0x7efc5c076960 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efc64034000 tx=0x7efc6402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:15.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.775+0000 7efc757fa640 1 -- 172.21.15.76:0/3352878986 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efc6c05e600 con 0x7efc7806d2b0 2024-04-25T19:15:15.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.961+0000 7efc7ef6c640 1 -- 172.21.15.76:0/3352878986 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efc780cd790 con 0x7efc7806d2b0 2024-04-25T19:15:15.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.962+0000 7efc757fa640 1 -- 172.21.15.76:0/3352878986 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7efc6c062900 con 0x7efc7806d2b0 2024-04-25T19:15:15.965 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-04-25T19:15:15.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.964+0000 7efc7ef6c640 1 -- 172.21.15.76:0/3352878986 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7efc5c074450 msgr2=0x7efc5c076960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:15.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.964+0000 7efc7ef6c640 1 --2- 172.21.15.76:0/3352878986 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7efc5c074450 0x7efc5c076960 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efc64034000 tx=0x7efc6402e000 comp rx=0 tx=0).stop 2024-04-25T19:15:15.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.964+0000 7efc7ef6c640 1 -- 172.21.15.76:0/3352878986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc7806d2b0 msgr2=0x7efc781507a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:15.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.964+0000 7efc7ef6c640 1 --2- 172.21.15.76:0/3352878986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc7806d2b0 0x7efc781507a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00c390 tx=0x7efc6c00c850 comp rx=0 tx=0).stop 2024-04-25T19:15:15.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.964+0000 7efc7ef6c640 1 -- 172.21.15.76:0/3352878986 shutdown_connections 2024-04-25T19:15:15.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.964+0000 7efc7ef6c640 1 --2- 172.21.15.76:0/3352878986 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7efc5c074450 0x7efc5c076960 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.964+0000 7efc7ef6c640 1 --2- 172.21.15.76:0/3352878986 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc78160260 0x7efc7815d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.964+0000 7efc7ef6c640 1 --2- 172.21.15.76:0/3352878986 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc781570d0 0x7efc7815cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.964+0000 7efc7ef6c640 1 --2- 172.21.15.76:0/3352878986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc7806d2b0 0x7efc781507a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:15.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.964+0000 7efc7ef6c640 1 -- 172.21.15.76:0/3352878986 >> 172.21.15.76:0/3352878986 conn(0x7efc780d22d0 msgr2=0x7efc780cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:15.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.964+0000 7efc7ef6c640 1 -- 172.21.15.76:0/3352878986 shutdown_connections 2024-04-25T19:15:15.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:15.964+0000 7efc7ef6c640 1 -- 172.21.15.76:0/3352878986 wait complete. 2024-04-25T19:15:15.978 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T19:15:16.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.084+0000 7feac5984640 1 Processor -- start 2024-04-25T19:15:16.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.084+0000 7feac5984640 1 -- start start 2024-04-25T19:15:16.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.085+0000 7feac5984640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feac00d0810 0x7feac006f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:16.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.085+0000 7feac5984640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7feac00cea70 0x7feac00cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:16.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.085+0000 7feac5984640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7feac006e160 0x7feac006e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:16.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.085+0000 7feac5984640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7feac015ce80 con 0x7feac006e160 2024-04-25T19:15:16.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.085+0000 7feac5984640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7feac015d160 con 0x7feac00cea70 2024-04-25T19:15:16.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.085+0000 7feac5984640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7feac015d440 con 0x7feac00d0810 2024-04-25T19:15:16.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.085+0000 7feabf7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7feac006e160 0x7feac006e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:16.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.085+0000 7feabf7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7feac006e160 0x7feac006e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60426/0 (socket says 172.21.15.76:60426) 2024-04-25T19:15:16.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.085+0000 7feabf7fe640 1 -- 172.21.15.76:0/846672395 learned_addr learned my addr 172.21.15.76:0/846672395 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:16.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.085+0000 7feabeffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feac00d0810 0x7feac006f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:16.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.085+0000 7feab7fff640 1 --2- 172.21.15.76:0/846672395 >> v2:172.21.15.105:3300/0 conn(0x7feac00cea70 0x7feac00cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:16.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.086+0000 7feabf7fe640 1 -- 172.21.15.76:0/846672395 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feac00d0810 msgr2=0x7feac006f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:16.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.086+0000 7feabf7fe640 1 --2- 172.21.15.76:0/846672395 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feac00d0810 0x7feac006f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:16.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.086+0000 7feabf7fe640 1 -- 172.21.15.76:0/846672395 >> v2:172.21.15.105:3300/0 conn(0x7feac00cea70 msgr2=0x7feac00cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:16.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.086+0000 7feabf7fe640 1 --2- 172.21.15.76:0/846672395 >> v2:172.21.15.105:3300/0 conn(0x7feac00cea70 0x7feac00cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:16.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.086+0000 7feabf7fe640 1 -- 172.21.15.76:0/846672395 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac015d720 con 0x7feac006e160 2024-04-25T19:15:16.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.086+0000 7feabf7fe640 1 --2- 172.21.15.76:0/846672395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feac006e160 0x7feac006e570 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feac00d3100 tx=0x7feab802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12452183a8224ee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:16.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.087+0000 7feabe7fc640 1 -- 172.21.15.76:0/846672395 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab8036070 con 0x7feac006e160 2024-04-25T19:15:16.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.087+0000 7feabe7fc640 1 -- 172.21.15.76:0/846672395 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feab8031600 con 0x7feac006e160 2024-04-25T19:15:16.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.087+0000 7feabe7fc640 1 -- 172.21.15.76:0/846672395 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab80319d0 con 0x7feac006e160 2024-04-25T19:15:16.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.087+0000 7feac5984640 1 -- 172.21.15.76:0/846672395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feac006e160 msgr2=0x7feac006e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:16.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.087+0000 7feac5984640 1 --2- 172.21.15.76:0/846672395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feac006e160 0x7feac006e570 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feac00d3100 tx=0x7feab802a6a0 comp rx=0 tx=0).stop 2024-04-25T19:15:16.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.087+0000 7feac5984640 1 -- 172.21.15.76:0/846672395 shutdown_connections 2024-04-25T19:15:16.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.087+0000 7feac5984640 1 --2- 172.21.15.76:0/846672395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feac006e160 0x7feac006e570 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:16.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.087+0000 7feac5984640 1 --2- 172.21.15.76:0/846672395 >> v2:172.21.15.105:3300/0 conn(0x7feac00cea70 0x7feac00cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:16.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.087+0000 7feac5984640 1 --2- 172.21.15.76:0/846672395 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feac00d0810 0x7feac006f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:16.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.087+0000 7feac5984640 1 -- 172.21.15.76:0/846672395 >> 172.21.15.76:0/846672395 conn(0x7feac00b3c00 msgr2=0x7feac00b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:16.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.087+0000 7feac5984640 1 -- 172.21.15.76:0/846672395 shutdown_connections 2024-04-25T19:15:16.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.087+0000 7feac5984640 1 -- 172.21.15.76:0/846672395 wait complete. 2024-04-25T19:15:16.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.089+0000 7feac5984640 1 Processor -- start 2024-04-25T19:15:16.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.089+0000 7feac5984640 1 -- start start 2024-04-25T19:15:16.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.090+0000 7feac5984640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feac006e160 0x7feac014d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:16.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.090+0000 7feac5984640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feac00cea70 0x7feac015fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:16.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.090+0000 7feac5984640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feac00d0810 0x7feac01600c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:16.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.090+0000 7feac5984640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7feac006eab0 con 0x7feac006e160 2024-04-25T19:15:16.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.090+0000 7feac5984640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feac006dc40 con 0x7feac00d0810 2024-04-25T19:15:16.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.090+0000 7feac5984640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7feac015cb70 con 0x7feac00cea70 2024-04-25T19:15:16.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.090+0000 7feabeffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feac006e160 0x7feac014d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:16.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.090+0000 7feabeffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feac006e160 0x7feac014d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60440/0 (socket says 172.21.15.76:60440) 2024-04-25T19:15:16.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.090+0000 7feabeffd640 1 -- 172.21.15.76:0/3185868457 learned_addr learned my addr 172.21.15.76:0/3185868457 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:16.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.090+0000 7feabf7fe640 1 --2- 172.21.15.76:0/3185868457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feac00d0810 0x7feac01600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:16.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.090+0000 7feab7fff640 1 --2- 172.21.15.76:0/3185868457 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feac00cea70 0x7feac015fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:16.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.091+0000 7feabeffd640 1 -- 172.21.15.76:0/3185868457 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feac00cea70 msgr2=0x7feac015fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:16.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.091+0000 7feabeffd640 1 --2- 172.21.15.76:0/3185868457 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feac00cea70 0x7feac015fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:16.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.091+0000 7feabeffd640 1 -- 172.21.15.76:0/3185868457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feac00d0810 msgr2=0x7feac01600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:16.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.091+0000 7feabeffd640 1 --2- 172.21.15.76:0/3185868457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feac00d0810 0x7feac01600c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:16.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.091+0000 7feabeffd640 1 -- 172.21.15.76:0/3185868457 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac01f29e0 con 0x7feac006e160 2024-04-25T19:15:16.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.091+0000 7feabeffd640 1 --2- 172.21.15.76:0/3185868457 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feac006e160 0x7feac014d850 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feab000e870 tx=0x7feab000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:16.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.091+0000 7feabcff9640 1 -- 172.21.15.76:0/3185868457 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab0017070 con 0x7feac006e160 2024-04-25T19:15:16.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.091+0000 7feabcff9640 1 -- 172.21.15.76:0/3185868457 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feab000c940 con 0x7feac006e160 2024-04-25T19:15:16.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.091+0000 7feac5984640 1 -- 172.21.15.76:0/3185868457 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac0154530 con 0x7feac006e160 2024-04-25T19:15:16.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.091+0000 7feabcff9640 1 -- 172.21.15.76:0/3185868457 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab000cd30 con 0x7feac006e160 2024-04-25T19:15:16.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.091+0000 7feac5984640 1 -- 172.21.15.76:0/3185868457 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac01f2f40 con 0x7feac006e160 2024-04-25T19:15:16.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.092+0000 7feac5984640 1 -- 172.21.15.76:0/3185868457 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feac006a390 con 0x7feac006e160 2024-04-25T19:15:16.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.093+0000 7feabcff9640 1 -- 172.21.15.76:0/3185868457 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 4) ==== 96726+0+0 (secure 0 0 0) 0x7feab001a070 con 0x7feac006e160 2024-04-25T19:15:16.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.093+0000 7feabcff9640 1 --2- 172.21.15.76:0/3185868457 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7feaa0074420 0x7feaa0076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:16.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.093+0000 7feabcff9640 1 -- 172.21.15.76:0/3185868457 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7feab0094ec0 con 0x7feac006e160 2024-04-25T19:15:16.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.093+0000 7feab7fff640 1 --2- 172.21.15.76:0/3185868457 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7feaa0074420 0x7feaa0076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:16.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.094+0000 7feab7fff640 1 --2- 172.21.15.76:0/3185868457 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7feaa0074420 0x7feaa0076930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feac01f3b10 tx=0x7feaa8009190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:16.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.098+0000 7feabcff9640 1 -- 172.21.15.76:0/3185868457 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feac01f29e0 con 0x7feac006e160 2024-04-25T19:15:16.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.281+0000 7feac5984640 1 -- 172.21.15.76:0/3185868457 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7feac00d01a0 con 0x7feac006e160 2024-04-25T19:15:16.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.281+0000 7feabcff9640 1 -- 172.21.15.76:0/3185868457 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7feab005f550 con 0x7feac006e160 2024-04-25T19:15:16.284 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:16.284 INFO:teuthology.orchestra.run.smithi076.stdout:[ 2024-04-25T19:15:16.284 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:15:16.284 INFO:teuthology.orchestra.run.smithi076.stdout: "poolnum": 1, 2024-04-25T19:15:16.284 INFO:teuthology.orchestra.run.smithi076.stdout: "poolname": ".mgr" 2024-04-25T19:15:16.284 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:15:16.284 INFO:teuthology.orchestra.run.smithi076.stdout:] 2024-04-25T19:15:16.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.283+0000 7feac5984640 1 -- 172.21.15.76:0/3185868457 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7feaa0074420 msgr2=0x7feaa0076930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:16.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.283+0000 7feac5984640 1 --2- 172.21.15.76:0/3185868457 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7feaa0074420 0x7feaa0076930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feac01f3b10 tx=0x7feaa8009190 comp rx=0 tx=0).stop 2024-04-25T19:15:16.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.283+0000 7feac5984640 1 -- 172.21.15.76:0/3185868457 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feac006e160 msgr2=0x7feac014d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:16.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.283+0000 7feac5984640 1 --2- 172.21.15.76:0/3185868457 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feac006e160 0x7feac014d850 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feab000e870 tx=0x7feab000ed30 comp rx=0 tx=0).stop 2024-04-25T19:15:16.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.283+0000 7feac5984640 1 -- 172.21.15.76:0/3185868457 shutdown_connections 2024-04-25T19:15:16.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.283+0000 7feac5984640 1 --2- 172.21.15.76:0/3185868457 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7feaa0074420 0x7feaa0076930 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:16.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.283+0000 7feac5984640 1 --2- 172.21.15.76:0/3185868457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feac00d0810 0x7feac01600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:16.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.283+0000 7feac5984640 1 --2- 172.21.15.76:0/3185868457 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7feac00cea70 0x7feac015fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:16.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.283+0000 7feac5984640 1 --2- 172.21.15.76:0/3185868457 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feac006e160 0x7feac014d850 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:16.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.283+0000 7feac5984640 1 -- 172.21.15.76:0/3185868457 >> 172.21.15.76:0/3185868457 conn(0x7feac00b3c00 msgr2=0x7feac00c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:16.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.283+0000 7feac5984640 1 -- 172.21.15.76:0/3185868457 shutdown_connections 2024-04-25T19:15:16.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.283+0000 7feac5984640 1 -- 172.21.15.76:0/3185868457 wait complete. 2024-04-25T19:15:16.296 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T19:15:16.297 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T19:15:16.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.408+0000 7f4b32015640 1 Processor -- start 2024-04-25T19:15:16.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.408+0000 7f4b32015640 1 -- start start 2024-04-25T19:15:16.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.410+0000 7f4b32015640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4b2c0c6d80 0x7f4b2c156e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:16.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.410+0000 7f4b32015640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4b2c1601f0 0x7f4b2c160600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:16.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.410+0000 7f4b32015640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b2c0cb9f0 con 0x7f4b2c0cb4b0 2024-04-25T19:15:16.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.410+0000 7f4b32015640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4b2c157350 con 0x7f4b2c1601f0 2024-04-25T19:15:16.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.410+0000 7f4b32015640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f4b2c15cd10 con 0x7f4b2c0c6d80 2024-04-25T19:15:16.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.410+0000 7f4b2bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4b2c1601f0 0x7f4b2c160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:16.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.410+0000 7f4b2bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4b2c1601f0 0x7f4b2c160600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60442/0 (socket says 172.21.15.76:60442) 2024-04-25T19:15:16.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.410+0000 7f4b2bfff640 1 -- 172.21.15.76:0/37881611 learned_addr learned my addr 172.21.15.76:0/37881611 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:16.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.410+0000 7f4b2affd640 1 --2- 172.21.15.76:0/37881611 >> v2:172.21.15.105:3300/0 conn(0x7f4b2c0c6d80 0x7f4b2c156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:16.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.410+0000 7f4b2bfff640 1 -- 172.21.15.76:0/37881611 >> v2:172.21.15.105:3300/0 conn(0x7f4b2c0c6d80 msgr2=0x7f4b2c156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:16.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.410+0000 7f4b2bfff640 1 --2- 172.21.15.76:0/37881611 >> v2:172.21.15.105:3300/0 conn(0x7f4b2c0c6d80 0x7f4b2c156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:16.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.410+0000 7f4b2bfff640 1 -- 172.21.15.76:0/37881611 >> v1:172.21.15.76:6789/0 conn(0x7f4b2c0cb4b0 legacy=0x7f4b2c0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:16.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.410+0000 7f4b2bfff640 1 -- 172.21.15.76:0/37881611 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2c15cff0 con 0x7f4b2c1601f0 2024-04-25T19:15:16.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.411+0000 7f4b2bfff640 1 --2- 172.21.15.76:0/37881611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b2c1601f0 0x7f4b2c160600 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c073b90 tx=0x7f4b2402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6e217528f6e562 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:16.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.411+0000 7f4b2a7fc640 1 -- 172.21.15.76:0/37881611 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b24036070 con 0x7f4b2c1601f0 2024-04-25T19:15:16.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.411+0000 7f4b2a7fc640 1 -- 172.21.15.76:0/37881611 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b24031600 con 0x7f4b2c1601f0 2024-04-25T19:15:16.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.411+0000 7f4b2a7fc640 1 -- 172.21.15.76:0/37881611 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b240319d0 con 0x7f4b2c1601f0 2024-04-25T19:15:16.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.412+0000 7f4b32015640 1 -- 172.21.15.76:0/37881611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b2c1601f0 msgr2=0x7f4b2c160600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:16.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.412+0000 7f4b32015640 1 --2- 172.21.15.76:0/37881611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b2c1601f0 0x7f4b2c160600 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c073b90 tx=0x7f4b2402a730 comp rx=0 tx=0).stop 2024-04-25T19:15:16.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.412+0000 7f4b32015640 1 -- 172.21.15.76:0/37881611 shutdown_connections 2024-04-25T19:15:16.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.412+0000 7f4b32015640 1 --2- 172.21.15.76:0/37881611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b2c1601f0 0x7f4b2c160600 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:16.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.412+0000 7f4b32015640 1 --2- 172.21.15.76:0/37881611 >> v2:172.21.15.105:3300/0 conn(0x7f4b2c0c6d80 0x7f4b2c156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:16.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.412+0000 7f4b32015640 1 -- 172.21.15.76:0/37881611 >> 172.21.15.76:0/37881611 conn(0x7f4b2c0684e0 msgr2=0x7f4b2c0688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:16.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.412+0000 7f4b32015640 1 -- 172.21.15.76:0/37881611 shutdown_connections 2024-04-25T19:15:16.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.412+0000 7f4b32015640 1 -- 172.21.15.76:0/37881611 wait complete. 2024-04-25T19:15:16.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.413+0000 7f4b32015640 1 Processor -- start 2024-04-25T19:15:16.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.414+0000 7f4b32015640 1 -- start start 2024-04-25T19:15:16.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.415+0000 7f4b32015640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b2c0c6d80 0x7f4b2c1e7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:16.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.415+0000 7f4b32015640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b2c0cb4b0 0x7f4b2c15f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:16.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.415+0000 7f4b32015640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b2c1601f0 0x7f4b2c155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:16.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.415+0000 7f4b32015640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c160b40 con 0x7f4b2c0c6d80 2024-04-25T19:15:16.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.415+0000 7f4b32015640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c15d5b0 con 0x7f4b2c0cb4b0 2024-04-25T19:15:16.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.415+0000 7f4b32015640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f4b2c0cc280 con 0x7f4b2c1601f0 2024-04-25T19:15:16.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.415+0000 7f4b2b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b2c0c6d80 0x7f4b2c1e7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:16.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.415+0000 7f4b2b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b2c0c6d80 0x7f4b2c1e7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60444/0 (socket says 172.21.15.76:60444) 2024-04-25T19:15:16.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.415+0000 7f4b2b7fe640 1 -- 172.21.15.76:0/229555922 learned_addr learned my addr 172.21.15.76:0/229555922 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:16.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.415+0000 7f4b2affd640 1 --2- 172.21.15.76:0/229555922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b2c0cb4b0 0x7f4b2c15f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:16.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.415+0000 7f4b2bfff640 1 --2- 172.21.15.76:0/229555922 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b2c1601f0 0x7f4b2c155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:16.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.415+0000 7f4b2b7fe640 1 -- 172.21.15.76:0/229555922 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b2c1601f0 msgr2=0x7f4b2c155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:16.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.415+0000 7f4b2b7fe640 1 --2- 172.21.15.76:0/229555922 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b2c1601f0 0x7f4b2c155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:16.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.415+0000 7f4b2b7fe640 1 -- 172.21.15.76:0/229555922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b2c0cb4b0 msgr2=0x7f4b2c15f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:16.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.415+0000 7f4b2b7fe640 1 --2- 172.21.15.76:0/229555922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b2c0cb4b0 0x7f4b2c15f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:16.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.415+0000 7f4b2b7fe640 1 -- 172.21.15.76:0/229555922 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2c1f62c0 con 0x7f4b2c0c6d80 2024-04-25T19:15:16.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.415+0000 7f4b2b7fe640 1 --2- 172.21.15.76:0/229555922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b2c0c6d80 0x7f4b2c1e7100 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00c3f0 tx=0x7f4b1c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:16.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.415+0000 7f4b13fff640 1 -- 172.21.15.76:0/229555922 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c0052d0 con 0x7f4b2c0c6d80 2024-04-25T19:15:16.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.416+0000 7f4b13fff640 1 -- 172.21.15.76:0/229555922 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b1c005490 con 0x7f4b2c0c6d80 2024-04-25T19:15:16.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.416+0000 7f4b32015640 1 -- 172.21.15.76:0/229555922 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b2c1f6560 con 0x7f4b2c0c6d80 2024-04-25T19:15:16.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.416+0000 7f4b13fff640 1 -- 172.21.15.76:0/229555922 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c007260 con 0x7f4b2c0c6d80 2024-04-25T19:15:16.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.416+0000 7f4b32015640 1 -- 172.21.15.76:0/229555922 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b2c1f6b20 con 0x7f4b2c0c6d80 2024-04-25T19:15:16.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.417+0000 7f4b32015640 1 -- 172.21.15.76:0/229555922 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b2c06a390 con 0x7f4b2c0c6d80 2024-04-25T19:15:16.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.417+0000 7f4b13fff640 1 -- 172.21.15.76:0/229555922 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 4) ==== 96726+0+0 (secure 0 0 0) 0x7f4b1c007400 con 0x7f4b2c0c6d80 2024-04-25T19:15:16.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.418+0000 7f4b13fff640 1 --2- 172.21.15.76:0/229555922 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4b18074420 0x7f4b18076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:16.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.418+0000 7f4b13fff640 1 -- 172.21.15.76:0/229555922 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f4b1c094150 con 0x7f4b2c0c6d80 2024-04-25T19:15:16.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.418+0000 7f4b2affd640 1 --2- 172.21.15.76:0/229555922 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4b18074420 0x7f4b18076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:16.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.418+0000 7f4b2affd640 1 --2- 172.21.15.76:0/229555922 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4b18074420 0x7f4b18076930 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c15a560 tx=0x7f4b20009190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:16.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.423+0000 7f4b13fff640 1 -- 172.21.15.76:0/229555922 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4b1c05e750 con 0x7f4b2c0c6d80 2024-04-25T19:15:16.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.601+0000 7f4b32015640 1 -- 172.21.15.76:0/229555922 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4b2c076570 con 0x7f4b2c0c6d80 2024-04-25T19:15:16.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:16.776+0000 7f4b13fff640 1 -- 172.21.15.76:0/229555922 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4b1c05a9e0 con 0x7f4b2c0c6d80 2024-04-25T19:15:17.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.388+0000 7f4b13fff640 1 -- 172.21.15.76:0/229555922 <== mon.0 v2:172.21.15.76:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7f4b1c007220 con 0x7f4b2c0c6d80 2024-04-25T19:15:17.391 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' created 2024-04-25T19:15:17.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.391+0000 7f4b32015640 1 -- 172.21.15.76:0/229555922 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4b18074420 msgr2=0x7f4b18076930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:17.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.391+0000 7f4b32015640 1 --2- 172.21.15.76:0/229555922 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4b18074420 0x7f4b18076930 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c15a560 tx=0x7f4b20009190 comp rx=0 tx=0).stop 2024-04-25T19:15:17.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.391+0000 7f4b32015640 1 -- 172.21.15.76:0/229555922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b2c0c6d80 msgr2=0x7f4b2c1e7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:17.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.391+0000 7f4b32015640 1 --2- 172.21.15.76:0/229555922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b2c0c6d80 0x7f4b2c1e7100 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00c3f0 tx=0x7f4b1c00c8b0 comp rx=0 tx=0).stop 2024-04-25T19:15:17.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.391+0000 7f4b32015640 1 -- 172.21.15.76:0/229555922 shutdown_connections 2024-04-25T19:15:17.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.391+0000 7f4b32015640 1 --2- 172.21.15.76:0/229555922 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4b18074420 0x7f4b18076930 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:17.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.391+0000 7f4b32015640 1 --2- 172.21.15.76:0/229555922 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b2c1601f0 0x7f4b2c155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:17.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.391+0000 7f4b32015640 1 --2- 172.21.15.76:0/229555922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b2c0cb4b0 0x7f4b2c15f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:17.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.391+0000 7f4b32015640 1 --2- 172.21.15.76:0/229555922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b2c0c6d80 0x7f4b2c1e7100 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:17.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.391+0000 7f4b32015640 1 -- 172.21.15.76:0/229555922 >> 172.21.15.76:0/229555922 conn(0x7f4b2c0684e0 msgr2=0x7f4b2c15c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:17.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.391+0000 7f4b32015640 1 -- 172.21.15.76:0/229555922 shutdown_connections 2024-04-25T19:15:17.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.391+0000 7f4b32015640 1 -- 172.21.15.76:0/229555922 wait complete. 2024-04-25T19:15:17.410 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T19:15:17.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.524+0000 7faf5be92640 1 Processor -- start 2024-04-25T19:15:17.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.524+0000 7faf5be92640 1 -- start start 2024-04-25T19:15:17.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.526+0000 7faf5be92640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faf5415da00 0x7faf540cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:17.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.526+0000 7faf5be92640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf540cc9c0 con 0x7faf540ced20 2024-04-25T19:15:17.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.526+0000 7faf5be92640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf540ccc60 con 0x7faf54160c80 2024-04-25T19:15:17.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.526+0000 7faf5be92640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7faf540ccf40 con 0x7faf5415da00 2024-04-25T19:15:17.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.526+0000 7faf5a42f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faf5415da00 0x7faf540cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:17.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.526+0000 7faf5a42f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faf5415da00 0x7faf540cf130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60458/0 (socket says 172.21.15.76:60458) 2024-04-25T19:15:17.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.526+0000 7faf5a42f640 1 -- 172.21.15.76:0/648867134 learned_addr learned my addr 172.21.15.76:0/648867134 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:17.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.526+0000 7faf5a42f640 1 -- 172.21.15.76:0/648867134 >> v1:172.21.15.105:6789/0 conn(0x7faf54160c80 legacy=0x7faf54156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:17.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.526+0000 7faf5a42f640 1 -- 172.21.15.76:0/648867134 >> v1:172.21.15.76:6789/0 conn(0x7faf540ced20 legacy=0x7faf5415d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:17.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.526+0000 7faf5a42f640 1 -- 172.21.15.76:0/648867134 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf540cd220 con 0x7faf5415da00 2024-04-25T19:15:17.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.527+0000 7faf5a42f640 1 --2- 172.21.15.76:0/648867134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf5415da00 0x7faf540cf130 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faf4c009870 tx=0x7faf4c02a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2c1e2c8c3389ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:17.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.527+0000 7faf58c2c640 1 -- 172.21.15.76:0/648867134 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c037070 con 0x7faf5415da00 2024-04-25T19:15:17.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.527+0000 7faf58c2c640 1 -- 172.21.15.76:0/648867134 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf4c032640 con 0x7faf5415da00 2024-04-25T19:15:17.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.527+0000 7faf58c2c640 1 -- 172.21.15.76:0/648867134 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c032a10 con 0x7faf5415da00 2024-04-25T19:15:17.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.528+0000 7faf5be92640 1 -- 172.21.15.76:0/648867134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf5415da00 msgr2=0x7faf540cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:17.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.528+0000 7faf5be92640 1 --2- 172.21.15.76:0/648867134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf5415da00 0x7faf540cf130 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faf4c009870 tx=0x7faf4c02a3c0 comp rx=0 tx=0).stop 2024-04-25T19:15:17.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.528+0000 7faf5be92640 1 -- 172.21.15.76:0/648867134 shutdown_connections 2024-04-25T19:15:17.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.528+0000 7faf5be92640 1 --2- 172.21.15.76:0/648867134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf5415da00 0x7faf540cf130 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:17.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.528+0000 7faf5be92640 1 -- 172.21.15.76:0/648867134 >> 172.21.15.76:0/648867134 conn(0x7faf540d2350 msgr2=0x7faf540d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:17.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.528+0000 7faf5be92640 1 -- 172.21.15.76:0/648867134 shutdown_connections 2024-04-25T19:15:17.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.528+0000 7faf5be92640 1 -- 172.21.15.76:0/648867134 wait complete. 2024-04-25T19:15:17.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.529+0000 7faf5be92640 1 Processor -- start 2024-04-25T19:15:17.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.529+0000 7faf5be92640 1 -- start start 2024-04-25T19:15:17.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.531+0000 7faf5be92640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faf540ced20 0x7faf540ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:17.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.531+0000 7faf5be92640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf54160c80 0x7faf541e78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:17.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.531+0000 7faf5be92640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf541f2b20 0x7faf540c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:17.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.531+0000 7faf5be92640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faf540cd680 con 0x7faf541f2b20 2024-04-25T19:15:17.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.531+0000 7faf5be92640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faf540cd500 con 0x7faf54160c80 2024-04-25T19:15:17.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.531+0000 7faf5be92640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7faf540cfd30 con 0x7faf540ced20 2024-04-25T19:15:17.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.531+0000 7faf5a42f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf541f2b20 0x7faf540c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:17.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.531+0000 7faf5a42f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf541f2b20 0x7faf540c6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60470/0 (socket says 172.21.15.76:60470) 2024-04-25T19:15:17.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.531+0000 7faf5a42f640 1 -- 172.21.15.76:0/674751960 learned_addr learned my addr 172.21.15.76:0/674751960 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:17.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.531+0000 7faf5942d640 1 --2- 172.21.15.76:0/674751960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf54160c80 0x7faf541e78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:17.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.531+0000 7faf5a42f640 1 -- 172.21.15.76:0/674751960 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faf540ced20 msgr2=0x7faf540ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:17.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.531+0000 7faf5a42f640 1 --2- 172.21.15.76:0/674751960 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faf540ced20 0x7faf540ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:17.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.531+0000 7faf5a42f640 1 -- 172.21.15.76:0/674751960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf54160c80 msgr2=0x7faf541e78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:17.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.531+0000 7faf5a42f640 1 --2- 172.21.15.76:0/674751960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf54160c80 0x7faf541e78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:17.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.531+0000 7faf5a42f640 1 -- 172.21.15.76:0/674751960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf540cb150 con 0x7faf541f2b20 2024-04-25T19:15:17.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.531+0000 7faf5a42f640 1 --2- 172.21.15.76:0/674751960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf541f2b20 0x7faf540c6930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faf4c02a400 tx=0x7faf4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:17.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.531+0000 7faf4a7fc640 1 -- 172.21.15.76:0/674751960 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c037040 con 0x7faf541f2b20 2024-04-25T19:15:17.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.531+0000 7faf4a7fc640 1 -- 172.21.15.76:0/674751960 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf4c042070 con 0x7faf541f2b20 2024-04-25T19:15:17.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.531+0000 7faf4a7fc640 1 -- 172.21.15.76:0/674751960 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c030bd0 con 0x7faf541f2b20 2024-04-25T19:15:17.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.531+0000 7faf5be92640 1 -- 172.21.15.76:0/674751960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf541f6090 con 0x7faf541f2b20 2024-04-25T19:15:17.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.531+0000 7faf5be92640 1 -- 172.21.15.76:0/674751960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf541f6650 con 0x7faf541f2b20 2024-04-25T19:15:17.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.532+0000 7faf5be92640 1 -- 172.21.15.76:0/674751960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf5406a390 con 0x7faf541f2b20 2024-04-25T19:15:17.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.533+0000 7faf4a7fc640 1 -- 172.21.15.76:0/674751960 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faf4c032500 con 0x7faf541f2b20 2024-04-25T19:15:17.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.533+0000 7faf4a7fc640 1 --2- 172.21.15.76:0/674751960 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7faf380744e0 0x7faf380769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:17.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.534+0000 7faf4a7fc640 1 -- 172.21.15.76:0/674751960 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7faf4c0b5b10 con 0x7faf541f2b20 2024-04-25T19:15:17.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.534+0000 7faf59c2e640 1 --2- 172.21.15.76:0/674751960 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7faf380744e0 0x7faf380769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:17.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.534+0000 7faf59c2e640 1 --2- 172.21.15.76:0/674751960 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7faf380744e0 0x7faf380769f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faf500012e0 tx=0x7faf50007d50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:17.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.538+0000 7faf4a7fc640 1 -- 172.21.15.76:0/674751960 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faf4c07fe30 con 0x7faf541f2b20 2024-04-25T19:15:17.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:17.714+0000 7faf5be92640 1 -- 172.21.15.76:0/674751960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7faf540b2d50 con 0x7faf541f2b20 2024-04-25T19:15:18.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.398+0000 7faf4a7fc640 1 -- 172.21.15.76:0/674751960 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) ==== 170+0+0 (secure 0 0 0) 0x7faf4c084130 con 0x7faf541f2b20 2024-04-25T19:15:18.400 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' created 2024-04-25T19:15:18.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.400+0000 7faf5be92640 1 -- 172.21.15.76:0/674751960 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7faf380744e0 msgr2=0x7faf380769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:18.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.400+0000 7faf5be92640 1 --2- 172.21.15.76:0/674751960 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7faf380744e0 0x7faf380769f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faf500012e0 tx=0x7faf50007d50 comp rx=0 tx=0).stop 2024-04-25T19:15:18.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.400+0000 7faf5be92640 1 -- 172.21.15.76:0/674751960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf541f2b20 msgr2=0x7faf540c6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:18.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.400+0000 7faf5be92640 1 --2- 172.21.15.76:0/674751960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf541f2b20 0x7faf540c6930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faf4c02a400 tx=0x7faf4c0046c0 comp rx=0 tx=0).stop 2024-04-25T19:15:18.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.400+0000 7faf5be92640 1 -- 172.21.15.76:0/674751960 shutdown_connections 2024-04-25T19:15:18.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.400+0000 7faf5be92640 1 --2- 172.21.15.76:0/674751960 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7faf380744e0 0x7faf380769f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:18.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.400+0000 7faf5be92640 1 --2- 172.21.15.76:0/674751960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf541f2b20 0x7faf540c6930 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:18.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.400+0000 7faf5be92640 1 --2- 172.21.15.76:0/674751960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf54160c80 0x7faf541e78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:18.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.400+0000 7faf5be92640 1 --2- 172.21.15.76:0/674751960 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faf540ced20 0x7faf540ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:18.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.400+0000 7faf5be92640 1 -- 172.21.15.76:0/674751960 >> 172.21.15.76:0/674751960 conn(0x7faf540d2350 msgr2=0x7faf540d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:18.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.400+0000 7faf5be92640 1 -- 172.21.15.76:0/674751960 shutdown_connections 2024-04-25T19:15:18.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.400+0000 7faf5be92640 1 -- 172.21.15.76:0/674751960 wait complete. 2024-04-25T19:15:18.425 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T19:15:18.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.538+0000 7f924396b640 1 Processor -- start 2024-04-25T19:15:18.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.538+0000 7f924396b640 1 -- start start 2024-04-25T19:15:18.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.539+0000 7f924396b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f923c157200 0x7f923c0d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:18.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.539+0000 7f924396b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f923c0cfd50 0x7f923c0cee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:18.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.539+0000 7f924396b640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f923c15dd00 con 0x7f923c0d0160 2024-04-25T19:15:18.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.539+0000 7f924396b640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f923c15cea0 con 0x7f923c0cfd50 2024-04-25T19:15:18.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.539+0000 7f924396b640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f923c15d180 con 0x7f923c157200 2024-04-25T19:15:18.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.540+0000 7f9241f08640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f923c0d0160 0x7f923c0ccb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52830/0 (socket says 172.21.15.76:52830) 2024-04-25T19:15:18.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.540+0000 7f9241f08640 1 -- 172.21.15.76:0/3519803349 learned_addr learned my addr 172.21.15.76:0/3519803349 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:18.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.540+0000 7f9241707640 1 --2- 172.21.15.76:0/3519803349 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f923c157200 0x7f923c0d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:18.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.540+0000 7f9240f06640 1 --2- 172.21.15.76:0/3519803349 >> v2:172.21.15.105:3300/0 conn(0x7f923c0cfd50 0x7f923c0cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:18.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.540+0000 7f923bfff640 1 -- 172.21.15.76:0/3519803349 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2644947665 0 0) 0x7f923c15dd00 con 0x7f923c0d0160 2024-04-25T19:15:18.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.540+0000 7f923bfff640 1 -- 172.21.15.76:0/3519803349 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9224003520 con 0x7f923c0d0160 2024-04-25T19:15:18.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.540+0000 7f9241707640 1 -- 172.21.15.76:0/3519803349 >> v2:172.21.15.105:3300/0 conn(0x7f923c0cfd50 msgr2=0x7f923c0cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:18.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.540+0000 7f9241707640 1 --2- 172.21.15.76:0/3519803349 >> v2:172.21.15.105:3300/0 conn(0x7f923c0cfd50 0x7f923c0cee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:18.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.540+0000 7f9241707640 1 -- 172.21.15.76:0/3519803349 >> v1:172.21.15.76:6789/0 conn(0x7f923c0d0160 legacy=0x7f923c0ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:18.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.540+0000 7f9241707640 1 -- 172.21.15.76:0/3519803349 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f923c15d460 con 0x7f923c157200 2024-04-25T19:15:18.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.540+0000 7f9240f06640 1 --2- 172.21.15.76:0/3519803349 >> v2:172.21.15.105:3300/0 conn(0x7f923c0cfd50 0x7f923c0cee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:18.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.541+0000 7f9241707640 1 --2- 172.21.15.76:0/3519803349 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f923c157200 0x7f923c0d0b10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9234009d90 tx=0x7f923402a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52ac1e5e794c7e37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:18.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.541+0000 7f923bfff640 1 -- 172.21.15.76:0/3519803349 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234007740 con 0x7f923c157200 2024-04-25T19:15:18.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.541+0000 7f923bfff640 1 -- 172.21.15.76:0/3519803349 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92340078e0 con 0x7f923c157200 2024-04-25T19:15:18.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.541+0000 7f923bfff640 1 -- 172.21.15.76:0/3519803349 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234007cb0 con 0x7f923c157200 2024-04-25T19:15:18.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.542+0000 7f924396b640 1 -- 172.21.15.76:0/3519803349 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f923c157200 msgr2=0x7f923c0d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:18.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.542+0000 7f924396b640 1 --2- 172.21.15.76:0/3519803349 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f923c157200 0x7f923c0d0b10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9234009d90 tx=0x7f923402a6f0 comp rx=0 tx=0).stop 2024-04-25T19:15:18.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.542+0000 7f924396b640 1 -- 172.21.15.76:0/3519803349 shutdown_connections 2024-04-25T19:15:18.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.542+0000 7f924396b640 1 --2- 172.21.15.76:0/3519803349 >> v2:172.21.15.105:3300/0 conn(0x7f923c0cfd50 0x7f923c0cee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:18.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.542+0000 7f924396b640 1 --2- 172.21.15.76:0/3519803349 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f923c157200 0x7f923c0d0b10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:18.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.542+0000 7f924396b640 1 -- 172.21.15.76:0/3519803349 >> 172.21.15.76:0/3519803349 conn(0x7f923c0d2370 msgr2=0x7f923c0d2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:18.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.542+0000 7f924396b640 1 -- 172.21.15.76:0/3519803349 shutdown_connections 2024-04-25T19:15:18.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.542+0000 7f924396b640 1 -- 172.21.15.76:0/3519803349 wait complete. 2024-04-25T19:15:18.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.543+0000 7f924396b640 1 Processor -- start 2024-04-25T19:15:18.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.544+0000 7f924396b640 1 -- start start 2024-04-25T19:15:18.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.545+0000 7f924396b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f923c0cfd50 0x7f923c0b6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:18.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.545+0000 7f924396b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f923c0d0160 0x7f923c156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:18.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.545+0000 7f924396b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f923c157200 0x7f923c156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:18.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.545+0000 7f924396b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f923c0cd280 con 0x7f923c0cfd50 2024-04-25T19:15:18.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.545+0000 7f924396b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f923c15da20 con 0x7f923c157200 2024-04-25T19:15:18.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.545+0000 7f924396b640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f923c0cd400 con 0x7f923c0d0160 2024-04-25T19:15:18.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.545+0000 7f9241707640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f923c0cfd50 0x7f923c0b6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:18.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.545+0000 7f9241707640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f923c0cfd50 0x7f923c0b6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:60486/0 (socket says 172.21.15.76:60486) 2024-04-25T19:15:18.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.545+0000 7f9241707640 1 -- 172.21.15.76:0/537347992 learned_addr learned my addr 172.21.15.76:0/537347992 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:18.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.545+0000 7f9240f06640 1 --2- 172.21.15.76:0/537347992 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f923c0d0160 0x7f923c156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:18.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.545+0000 7f9241f08640 1 --2- 172.21.15.76:0/537347992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f923c157200 0x7f923c156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:18.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.545+0000 7f9241707640 1 -- 172.21.15.76:0/537347992 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f923c0d0160 msgr2=0x7f923c156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:18.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.545+0000 7f9241707640 1 --2- 172.21.15.76:0/537347992 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f923c0d0160 0x7f923c156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:18.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.545+0000 7f9241707640 1 -- 172.21.15.76:0/537347992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f923c157200 msgr2=0x7f923c156750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:18.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.545+0000 7f9241707640 1 --2- 172.21.15.76:0/537347992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f923c157200 0x7f923c156750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:18.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.545+0000 7f9241707640 1 -- 172.21.15.76:0/537347992 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f923c0b7370 con 0x7f923c0cfd50 2024-04-25T19:15:18.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.545+0000 7f9241707640 1 --2- 172.21.15.76:0/537347992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f923c0cfd50 0x7f923c0b6e30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9234009d90 tx=0x7f9234004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:18.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.545+0000 7f9239ffb640 1 -- 172.21.15.76:0/537347992 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234040030 con 0x7f923c0cfd50 2024-04-25T19:15:18.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.545+0000 7f9239ffb640 1 -- 172.21.15.76:0/537347992 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9234031a40 con 0x7f923c0cfd50 2024-04-25T19:15:18.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.545+0000 7f924396b640 1 -- 172.21.15.76:0/537347992 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f923c0b4cc0 con 0x7f923c0cfd50 2024-04-25T19:15:18.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.546+0000 7f9239ffb640 1 -- 172.21.15.76:0/537347992 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234009450 con 0x7f923c0cfd50 2024-04-25T19:15:18.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.546+0000 7f924396b640 1 -- 172.21.15.76:0/537347992 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f923c0b6350 con 0x7f923c0cfd50 2024-04-25T19:15:18.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.547+0000 7f924396b640 1 -- 172.21.15.76:0/537347992 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f923c06a390 con 0x7f923c0cfd50 2024-04-25T19:15:18.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.547+0000 7f9239ffb640 1 -- 172.21.15.76:0/537347992 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9234043030 con 0x7f923c0cfd50 2024-04-25T19:15:18.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.548+0000 7f9239ffb640 1 --2- 172.21.15.76:0/537347992 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f92280744b0 0x7f92280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:18.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.548+0000 7f9239ffb640 1 -- 172.21.15.76:0/537347992 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f92340b5f80 con 0x7f923c0cfd50 2024-04-25T19:15:18.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.548+0000 7f9240f06640 1 --2- 172.21.15.76:0/537347992 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f92280744b0 0x7f92280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:18.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.549+0000 7f9240f06640 1 --2- 172.21.15.76:0/537347992 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f92280744b0 0x7f92280769c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f923c0b5970 tx=0x7f9230009190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:18.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.552+0000 7f9239ffb640 1 -- 172.21.15.76:0/537347992 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9234080120 con 0x7f923c0cfd50 2024-04-25T19:15:18.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:18.767+0000 7f924396b640 1 -- 172.21.15.76:0/537347992 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f923c0b7ed0 con 0x7f923c0cfd50 2024-04-25T19:15:19.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.399+0000 7f9239ffb640 1 -- 172.21.15.76:0/537347992 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f9234084420 con 0x7f923c0cfd50 2024-04-25T19:15:19.402 INFO:teuthology.orchestra.run.smithi076.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T19:15:19.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.404+0000 7f924396b640 1 -- 172.21.15.76:0/537347992 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f92280744b0 msgr2=0x7f92280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:19.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.404+0000 7f924396b640 1 --2- 172.21.15.76:0/537347992 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f92280744b0 0x7f92280769c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f923c0b5970 tx=0x7f9230009190 comp rx=0 tx=0).stop 2024-04-25T19:15:19.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.404+0000 7f924396b640 1 -- 172.21.15.76:0/537347992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f923c0cfd50 msgr2=0x7f923c0b6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:19.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.404+0000 7f924396b640 1 --2- 172.21.15.76:0/537347992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f923c0cfd50 0x7f923c0b6e30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9234009d90 tx=0x7f9234004a40 comp rx=0 tx=0).stop 2024-04-25T19:15:19.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.406+0000 7f924396b640 1 -- 172.21.15.76:0/537347992 shutdown_connections 2024-04-25T19:15:19.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.406+0000 7f924396b640 1 --2- 172.21.15.76:0/537347992 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f92280744b0 0x7f92280769c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.406+0000 7f924396b640 1 --2- 172.21.15.76:0/537347992 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f923c157200 0x7f923c156750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.406+0000 7f924396b640 1 --2- 172.21.15.76:0/537347992 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f923c0d0160 0x7f923c156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.406+0000 7f924396b640 1 --2- 172.21.15.76:0/537347992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f923c0cfd50 0x7f923c0b6e30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.406+0000 7f924396b640 1 -- 172.21.15.76:0/537347992 >> 172.21.15.76:0/537347992 conn(0x7f923c0d2370 msgr2=0x7f923c06f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:19.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.411+0000 7f924396b640 1 -- 172.21.15.76:0/537347992 shutdown_connections 2024-04-25T19:15:19.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.411+0000 7f924396b640 1 -- 172.21.15.76:0/537347992 wait complete. 2024-04-25T19:15:19.437 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:15:19.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.545+0000 7fbd0346e640 1 Processor -- start 2024-04-25T19:15:19.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.545+0000 7fbd0346e640 1 -- start start 2024-04-25T19:15:19.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.547+0000 7fbd0346e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbcfc154d60 0x7fbcfc0ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.547+0000 7fbd0346e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcfc15af40 con 0x7fbcfc15e740 2024-04-25T19:15:19.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.547+0000 7fbd0346e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcfc15b220 con 0x7fbcfc0caf10 2024-04-25T19:15:19.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.547+0000 7fbd0346e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fbcfc15bac0 con 0x7fbcfc154d60 2024-04-25T19:15:19.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.547+0000 7fbd0120a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbcfc154d60 0x7fbcfc0ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.547+0000 7fbd0120a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbcfc154d60 0x7fbcfc0ca9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58864/0 (socket says 172.21.15.76:58864) 2024-04-25T19:15:19.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.547+0000 7fbd01a0b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fbcfc15e740 0x7fbcfc165480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35158/0 (socket says 172.21.15.76:35158) 2024-04-25T19:15:19.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.547+0000 7fbd0120a640 1 -- 172.21.15.76:0/426288668 learned_addr learned my addr 172.21.15.76:0/426288668 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:19.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.547+0000 7fbd0120a640 1 -- 172.21.15.76:0/426288668 >> v1:172.21.15.105:6789/0 conn(0x7fbcfc0caf10 legacy=0x7fbcfc072b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:19.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.547+0000 7fbd0120a640 1 -- 172.21.15.76:0/426288668 >> v1:172.21.15.76:6789/0 conn(0x7fbcfc15e740 legacy=0x7fbcfc165480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:19.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.547+0000 7fbd0120a640 1 -- 172.21.15.76:0/426288668 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcfc15b500 con 0x7fbcfc154d60 2024-04-25T19:15:19.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.548+0000 7fbd0120a640 1 --2- 172.21.15.76:0/426288668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcfc154d60 0x7fbcfc0ca9d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8009e50 tx=0x7fbcf802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba38ebd1f5236ce9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:19.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.548+0000 7fbcf3fff640 1 -- 172.21.15.76:0/426288668 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8036070 con 0x7fbcfc154d60 2024-04-25T19:15:19.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.548+0000 7fbcf3fff640 1 -- 172.21.15.76:0/426288668 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcf8031600 con 0x7fbcfc154d60 2024-04-25T19:15:19.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.548+0000 7fbcf3fff640 1 -- 172.21.15.76:0/426288668 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf80319d0 con 0x7fbcfc154d60 2024-04-25T19:15:19.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.549+0000 7fbd0346e640 1 -- 172.21.15.76:0/426288668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcfc154d60 msgr2=0x7fbcfc0ca9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:19.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.549+0000 7fbd0346e640 1 --2- 172.21.15.76:0/426288668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcfc154d60 0x7fbcfc0ca9d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8009e50 tx=0x7fbcf802a730 comp rx=0 tx=0).stop 2024-04-25T19:15:19.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.549+0000 7fbd0346e640 1 -- 172.21.15.76:0/426288668 shutdown_connections 2024-04-25T19:15:19.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.549+0000 7fbd0346e640 1 --2- 172.21.15.76:0/426288668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcfc154d60 0x7fbcfc0ca9d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.549+0000 7fbd0346e640 1 -- 172.21.15.76:0/426288668 >> 172.21.15.76:0/426288668 conn(0x7fbcfc070b00 msgr2=0x7fbcfc070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:19.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.549+0000 7fbd0346e640 1 -- 172.21.15.76:0/426288668 shutdown_connections 2024-04-25T19:15:19.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.549+0000 7fbd0346e640 1 -- 172.21.15.76:0/426288668 wait complete. 2024-04-25T19:15:19.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.550+0000 7fbd0346e640 1 Processor -- start 2024-04-25T19:15:19.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.551+0000 7fbd0346e640 1 -- start start 2024-04-25T19:15:19.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.552+0000 7fbd0346e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcfc0caf10 0x7fbcfc14b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.552+0000 7fbd0346e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbcfc154d60 0x7fbcfc152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.552+0000 7fbd0346e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcfc15e740 0x7fbcfc15c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.552+0000 7fbd0346e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc155170 con 0x7fbcfc15e740 2024-04-25T19:15:19.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.552+0000 7fbd0346e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc15e2b0 con 0x7fbcfc0caf10 2024-04-25T19:15:19.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.552+0000 7fbd0346e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fbcfc0c6330 con 0x7fbcfc154d60 2024-04-25T19:15:19.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.552+0000 7fbd01a0b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcfc15e740 0x7fbcfc15c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.552+0000 7fbd01a0b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcfc15e740 0x7fbcfc15c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58874/0 (socket says 172.21.15.76:58874) 2024-04-25T19:15:19.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.552+0000 7fbd01a0b640 1 -- 172.21.15.76:0/2848931702 learned_addr learned my addr 172.21.15.76:0/2848931702 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:19.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.552+0000 7fbd0120a640 1 --2- 172.21.15.76:0/2848931702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcfc0caf10 0x7fbcfc14b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.552+0000 7fbd00a09640 1 --2- 172.21.15.76:0/2848931702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbcfc154d60 0x7fbcfc152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.552+0000 7fbd01a0b640 1 -- 172.21.15.76:0/2848931702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbcfc154d60 msgr2=0x7fbcfc152900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:19.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.552+0000 7fbd01a0b640 1 --2- 172.21.15.76:0/2848931702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbcfc154d60 0x7fbcfc152900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.552+0000 7fbd01a0b640 1 -- 172.21.15.76:0/2848931702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcfc0caf10 msgr2=0x7fbcfc14b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:19.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.552+0000 7fbd01a0b640 1 --2- 172.21.15.76:0/2848931702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcfc0caf10 0x7fbcfc14b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.552+0000 7fbd01a0b640 1 -- 172.21.15.76:0/2848931702 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcfc152e40 con 0x7fbcfc15e740 2024-04-25T19:15:19.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.552+0000 7fbd00a09640 1 --2- 172.21.15.76:0/2848931702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbcfc154d60 0x7fbcfc152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:15:19.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.552+0000 7fbd01a0b640 1 --2- 172.21.15.76:0/2848931702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcfc15e740 0x7fbcfc15c440 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbcf400c3c0 tx=0x7fbcf400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:19.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.552+0000 7fbcf1ffb640 1 -- 172.21.15.76:0/2848931702 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf40074e0 con 0x7fbcfc15e740 2024-04-25T19:15:19.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.552+0000 7fbcf1ffb640 1 -- 172.21.15.76:0/2848931702 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcf4007680 con 0x7fbcfc15e740 2024-04-25T19:15:19.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.552+0000 7fbd0346e640 1 -- 172.21.15.76:0/2848931702 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcfc150a70 con 0x7fbcfc15e740 2024-04-25T19:15:19.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.553+0000 7fbcf1ffb640 1 -- 172.21.15.76:0/2848931702 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4004a20 con 0x7fbcfc15e740 2024-04-25T19:15:19.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.553+0000 7fbd0346e640 1 -- 172.21.15.76:0/2848931702 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcfc1f1280 con 0x7fbcfc15e740 2024-04-25T19:15:19.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.554+0000 7fbd0346e640 1 -- 172.21.15.76:0/2848931702 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcfc1f0d10 con 0x7fbcfc15e740 2024-04-25T19:15:19.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.554+0000 7fbcf1ffb640 1 -- 172.21.15.76:0/2848931702 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbcf401a020 con 0x7fbcfc15e740 2024-04-25T19:15:19.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.555+0000 7fbcf1ffb640 1 --2- 172.21.15.76:0/2848931702 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbce00744b0 0x7fbce00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.555+0000 7fbcf1ffb640 1 -- 172.21.15.76:0/2848931702 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbcf40948f0 con 0x7fbcfc15e740 2024-04-25T19:15:19.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.555+0000 7fbd0120a640 1 --2- 172.21.15.76:0/2848931702 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbce00744b0 0x7fbce00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.555+0000 7fbd0120a640 1 --2- 172.21.15.76:0/2848931702 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbce00744b0 0x7fbce00769c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8033000 tx=0x7fbcf8009000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:19.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.559+0000 7fbcf1ffb640 1 -- 172.21.15.76:0/2848931702 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbcf405f110 con 0x7fbcfc15e740 2024-04-25T19:15:19.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.734+0000 7fbd0346e640 1 -- 172.21.15.76:0/2848931702 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbcfc06a390 con 0x7fbcfc15e740 2024-04-25T19:15:19.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.735+0000 7fbcf1ffb640 1 -- 172.21.15.76:0/2848931702 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16618 (secure 0 0 0) 0x7fbcf4067100 con 0x7fbcfc15e740 2024-04-25T19:15:19.738 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:19.738 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":20,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","created":"2024-04-25T19:15:01.693137+0000","modified":"2024-04-25T19:15:18.769491+0000","last_up_change":"2024-04-25T19:15:11.891681+0000","last_in_change":"2024-04-25T19:15:03.898065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:15:13.846514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:15:16.606533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:15:17.719197+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72a16c02-48f9-45aa-9447-fcf755ebf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409574998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409574998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409574998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409574998}]},"public_addr":"172.21.15.76:6817/1409574998","cluster_addr":"172.21.15.76:6819/1409574998","heartbeat_back_addr":"172.21.15.76:6823/1409574998","heartbeat_front_addr":"172.21.15.76:6821/1409574998","state":["exists","up"]},{"osd":1,"uuid":"818d50f0-76c6-45f8-bf79-5e304f14c5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6801","nonce":1302663821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6803","nonce":1302663821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6807","nonce":1302663821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6805","nonce":1302663821}]},"public_addr":"172.21.15.76:6801/1302663821","cluster_addr":"172.21.15.76:6803/1302663821","heartbeat_back_addr":"172.21.15.76:6807/1302663821","heartbeat_front_addr":"172.21.15.76:6805/1302663821","state":["exists","up"]},{"osd":2,"uuid":"5714c8f5-ae23-473c-8cf1-db73811acded","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6809","nonce":1992938181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6811","nonce":1992938181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6815","nonce":1992938181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6813","nonce":1992938181}]},"public_addr":"172.21.15.76:6809/1992938181","cluster_addr":"172.21.15.76:6811/1992938181","heartbeat_back_addr":"172.21.15.76:6815/1992938181","heartbeat_front_addr":"172.21.15.76:6813/1992938181","state":["exists","up"]},{"osd":3,"uuid":"13204cdb-d8e6-4f97-bc6e-9561f442decf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6825","nonce":3418257163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6827","nonce":3418257163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6831","nonce":3418257163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6829","nonce":3418257163}]},"public_addr":"172.21.15.76:6825/3418257163","cluster_addr":"172.21.15.76:6827/3418257163","heartbeat_back_addr":"172.21.15.76:6831/3418257163","heartbeat_front_addr":"172.21.15.76:6829/3418257163","state":["exists","up"]},{"osd":4,"uuid":"5bea5166-1986-4d56-b5f5-ecc8cbe7f3ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6801","nonce":2125002468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6803","nonce":2125002468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6807","nonce":2125002468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6805","nonce":2125002468}]},"public_addr":"172.21.15.105:6801/2125002468","cluster_addr":"172.21.15.105:6803/2125002468","heartbeat_back_addr":"172.21.15.105:6807/2125002468","heartbeat_front_addr":"172.21.15.105:6805/2125002468","state":["exists","up"]},{"osd":5,"uuid":"96f89dcc-270d-4421-b8e8-3b92ba5f9844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6819","nonce":3413103121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6821","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6823","nonce":3413103121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6831","nonce":3413103121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6828","nonce":3413103121}]},"public_addr":"172.21.15.105:6819/3413103121","cluster_addr":"172.21.15.105:6823/3413103121","heartbeat_back_addr":"172.21.15.105:6831/3413103121","heartbeat_front_addr":"172.21.15.105:6828/3413103121","state":["exists","up"]},{"osd":6,"uuid":"93eb087a-d9ba-4096-b85d-b74d96e819a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6809","nonce":2721832954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6811","nonce":2721832954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6815","nonce":2721832954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6813","nonce":2721832954}]},"public_addr":"172.21.15.105:6809/2721832954","cluster_addr":"172.21.15.105:6811/2721832954","heartbeat_back_addr":"172.21.15.105:6815/2721832954","heartbeat_front_addr":"172.21.15.105:6813/2721832954","state":["exists","up"]},{"osd":7,"uuid":"2f80f790-ad2d-460e-a37d-47bf1771ef43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6818","nonce":548350430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6822","nonce":548350430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6829","nonce":548350430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6825","nonce":548350430}]},"public_addr":"172.21.15.105:6818/548350430","cluster_addr":"172.21.15.105:6822/548350430","heartbeat_back_addr":"172.21.15.105:6829/548350430","heartbeat_front_addr":"172.21.15.105:6825/548350430","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.106901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.072819+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.126865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.053464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:08.982614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.104825+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.112714+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.107387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:15:19.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.737+0000 7fbd0346e640 1 -- 172.21.15.76:0/2848931702 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbce00744b0 msgr2=0x7fbce00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:19.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.737+0000 7fbd0346e640 1 --2- 172.21.15.76:0/2848931702 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbce00744b0 0x7fbce00769c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8033000 tx=0x7fbcf8009000 comp rx=0 tx=0).stop 2024-04-25T19:15:19.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.737+0000 7fbd0346e640 1 -- 172.21.15.76:0/2848931702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcfc15e740 msgr2=0x7fbcfc15c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:19.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.737+0000 7fbd0346e640 1 --2- 172.21.15.76:0/2848931702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcfc15e740 0x7fbcfc15c440 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbcf400c3c0 tx=0x7fbcf400c880 comp rx=0 tx=0).stop 2024-04-25T19:15:19.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.737+0000 7fbd0346e640 1 -- 172.21.15.76:0/2848931702 shutdown_connections 2024-04-25T19:15:19.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.737+0000 7fbd0346e640 1 --2- 172.21.15.76:0/2848931702 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbce00744b0 0x7fbce00769c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.737+0000 7fbd0346e640 1 --2- 172.21.15.76:0/2848931702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcfc15e740 0x7fbcfc15c440 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.737+0000 7fbd0346e640 1 --2- 172.21.15.76:0/2848931702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbcfc154d60 0x7fbcfc152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.737+0000 7fbd0346e640 1 --2- 172.21.15.76:0/2848931702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbcfc0caf10 0x7fbcfc14b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.737+0000 7fbd0346e640 1 -- 172.21.15.76:0/2848931702 >> 172.21.15.76:0/2848931702 conn(0x7fbcfc070b00 msgr2=0x7fbcfc150d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:19.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.737+0000 7fbd0346e640 1 -- 172.21.15.76:0/2848931702 shutdown_connections 2024-04-25T19:15:19.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.737+0000 7fbd0346e640 1 -- 172.21.15.76:0/2848931702 wait complete. 2024-04-25T19:15:19.751 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:15:19.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.856+0000 7febcbc51640 1 Processor -- start 2024-04-25T19:15:19.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.856+0000 7febcbc51640 1 -- start start 2024-04-25T19:15:19.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.858+0000 7febcbc51640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7febc40cdc70 0x7febc40ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.858+0000 7febcbc51640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febc41609a0 con 0x7febc41570d0 2024-04-25T19:15:19.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.858+0000 7febcbc51640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febc4160c80 con 0x7febc4160260 2024-04-25T19:15:19.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.858+0000 7febcbc51640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7febc41574e0 con 0x7febc40cdc70 2024-04-25T19:15:19.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.858+0000 7febca1ee640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7febc40cdc70 0x7febc40ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.858+0000 7febca1ee640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7febc40cdc70 0x7febc40ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58880/0 (socket says 172.21.15.76:58880) 2024-04-25T19:15:19.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.858+0000 7febca1ee640 1 -- 172.21.15.76:0/2394875165 learned_addr learned my addr 172.21.15.76:0/2394875165 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:19.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.858+0000 7febca1ee640 1 -- 172.21.15.76:0/2394875165 >> v1:172.21.15.105:6789/0 conn(0x7febc4160260 legacy=0x7febc406cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:19.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.858+0000 7febca1ee640 1 -- 172.21.15.76:0/2394875165 >> v1:172.21.15.76:6789/0 conn(0x7febc41570d0 legacy=0x7febc406e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:19.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.858+0000 7febca1ee640 1 -- 172.21.15.76:0/2394875165 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc4160f60 con 0x7febc40cdc70 2024-04-25T19:15:19.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.859+0000 7febca1ee640 1 --2- 172.21.15.76:0/2394875165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febc40cdc70 0x7febc40ce080 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7febbc009870 tx=0x7febbc02a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884d42696f53865a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:19.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.859+0000 7febc89eb640 1 -- 172.21.15.76:0/2394875165 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc037070 con 0x7febc40cdc70 2024-04-25T19:15:19.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.859+0000 7febc89eb640 1 -- 172.21.15.76:0/2394875165 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febbc032640 con 0x7febc40cdc70 2024-04-25T19:15:19.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.859+0000 7febc89eb640 1 -- 172.21.15.76:0/2394875165 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc032a10 con 0x7febc40cdc70 2024-04-25T19:15:19.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.860+0000 7febcbc51640 1 -- 172.21.15.76:0/2394875165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febc40cdc70 msgr2=0x7febc40ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:19.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.860+0000 7febcbc51640 1 --2- 172.21.15.76:0/2394875165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febc40cdc70 0x7febc40ce080 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7febbc009870 tx=0x7febbc02a330 comp rx=0 tx=0).stop 2024-04-25T19:15:19.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.860+0000 7febcbc51640 1 -- 172.21.15.76:0/2394875165 shutdown_connections 2024-04-25T19:15:19.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.860+0000 7febcbc51640 1 --2- 172.21.15.76:0/2394875165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febc40cdc70 0x7febc40ce080 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.860+0000 7febcbc51640 1 -- 172.21.15.76:0/2394875165 >> 172.21.15.76:0/2394875165 conn(0x7febc40d22d0 msgr2=0x7febc40d26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:19.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.860+0000 7febcbc51640 1 -- 172.21.15.76:0/2394875165 shutdown_connections 2024-04-25T19:15:19.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.860+0000 7febcbc51640 1 -- 172.21.15.76:0/2394875165 wait complete. 2024-04-25T19:15:19.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.861+0000 7febcbc51640 1 Processor -- start 2024-04-25T19:15:19.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.861+0000 7febcbc51640 1 -- start start 2024-04-25T19:15:19.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.863+0000 7febcbc51640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7febc40cdc70 0x7febc41e8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.863+0000 7febcbc51640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febc41570d0 0x7febc41522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.863+0000 7febcbc51640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febc4160260 0x7febc4152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.863+0000 7febcbc51640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7febc4160670 con 0x7febc4160260 2024-04-25T19:15:19.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.863+0000 7febcbc51640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7febc406db20 con 0x7febc41570d0 2024-04-25T19:15:19.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.863+0000 7febcbc51640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7febc406d480 con 0x7febc40cdc70 2024-04-25T19:15:19.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.863+0000 7febca1ee640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febc4160260 0x7febc4152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.863+0000 7febca1ee640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febc4160260 0x7febc4152830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58882/0 (socket says 172.21.15.76:58882) 2024-04-25T19:15:19.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.863+0000 7febca1ee640 1 -- 172.21.15.76:0/2481412467 learned_addr learned my addr 172.21.15.76:0/2481412467 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:19.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.863+0000 7febc99ed640 1 --2- 172.21.15.76:0/2481412467 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7febc40cdc70 0x7febc41e8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.863+0000 7febc91ec640 1 --2- 172.21.15.76:0/2481412467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febc41570d0 0x7febc41522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.863+0000 7febca1ee640 1 -- 172.21.15.76:0/2481412467 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7febc40cdc70 msgr2=0x7febc41e8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:19.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.863+0000 7febca1ee640 1 --2- 172.21.15.76:0/2481412467 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7febc40cdc70 0x7febc41e8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.863+0000 7febca1ee640 1 -- 172.21.15.76:0/2481412467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febc41570d0 msgr2=0x7febc41522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:19.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.863+0000 7febca1ee640 1 --2- 172.21.15.76:0/2481412467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febc41570d0 0x7febc41522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:19.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.863+0000 7febca1ee640 1 -- 172.21.15.76:0/2481412467 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc41e9170 con 0x7febc4160260 2024-04-25T19:15:19.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.863+0000 7febca1ee640 1 --2- 172.21.15.76:0/2481412467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febc4160260 0x7febc4152830 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7febbc02a370 tx=0x7febbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:19.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.863+0000 7febba7fc640 1 -- 172.21.15.76:0/2481412467 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc041020 con 0x7febc4160260 2024-04-25T19:15:19.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.863+0000 7febba7fc640 1 -- 172.21.15.76:0/2481412467 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febbc030990 con 0x7febc4160260 2024-04-25T19:15:19.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.863+0000 7febcbc51640 1 -- 172.21.15.76:0/2481412467 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc415d820 con 0x7febc4160260 2024-04-25T19:15:19.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.863+0000 7febba7fc640 1 -- 172.21.15.76:0/2481412467 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc037040 con 0x7febc4160260 2024-04-25T19:15:19.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.863+0000 7febcbc51640 1 -- 172.21.15.76:0/2481412467 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc415afe0 con 0x7febc4160260 2024-04-25T19:15:19.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.864+0000 7febcbc51640 1 -- 172.21.15.76:0/2481412467 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febc415a4f0 con 0x7febc4160260 2024-04-25T19:15:19.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.865+0000 7febba7fc640 1 -- 172.21.15.76:0/2481412467 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7febbc044020 con 0x7febc4160260 2024-04-25T19:15:19.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.866+0000 7febba7fc640 1 --2- 172.21.15.76:0/2481412467 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7feba80744b0 0x7feba80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:19.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.866+0000 7febba7fc640 1 -- 172.21.15.76:0/2481412467 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7febbc0b5b00 con 0x7febc4160260 2024-04-25T19:15:19.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.866+0000 7febc99ed640 1 --2- 172.21.15.76:0/2481412467 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7feba80744b0 0x7feba80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:19.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.866+0000 7febc99ed640 1 --2- 172.21.15.76:0/2481412467 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7feba80744b0 0x7feba80769c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7febc0001e70 tx=0x7febc0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:19.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:19.870+0000 7febba7fc640 1 -- 172.21.15.76:0/2481412467 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7febbc04b030 con 0x7febc4160260 2024-04-25T19:15:20.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.048+0000 7febcbc51640 1 -- 172.21.15.76:0/2481412467 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7febc406a390 con 0x7febc4160260 2024-04-25T19:15:20.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.049+0000 7febba7fc640 1 -- 172.21.15.76:0/2481412467 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16618 (secure 0 0 0) 0x7febbc07fc40 con 0x7febc4160260 2024-04-25T19:15:20.052 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:20.052 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":20,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","created":"2024-04-25T19:15:01.693137+0000","modified":"2024-04-25T19:15:18.769491+0000","last_up_change":"2024-04-25T19:15:11.891681+0000","last_in_change":"2024-04-25T19:15:03.898065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:15:13.846514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:15:16.606533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:15:17.719197+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72a16c02-48f9-45aa-9447-fcf755ebf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409574998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409574998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409574998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409574998}]},"public_addr":"172.21.15.76:6817/1409574998","cluster_addr":"172.21.15.76:6819/1409574998","heartbeat_back_addr":"172.21.15.76:6823/1409574998","heartbeat_front_addr":"172.21.15.76:6821/1409574998","state":["exists","up"]},{"osd":1,"uuid":"818d50f0-76c6-45f8-bf79-5e304f14c5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6801","nonce":1302663821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6803","nonce":1302663821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6807","nonce":1302663821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6805","nonce":1302663821}]},"public_addr":"172.21.15.76:6801/1302663821","cluster_addr":"172.21.15.76:6803/1302663821","heartbeat_back_addr":"172.21.15.76:6807/1302663821","heartbeat_front_addr":"172.21.15.76:6805/1302663821","state":["exists","up"]},{"osd":2,"uuid":"5714c8f5-ae23-473c-8cf1-db73811acded","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6809","nonce":1992938181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6811","nonce":1992938181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6815","nonce":1992938181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6813","nonce":1992938181}]},"public_addr":"172.21.15.76:6809/1992938181","cluster_addr":"172.21.15.76:6811/1992938181","heartbeat_back_addr":"172.21.15.76:6815/1992938181","heartbeat_front_addr":"172.21.15.76:6813/1992938181","state":["exists","up"]},{"osd":3,"uuid":"13204cdb-d8e6-4f97-bc6e-9561f442decf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6825","nonce":3418257163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6827","nonce":3418257163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6831","nonce":3418257163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6829","nonce":3418257163}]},"public_addr":"172.21.15.76:6825/3418257163","cluster_addr":"172.21.15.76:6827/3418257163","heartbeat_back_addr":"172.21.15.76:6831/3418257163","heartbeat_front_addr":"172.21.15.76:6829/3418257163","state":["exists","up"]},{"osd":4,"uuid":"5bea5166-1986-4d56-b5f5-ecc8cbe7f3ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6801","nonce":2125002468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6803","nonce":2125002468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6807","nonce":2125002468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6805","nonce":2125002468}]},"public_addr":"172.21.15.105:6801/2125002468","cluster_addr":"172.21.15.105:6803/2125002468","heartbeat_back_addr":"172.21.15.105:6807/2125002468","heartbeat_front_addr":"172.21.15.105:6805/2125002468","state":["exists","up"]},{"osd":5,"uuid":"96f89dcc-270d-4421-b8e8-3b92ba5f9844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6819","nonce":3413103121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6821","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6823","nonce":3413103121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6831","nonce":3413103121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6828","nonce":3413103121}]},"public_addr":"172.21.15.105:6819/3413103121","cluster_addr":"172.21.15.105:6823/3413103121","heartbeat_back_addr":"172.21.15.105:6831/3413103121","heartbeat_front_addr":"172.21.15.105:6828/3413103121","state":["exists","up"]},{"osd":6,"uuid":"93eb087a-d9ba-4096-b85d-b74d96e819a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6809","nonce":2721832954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6811","nonce":2721832954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6815","nonce":2721832954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6813","nonce":2721832954}]},"public_addr":"172.21.15.105:6809/2721832954","cluster_addr":"172.21.15.105:6811/2721832954","heartbeat_back_addr":"172.21.15.105:6815/2721832954","heartbeat_front_addr":"172.21.15.105:6813/2721832954","state":["exists","up"]},{"osd":7,"uuid":"2f80f790-ad2d-460e-a37d-47bf1771ef43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6818","nonce":548350430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6822","nonce":548350430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6829","nonce":548350430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6825","nonce":548350430}]},"public_addr":"172.21.15.105:6818/548350430","cluster_addr":"172.21.15.105:6822/548350430","heartbeat_back_addr":"172.21.15.105:6829/548350430","heartbeat_front_addr":"172.21.15.105:6825/548350430","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.106901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.072819+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.126865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.053464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:08.982614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.104825+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.112714+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.107387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:15:20.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.051+0000 7febcbc51640 1 -- 172.21.15.76:0/2481412467 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7feba80744b0 msgr2=0x7feba80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:20.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.051+0000 7febcbc51640 1 --2- 172.21.15.76:0/2481412467 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7feba80744b0 0x7feba80769c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7febc0001e70 tx=0x7febc0009040 comp rx=0 tx=0).stop 2024-04-25T19:15:20.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.051+0000 7febcbc51640 1 -- 172.21.15.76:0/2481412467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febc4160260 msgr2=0x7febc4152830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:20.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.051+0000 7febcbc51640 1 --2- 172.21.15.76:0/2481412467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febc4160260 0x7febc4152830 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7febbc02a370 tx=0x7febbc0046c0 comp rx=0 tx=0).stop 2024-04-25T19:15:20.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.051+0000 7febcbc51640 1 -- 172.21.15.76:0/2481412467 shutdown_connections 2024-04-25T19:15:20.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.051+0000 7febcbc51640 1 --2- 172.21.15.76:0/2481412467 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7feba80744b0 0x7feba80769c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:20.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.051+0000 7febcbc51640 1 --2- 172.21.15.76:0/2481412467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febc4160260 0x7febc4152830 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:20.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.051+0000 7febcbc51640 1 --2- 172.21.15.76:0/2481412467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febc41570d0 0x7febc41522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:20.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.051+0000 7febcbc51640 1 --2- 172.21.15.76:0/2481412467 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7febc40cdc70 0x7febc41e8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:20.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.051+0000 7febcbc51640 1 -- 172.21.15.76:0/2481412467 >> 172.21.15.76:0/2481412467 conn(0x7febc40d22d0 msgr2=0x7febc40d26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:20.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.051+0000 7febcbc51640 1 -- 172.21.15.76:0/2481412467 shutdown_connections 2024-04-25T19:15:20.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.051+0000 7febcbc51640 1 -- 172.21.15.76:0/2481412467 wait complete. 2024-04-25T19:15:20.065 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T19:15:20.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.172+0000 7fce647d7640 1 Processor -- start 2024-04-25T19:15:20.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.172+0000 7fce647d7640 1 -- start start 2024-04-25T19:15:20.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.173+0000 7fce647d7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fce5c1600b0 0x7fce5c15d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:20.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.173+0000 7fce647d7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce5c0cce00 0x7fce5c0cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:20.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.173+0000 7fce647d7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce5c1606f0 con 0x7fce5c0ca750 2024-04-25T19:15:20.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.173+0000 7fce647d7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fce5c1604c0 con 0x7fce5c1600b0 2024-04-25T19:15:20.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.173+0000 7fce647d7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fce5c1609d0 con 0x7fce5c0cce00 2024-04-25T19:15:20.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.173+0000 7fce62573640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fce5c1600b0 0x7fce5c15d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:20.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.173+0000 7fce62573640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fce5c1600b0 0x7fce5c15d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58886/0 (socket says 172.21.15.76:58886) 2024-04-25T19:15:20.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.173+0000 7fce62573640 1 -- 172.21.15.76:0/2143211104 learned_addr learned my addr 172.21.15.76:0/2143211104 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:20.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.173+0000 7fce62d74640 1 --2- 172.21.15.76:0/2143211104 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce5c0cce00 0x7fce5c0cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:20.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.174+0000 7fce62573640 1 -- 172.21.15.76:0/2143211104 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce5c0cce00 msgr2=0x7fce5c0cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:20.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.174+0000 7fce62573640 1 --2- 172.21.15.76:0/2143211104 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce5c0cce00 0x7fce5c0cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:20.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.174+0000 7fce62573640 1 -- 172.21.15.76:0/2143211104 >> v1:172.21.15.76:6789/0 conn(0x7fce5c0ca750 legacy=0x7fce5c0cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:20.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.174+0000 7fce62573640 1 -- 172.21.15.76:0/2143211104 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5c160cb0 con 0x7fce5c1600b0 2024-04-25T19:15:20.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.174+0000 7fce62573640 1 --2- 172.21.15.76:0/2143211104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce5c1600b0 0x7fce5c15d3d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fce58009e50 tx=0x7fce5802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ddb02a7f3ab7f4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:20.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.174+0000 7fce61571640 1 -- 172.21.15.76:0/2143211104 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58036070 con 0x7fce5c1600b0 2024-04-25T19:15:20.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.174+0000 7fce61571640 1 -- 172.21.15.76:0/2143211104 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce58031600 con 0x7fce5c1600b0 2024-04-25T19:15:20.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.174+0000 7fce61571640 1 -- 172.21.15.76:0/2143211104 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce580319d0 con 0x7fce5c1600b0 2024-04-25T19:15:20.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.175+0000 7fce647d7640 1 -- 172.21.15.76:0/2143211104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce5c1600b0 msgr2=0x7fce5c15d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:20.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.175+0000 7fce647d7640 1 --2- 172.21.15.76:0/2143211104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce5c1600b0 0x7fce5c15d3d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fce58009e50 tx=0x7fce5802a7a0 comp rx=0 tx=0).stop 2024-04-25T19:15:20.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.175+0000 7fce647d7640 1 -- 172.21.15.76:0/2143211104 shutdown_connections 2024-04-25T19:15:20.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.175+0000 7fce647d7640 1 --2- 172.21.15.76:0/2143211104 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce5c0cce00 0x7fce5c0cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:20.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.175+0000 7fce647d7640 1 --2- 172.21.15.76:0/2143211104 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce5c1600b0 0x7fce5c15d3d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:20.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.175+0000 7fce647d7640 1 -- 172.21.15.76:0/2143211104 >> 172.21.15.76:0/2143211104 conn(0x7fce5c068b10 msgr2=0x7fce5c072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:20.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.175+0000 7fce647d7640 1 -- 172.21.15.76:0/2143211104 shutdown_connections 2024-04-25T19:15:20.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.175+0000 7fce647d7640 1 -- 172.21.15.76:0/2143211104 wait complete. 2024-04-25T19:15:20.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.177+0000 7fce647d7640 1 Processor -- start 2024-04-25T19:15:20.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.177+0000 7fce647d7640 1 -- start start 2024-04-25T19:15:20.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.178+0000 7fce647d7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce5c0ca750 0x7fce5c159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:20.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.178+0000 7fce647d7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce5c0cce00 0x7fce5c155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:20.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.178+0000 7fce647d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce5c1600b0 0x7fce5c0d0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:20.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.178+0000 7fce647d7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c160f90 con 0x7fce5c0ca750 2024-04-25T19:15:20.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.178+0000 7fce647d7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c15ceb0 con 0x7fce5c1600b0 2024-04-25T19:15:20.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.178+0000 7fce647d7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fce5c0cb3f0 con 0x7fce5c0cce00 2024-04-25T19:15:20.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.178+0000 7fce62573640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce5c0ca750 0x7fce5c159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:20.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.178+0000 7fce62573640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce5c0ca750 0x7fce5c159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58892/0 (socket says 172.21.15.76:58892) 2024-04-25T19:15:20.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.178+0000 7fce62573640 1 -- 172.21.15.76:0/1080349430 learned_addr learned my addr 172.21.15.76:0/1080349430 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:20.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.178+0000 7fce62d74640 1 --2- 172.21.15.76:0/1080349430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce5c1600b0 0x7fce5c0d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:20.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.178+0000 7fce61d72640 1 --2- 172.21.15.76:0/1080349430 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce5c0cce00 0x7fce5c155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:20.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.178+0000 7fce62573640 1 -- 172.21.15.76:0/1080349430 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce5c0cce00 msgr2=0x7fce5c155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:20.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.178+0000 7fce62573640 1 --2- 172.21.15.76:0/1080349430 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce5c0cce00 0x7fce5c155860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:20.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.178+0000 7fce62573640 1 -- 172.21.15.76:0/1080349430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce5c1600b0 msgr2=0x7fce5c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:20.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.178+0000 7fce62573640 1 --2- 172.21.15.76:0/1080349430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce5c1600b0 0x7fce5c0d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:20.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.179+0000 7fce62573640 1 -- 172.21.15.76:0/1080349430 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5c1e85d0 con 0x7fce5c0ca750 2024-04-25T19:15:20.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.179+0000 7fce62d74640 1 --2- 172.21.15.76:0/1080349430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce5c1600b0 0x7fce5c0d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:15:20.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.179+0000 7fce62573640 1 --2- 172.21.15.76:0/1080349430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce5c0ca750 0x7fce5c159da0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fce58009e50 tx=0x7fce58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:20.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.179+0000 7fce52ffd640 1 -- 172.21.15.76:0/1080349430 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58036050 con 0x7fce5c0ca750 2024-04-25T19:15:20.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.179+0000 7fce52ffd640 1 -- 172.21.15.76:0/1080349430 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce58042070 con 0x7fce5c0ca750 2024-04-25T19:15:20.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.179+0000 7fce647d7640 1 -- 172.21.15.76:0/1080349430 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce5c1e8b70 con 0x7fce5c0ca750 2024-04-25T19:15:20.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.179+0000 7fce52ffd640 1 -- 172.21.15.76:0/1080349430 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58009510 con 0x7fce5c0ca750 2024-04-25T19:15:20.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.179+0000 7fce647d7640 1 -- 172.21.15.76:0/1080349430 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce5c1f6490 con 0x7fce5c0ca750 2024-04-25T19:15:20.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.180+0000 7fce647d7640 1 -- 172.21.15.76:0/1080349430 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce5c06a390 con 0x7fce5c0ca750 2024-04-25T19:15:20.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.181+0000 7fce52ffd640 1 -- 172.21.15.76:0/1080349430 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fce58007c50 con 0x7fce5c0ca750 2024-04-25T19:15:20.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.181+0000 7fce52ffd640 1 --2- 172.21.15.76:0/1080349430 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fce400744b0 0x7fce400769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:20.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.181+0000 7fce52ffd640 1 -- 172.21.15.76:0/1080349430 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fce580b60e0 con 0x7fce5c0ca750 2024-04-25T19:15:20.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.181+0000 7fce61d72640 1 --2- 172.21.15.76:0/1080349430 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fce400744b0 0x7fce400769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:20.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.182+0000 7fce61d72640 1 --2- 172.21.15.76:0/1080349430 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fce400744b0 0x7fce400769c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fce5c1f3d30 tx=0x7fce4c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:20.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.186+0000 7fce52ffd640 1 -- 172.21.15.76:0/1080349430 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fce58080870 con 0x7fce5c0ca750 2024-04-25T19:15:20.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:20.401+0000 7fce647d7640 1 -- 172.21.15.76:0/1080349430 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fce5c0cb1d0 con 0x7fce5c0ca750 2024-04-25T19:15:21.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.403+0000 7fce52ffd640 1 -- 172.21.15.76:0/1080349430 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) ==== 118+0+0 (secure 0 0 0) 0x7fce58080170 con 0x7fce5c0ca750 2024-04-25T19:15:21.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.406+0000 7fce647d7640 1 -- 172.21.15.76:0/1080349430 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fce400744b0 msgr2=0x7fce400769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:21.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.406+0000 7fce647d7640 1 --2- 172.21.15.76:0/1080349430 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fce400744b0 0x7fce400769c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fce5c1f3d30 tx=0x7fce4c009040 comp rx=0 tx=0).stop 2024-04-25T19:15:21.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.406+0000 7fce647d7640 1 -- 172.21.15.76:0/1080349430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce5c0ca750 msgr2=0x7fce5c159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:21.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.406+0000 7fce647d7640 1 --2- 172.21.15.76:0/1080349430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce5c0ca750 0x7fce5c159da0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fce58009e50 tx=0x7fce58004a40 comp rx=0 tx=0).stop 2024-04-25T19:15:21.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.406+0000 7fce647d7640 1 -- 172.21.15.76:0/1080349430 shutdown_connections 2024-04-25T19:15:21.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.406+0000 7fce647d7640 1 --2- 172.21.15.76:0/1080349430 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fce400744b0 0x7fce400769c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:21.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.406+0000 7fce647d7640 1 --2- 172.21.15.76:0/1080349430 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce5c1600b0 0x7fce5c0d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:21.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.406+0000 7fce647d7640 1 --2- 172.21.15.76:0/1080349430 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fce5c0cce00 0x7fce5c155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:21.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.406+0000 7fce647d7640 1 --2- 172.21.15.76:0/1080349430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce5c0ca750 0x7fce5c159da0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:21.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.406+0000 7fce647d7640 1 -- 172.21.15.76:0/1080349430 >> 172.21.15.76:0/1080349430 conn(0x7fce5c068b10 msgr2=0x7fce5c150810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:21.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.406+0000 7fce647d7640 1 -- 172.21.15.76:0/1080349430 shutdown_connections 2024-04-25T19:15:21.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.406+0000 7fce647d7640 1 -- 172.21.15.76:0/1080349430 wait complete. 2024-04-25T19:15:21.421 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300} 2024-04-25T19:15:21.421 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-25T19:15:21.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.536+0000 7f194e19f640 1 Processor -- start 2024-04-25T19:15:21.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.536+0000 7f194e19f640 1 -- start start 2024-04-25T19:15:21.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.538+0000 7f194e19f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f19480c7fd0 0x7f1948156b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:21.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.538+0000 7f194e19f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f19480ceb50 0x7f194815c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:21.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.538+0000 7f194e19f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19480cef60 con 0x7f19480ce740 2024-04-25T19:15:21.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.538+0000 7f194e19f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f19480ce3b0 con 0x7f19480c7fd0 2024-04-25T19:15:21.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.538+0000 7f194e19f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f19480cf1f0 con 0x7f19480ceb50 2024-04-25T19:15:21.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.538+0000 7f194d19d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f19480c7fd0 0x7f1948156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:21.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.538+0000 7f194d19d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f19480c7fd0 0x7f1948156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58902/0 (socket says 172.21.15.76:58902) 2024-04-25T19:15:21.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.538+0000 7f194d19d640 1 -- 172.21.15.76:0/2945956998 learned_addr learned my addr 172.21.15.76:0/2945956998 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:21.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.538+0000 7f194d99e640 1 --2- 172.21.15.76:0/2945956998 >> v2:172.21.15.105:3300/0 conn(0x7f19480ceb50 0x7f194815c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:21.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.538+0000 7f194d19d640 1 -- 172.21.15.76:0/2945956998 >> v2:172.21.15.105:3300/0 conn(0x7f19480ceb50 msgr2=0x7f194815c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:21.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.538+0000 7f194d19d640 1 --2- 172.21.15.76:0/2945956998 >> v2:172.21.15.105:3300/0 conn(0x7f19480ceb50 0x7f194815c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:21.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.538+0000 7f194d19d640 1 -- 172.21.15.76:0/2945956998 >> v1:172.21.15.105:6789/0 conn(0x7f19480ce740 legacy=0x7f19480cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:21.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.538+0000 7f194d19d640 1 -- 172.21.15.76:0/2945956998 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19480cd7a0 con 0x7f19480c7fd0 2024-04-25T19:15:21.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.539+0000 7f194d19d640 1 --2- 172.21.15.76:0/2945956998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19480c7fd0 0x7f1948156b90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f194808fe30 tx=0x7f194402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc49e0c8bb928a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:21.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.539+0000 7f193ffff640 1 -- 172.21.15.76:0/2945956998 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1944036070 con 0x7f19480c7fd0 2024-04-25T19:15:21.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.539+0000 7f193ffff640 1 -- 172.21.15.76:0/2945956998 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1944031600 con 0x7f19480c7fd0 2024-04-25T19:15:21.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.539+0000 7f193ffff640 1 -- 172.21.15.76:0/2945956998 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19440319d0 con 0x7f19480c7fd0 2024-04-25T19:15:21.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.540+0000 7f194e19f640 1 -- 172.21.15.76:0/2945956998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19480c7fd0 msgr2=0x7f1948156b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:21.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.540+0000 7f194e19f640 1 --2- 172.21.15.76:0/2945956998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19480c7fd0 0x7f1948156b90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f194808fe30 tx=0x7f194402a780 comp rx=0 tx=0).stop 2024-04-25T19:15:21.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.540+0000 7f194e19f640 1 -- 172.21.15.76:0/2945956998 shutdown_connections 2024-04-25T19:15:21.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.540+0000 7f194e19f640 1 --2- 172.21.15.76:0/2945956998 >> v2:172.21.15.105:3300/0 conn(0x7f19480ceb50 0x7f194815c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:21.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.540+0000 7f194e19f640 1 --2- 172.21.15.76:0/2945956998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19480c7fd0 0x7f1948156b90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:21.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.540+0000 7f194e19f640 1 -- 172.21.15.76:0/2945956998 >> 172.21.15.76:0/2945956998 conn(0x7f19480684e0 msgr2=0x7f19480688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:21.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.540+0000 7f194e19f640 1 -- 172.21.15.76:0/2945956998 shutdown_connections 2024-04-25T19:15:21.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.540+0000 7f194e19f640 1 -- 172.21.15.76:0/2945956998 wait complete. 2024-04-25T19:15:21.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.541+0000 7f194e19f640 1 Processor -- start 2024-04-25T19:15:21.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.541+0000 7f194e19f640 1 -- start start 2024-04-25T19:15:21.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.543+0000 7f194e19f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19480c7fd0 0x7f19481e8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:21.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.543+0000 7f194e19f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19480ce740 0x7f194814e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:21.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.543+0000 7f194e19f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19480ceb50 0x7f194815c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:21.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.543+0000 7f194e19f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f19480ce040 con 0x7f19480c7fd0 2024-04-25T19:15:21.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.543+0000 7f194e19f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f19480cdd60 con 0x7f19480ce740 2024-04-25T19:15:21.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.543+0000 7f194e19f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f19480cd0b0 con 0x7f19480ceb50 2024-04-25T19:15:21.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.543+0000 7f194d19d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19480c7fd0 0x7f19481e8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:21.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.543+0000 7f194d19d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19480c7fd0 0x7f19481e8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58916/0 (socket says 172.21.15.76:58916) 2024-04-25T19:15:21.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.543+0000 7f194d19d640 1 -- 172.21.15.76:0/2341722760 learned_addr learned my addr 172.21.15.76:0/2341722760 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:21.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.543+0000 7f194d99e640 1 --2- 172.21.15.76:0/2341722760 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19480ceb50 0x7f194815c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:21.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.543+0000 7f194d19d640 1 -- 172.21.15.76:0/2341722760 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19480ceb50 msgr2=0x7f194815c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:21.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.543+0000 7f194d19d640 1 --2- 172.21.15.76:0/2341722760 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19480ceb50 0x7f194815c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:21.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.543+0000 7f194d19d640 1 -- 172.21.15.76:0/2341722760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19480ce740 msgr2=0x7f194814e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:21.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.543+0000 7f194c99c640 1 --2- 172.21.15.76:0/2341722760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19480ce740 0x7f194814e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:21.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.543+0000 7f194d19d640 1 --2- 172.21.15.76:0/2341722760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19480ce740 0x7f194814e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:21.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.543+0000 7f194d19d640 1 -- 172.21.15.76:0/2341722760 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19481e8bb0 con 0x7f19480c7fd0 2024-04-25T19:15:21.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.543+0000 7f194d99e640 1 --2- 172.21.15.76:0/2341722760 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19480ceb50 0x7f194815c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:15:21.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.543+0000 7f194c99c640 1 --2- 172.21.15.76:0/2341722760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19480ce740 0x7f194814e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:15:21.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.543+0000 7f194d19d640 1 --2- 172.21.15.76:0/2341722760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19480c7fd0 0x7f19481e8670 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1944009f40 tx=0x7f1944004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:21.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.543+0000 7f193dffb640 1 -- 172.21.15.76:0/2341722760 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1944036050 con 0x7f19480c7fd0 2024-04-25T19:15:21.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.543+0000 7f193dffb640 1 -- 172.21.15.76:0/2341722760 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1944042070 con 0x7f19480c7fd0 2024-04-25T19:15:21.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.543+0000 7f194e19f640 1 -- 172.21.15.76:0/2341722760 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19481549f0 con 0x7f19480c7fd0 2024-04-25T19:15:21.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.543+0000 7f193dffb640 1 -- 172.21.15.76:0/2341722760 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1944009510 con 0x7f19480c7fd0 2024-04-25T19:15:21.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.544+0000 7f194e19f640 1 -- 172.21.15.76:0/2341722760 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f194815c6b0 con 0x7f19480c7fd0 2024-04-25T19:15:21.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.545+0000 7f194e19f640 1 -- 172.21.15.76:0/2341722760 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1948050740 con 0x7f19480c7fd0 2024-04-25T19:15:21.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.545+0000 7f193dffb640 1 -- 172.21.15.76:0/2341722760 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1944007c50 con 0x7f19480c7fd0 2024-04-25T19:15:21.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.546+0000 7f193dffb640 1 --2- 172.21.15.76:0/2341722760 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f192c0744e0 0x7f192c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:21.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.546+0000 7f193dffb640 1 -- 172.21.15.76:0/2341722760 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f19440b5a60 con 0x7f19480c7fd0 2024-04-25T19:15:21.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.546+0000 7f194c99c640 1 --2- 172.21.15.76:0/2341722760 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f192c0744e0 0x7f192c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:21.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.546+0000 7f194c99c640 1 --2- 172.21.15.76:0/2341722760 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f192c0744e0 0x7f192c0769f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f19481f07e0 tx=0x7f1938008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:21.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.550+0000 7f193dffb640 1 -- 172.21.15.76:0/2341722760 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f194404b020 con 0x7f19480c7fd0 2024-04-25T19:15:21.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:21.763+0000 7f194e19f640 1 -- 172.21.15.76:0/2341722760 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f194815a120 con 0x7f19480c7fd0 2024-04-25T19:15:22.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.424+0000 7f193dffb640 1 -- 172.21.15.76:0/2341722760 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v6) ==== 105+0+0 (secure 0 0 0) 0x7f19481549f0 con 0x7f19480c7fd0 2024-04-25T19:15:22.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.427+0000 7f194e19f640 1 -- 172.21.15.76:0/2341722760 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f192c0744e0 msgr2=0x7f192c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:22.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.427+0000 7f194e19f640 1 --2- 172.21.15.76:0/2341722760 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f192c0744e0 0x7f192c0769f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f19481f07e0 tx=0x7f1938008040 comp rx=0 tx=0).stop 2024-04-25T19:15:22.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.427+0000 7f194e19f640 1 -- 172.21.15.76:0/2341722760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19480c7fd0 msgr2=0x7f19481e8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:22.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.427+0000 7f194e19f640 1 --2- 172.21.15.76:0/2341722760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19480c7fd0 0x7f19481e8670 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1944009f40 tx=0x7f1944004a40 comp rx=0 tx=0).stop 2024-04-25T19:15:22.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.427+0000 7f194e19f640 1 -- 172.21.15.76:0/2341722760 shutdown_connections 2024-04-25T19:15:22.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.427+0000 7f194e19f640 1 --2- 172.21.15.76:0/2341722760 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f192c0744e0 0x7f192c0769f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:22.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.427+0000 7f194e19f640 1 --2- 172.21.15.76:0/2341722760 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19480ceb50 0x7f194815c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:22.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.427+0000 7f194e19f640 1 --2- 172.21.15.76:0/2341722760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19480ce740 0x7f194814e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:22.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.427+0000 7f194e19f640 1 --2- 172.21.15.76:0/2341722760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19480c7fd0 0x7f19481e8670 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:22.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.427+0000 7f194e19f640 1 -- 172.21.15.76:0/2341722760 >> 172.21.15.76:0/2341722760 conn(0x7f19480684e0 msgr2=0x7f1948075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:22.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.427+0000 7f194e19f640 1 -- 172.21.15.76:0/2341722760 shutdown_connections 2024-04-25T19:15:22.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.427+0000 7f194e19f640 1 -- 172.21.15.76:0/2341722760 wait complete. 2024-04-25T19:15:22.441 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T19:15:22.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.553+0000 7f7264d9d640 1 Processor -- start 2024-04-25T19:15:22.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.553+0000 7f7264d9d640 1 -- start start 2024-04-25T19:15:22.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.554+0000 7f7264d9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72600c7fd0 0x7f7260156b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:22.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.554+0000 7f7264d9d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72600cef60 con 0x7f72600ceb50 2024-04-25T19:15:22.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.554+0000 7f7264d9d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72600cf1f0 con 0x7f72600ce740 2024-04-25T19:15:22.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.554+0000 7f7264d9d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f72600ce3b0 con 0x7f72600c7fd0 2024-04-25T19:15:22.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.554+0000 7f725ffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f72600ceb50 0x7f726015c970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35178/0 (socket says 172.21.15.76:35178) 2024-04-25T19:15:22.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.554+0000 7f725ffff640 1 -- 172.21.15.76:0/3981843983 learned_addr learned my addr 172.21.15.76:0/3981843983 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:22.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.555+0000 7f725f7fe640 1 --2- 172.21.15.76:0/3981843983 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72600c7fd0 0x7f7260156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:22.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.555+0000 7f725e7fc640 1 -- 172.21.15.76:0/3981843983 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 632970203 0 0) 0x7f72600cef60 con 0x7f72600ceb50 2024-04-25T19:15:22.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.555+0000 7f725e7fc640 1 -- 172.21.15.76:0/3981843983 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7248003520 con 0x7f72600ceb50 2024-04-25T19:15:22.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.555+0000 7f725e7fc640 1 -- 172.21.15.76:0/3981843983 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3748461188 0 0) 0x7f72600cf1f0 con 0x7f72600ce740 2024-04-25T19:15:22.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.555+0000 7f725e7fc640 1 -- 172.21.15.76:0/3981843983 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72600cef60 con 0x7f72600ce740 2024-04-25T19:15:22.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.555+0000 7f725e7fc640 1 -- 172.21.15.76:0/3981843983 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1630402768 0 0) 0x7f7248003520 con 0x7f72600ceb50 2024-04-25T19:15:22.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.555+0000 7f725e7fc640 1 -- 172.21.15.76:0/3981843983 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72600c7fd0 msgr2=0x7f7260156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:22.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.555+0000 7f725e7fc640 1 --2- 172.21.15.76:0/3981843983 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72600c7fd0 0x7f7260156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:22.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.555+0000 7f725e7fc640 1 -- 172.21.15.76:0/3981843983 >> v1:172.21.15.105:6789/0 conn(0x7f72600ce740 legacy=0x7f72600cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:22.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.555+0000 7f725e7fc640 1 -- 172.21.15.76:0/3981843983 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72600cd7a0 con 0x7f72600ceb50 2024-04-25T19:15:22.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.555+0000 7f725e7fc640 1 -- 172.21.15.76:0/3981843983 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f7258001970 con 0x7f72600ceb50 2024-04-25T19:15:22.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.555+0000 7f725e7fc640 1 -- 172.21.15.76:0/3981843983 >> v1:172.21.15.76:6789/0 conn(0x7f72600ceb50 legacy=0x7f726015c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:22.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.555+0000 7f725e7fc640 1 --2- 172.21.15.76:0/3981843983 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72600c7fd0 0x7f7248003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:22.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.555+0000 7f725e7fc640 1 --2- 172.21.15.76:0/3981843983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7248004120 0x7f7248004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:22.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.555+0000 7f725e7fc640 1 --2- 172.21.15.76:0/3981843983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7248004d40 0x7f7248005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:22.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.555+0000 7f725e7fc640 1 -- 172.21.15.76:0/3981843983 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f72600ce3b0 con 0x7f7248004d40 2024-04-25T19:15:22.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.555+0000 7f725e7fc640 1 -- 172.21.15.76:0/3981843983 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f724800bb80 con 0x7f7248004120 2024-04-25T19:15:22.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.555+0000 7f725e7fc640 1 -- 172.21.15.76:0/3981843983 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f724800bd80 con 0x7f72600c7fd0 2024-04-25T19:15:22.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.555+0000 7f725ffff640 1 --2- 172.21.15.76:0/3981843983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7248004d40 0x7f7248005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:22.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.555+0000 7f725f7fe640 1 --2- 172.21.15.76:0/3981843983 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72600c7fd0 0x7f7248003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:22.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.555+0000 7f725effd640 1 --2- 172.21.15.76:0/3981843983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7248004120 0x7f7248004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:22.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.555+0000 7f725ffff640 1 -- 172.21.15.76:0/3981843983 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72600c7fd0 msgr2=0x7f7248003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:22.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.555+0000 7f725ffff640 1 --2- 172.21.15.76:0/3981843983 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72600c7fd0 0x7f7248003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:22.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.555+0000 7f725ffff640 1 -- 172.21.15.76:0/3981843983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7248004120 msgr2=0x7f7248004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:22.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.555+0000 7f725ffff640 1 --2- 172.21.15.76:0/3981843983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7248004120 0x7f7248004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:22.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.555+0000 7f725ffff640 1 -- 172.21.15.76:0/3981843983 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7248003520 con 0x7f7248004d40 2024-04-25T19:15:22.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.556+0000 7f725ffff640 1 --2- 172.21.15.76:0/3981843983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7248004d40 0x7f7248005190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f726008fe50 tx=0x7f7248005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1bcd269805585f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:22.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.556+0000 7f725e7fc640 1 -- 172.21.15.76:0/3981843983 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725800a2b0 con 0x7f7248004d40 2024-04-25T19:15:22.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.556+0000 7f725e7fc640 1 -- 172.21.15.76:0/3981843983 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f725802f070 con 0x7f7248004d40 2024-04-25T19:15:22.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.556+0000 7f725e7fc640 1 -- 172.21.15.76:0/3981843983 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725802ba80 con 0x7f7248004d40 2024-04-25T19:15:22.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.557+0000 7f7264d9d640 1 -- 172.21.15.76:0/3981843983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7248004d40 msgr2=0x7f7248005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:22.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.557+0000 7f7264d9d640 1 --2- 172.21.15.76:0/3981843983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7248004d40 0x7f7248005190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f726008fe50 tx=0x7f7248005830 comp rx=0 tx=0).stop 2024-04-25T19:15:22.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.557+0000 7f725ffff640 1 -- 172.21.15.76:0/3981843983 reap_dead start 2024-04-25T19:15:22.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.557+0000 7f7264d9d640 1 -- 172.21.15.76:0/3981843983 shutdown_connections 2024-04-25T19:15:22.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.557+0000 7f7264d9d640 1 -- 172.21.15.76:0/3981843983 >> 172.21.15.76:0/3981843983 conn(0x7f72600684e0 msgr2=0x7f72600688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:22.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.557+0000 7f7264d9d640 1 -- 172.21.15.76:0/3981843983 shutdown_connections 2024-04-25T19:15:22.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.557+0000 7f7264d9d640 1 -- 172.21.15.76:0/3981843983 wait complete. 2024-04-25T19:15:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.559+0000 7f7264d9d640 1 Processor -- start 2024-04-25T19:15:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.559+0000 7f7264d9d640 1 -- start start 2024-04-25T19:15:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.560+0000 7f7264d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72601e9880 0x7f7260151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.560+0000 7f7264d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72601f6770 0x7f7260152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.560+0000 7f7264d9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72601e7e50 0x7f72601e8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.560+0000 7f7264d9d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f72600ce040 con 0x7f72601e9880 2024-04-25T19:15:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.560+0000 7f7264d9d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f72600cdd60 con 0x7f72601f6770 2024-04-25T19:15:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.560+0000 7f7264d9d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f72600cd0b0 con 0x7f72601e7e50 2024-04-25T19:15:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.560+0000 7f725f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72601e9880 0x7f7260151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.560+0000 7f725f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72601e9880 0x7f7260151500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58938/0 (socket says 172.21.15.76:58938) 2024-04-25T19:15:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.560+0000 7f725f7fe640 1 -- 172.21.15.76:0/844843411 learned_addr learned my addr 172.21.15.76:0/844843411 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.560+0000 7f725effd640 1 --2- 172.21.15.76:0/844843411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72601f6770 0x7f7260152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.560+0000 7f725ffff640 1 --2- 172.21.15.76:0/844843411 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72601e7e50 0x7f72601e8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.560+0000 7f725f7fe640 1 -- 172.21.15.76:0/844843411 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72601e7e50 msgr2=0x7f72601e8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:22.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.560+0000 7f725f7fe640 1 --2- 172.21.15.76:0/844843411 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72601e7e50 0x7f72601e8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:22.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.560+0000 7f725f7fe640 1 -- 172.21.15.76:0/844843411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72601f6770 msgr2=0x7f7260152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:22.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.560+0000 7f725f7fe640 1 --2- 172.21.15.76:0/844843411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72601f6770 0x7f7260152290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:22.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.560+0000 7f725f7fe640 1 -- 172.21.15.76:0/844843411 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f726015a770 con 0x7f72601e9880 2024-04-25T19:15:22.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.560+0000 7f725f7fe640 1 --2- 172.21.15.76:0/844843411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72601e9880 0x7f7260151500 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7250011910 tx=0x7f7250011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:22.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.561+0000 7f7247fff640 1 -- 172.21.15.76:0/844843411 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725000fa90 con 0x7f72601e9880 2024-04-25T19:15:22.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.561+0000 7f7247fff640 1 -- 172.21.15.76:0/844843411 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f725000fc30 con 0x7f72601e9880 2024-04-25T19:15:22.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.561+0000 7f7264d9d640 1 -- 172.21.15.76:0/844843411 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f726015ab70 con 0x7f72601e9880 2024-04-25T19:15:22.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.561+0000 7f7247fff640 1 -- 172.21.15.76:0/844843411 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250014440 con 0x7f72601e9880 2024-04-25T19:15:22.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.561+0000 7f7264d9d640 1 -- 172.21.15.76:0/844843411 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72601e8da0 con 0x7f72601e9880 2024-04-25T19:15:22.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.562+0000 7f7264d9d640 1 -- 172.21.15.76:0/844843411 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7260050740 con 0x7f72601e9880 2024-04-25T19:15:22.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.562+0000 7f7247fff640 1 -- 172.21.15.76:0/844843411 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7250012040 con 0x7f72601e9880 2024-04-25T19:15:22.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.563+0000 7f7247fff640 1 --2- 172.21.15.76:0/844843411 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f724c0744b0 0x7f724c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:22.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.563+0000 7f7247fff640 1 -- 172.21.15.76:0/844843411 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7250095750 con 0x7f72601e9880 2024-04-25T19:15:22.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.563+0000 7f725effd640 1 --2- 172.21.15.76:0/844843411 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f724c0744b0 0x7f724c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:22.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.563+0000 7f725effd640 1 --2- 172.21.15.76:0/844843411 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f724c0744b0 0x7f724c0769c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f726008ef90 tx=0x7f7254009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:22.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.567+0000 7f7247fff640 1 -- 172.21.15.76:0/844843411 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7250019030 con 0x7f72601e9880 2024-04-25T19:15:22.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:22.781+0000 7f7264d9d640 1 -- 172.21.15.76:0/844843411 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f72600cd4e0 con 0x7f72601e9880 2024-04-25T19:15:23.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.426+0000 7f7247fff640 1 -- 172.21.15.76:0/844843411 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7f7250060000 con 0x7f72601e9880 2024-04-25T19:15:23.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.428+0000 7f7264d9d640 1 -- 172.21.15.76:0/844843411 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f724c0744b0 msgr2=0x7f724c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:23.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.428+0000 7f7264d9d640 1 --2- 172.21.15.76:0/844843411 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f724c0744b0 0x7f724c0769c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f726008ef90 tx=0x7f7254009040 comp rx=0 tx=0).stop 2024-04-25T19:15:23.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.428+0000 7f7264d9d640 1 -- 172.21.15.76:0/844843411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72601e9880 msgr2=0x7f7260151500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:23.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.428+0000 7f7264d9d640 1 --2- 172.21.15.76:0/844843411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72601e9880 0x7f7260151500 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7250011910 tx=0x7f7250011dd0 comp rx=0 tx=0).stop 2024-04-25T19:15:23.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.428+0000 7f7264d9d640 1 -- 172.21.15.76:0/844843411 shutdown_connections 2024-04-25T19:15:23.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.428+0000 7f7264d9d640 1 --2- 172.21.15.76:0/844843411 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f724c0744b0 0x7f724c0769c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:23.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.428+0000 7f7264d9d640 1 --2- 172.21.15.76:0/844843411 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f72601e7e50 0x7f72601e8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:23.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.428+0000 7f7264d9d640 1 --2- 172.21.15.76:0/844843411 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72601f6770 0x7f7260152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:23.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.428+0000 7f7264d9d640 1 --2- 172.21.15.76:0/844843411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72601e9880 0x7f7260151500 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:23.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.428+0000 7f7264d9d640 1 -- 172.21.15.76:0/844843411 >> 172.21.15.76:0/844843411 conn(0x7f72600684e0 msgr2=0x7f72600c77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:23.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.428+0000 7f7264d9d640 1 -- 172.21.15.76:0/844843411 shutdown_connections 2024-04-25T19:15:23.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.428+0000 7f7264d9d640 1 -- 172.21.15.76:0/844843411 wait complete. 2024-04-25T19:15:23.442 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T19:15:23.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.550+0000 7f686de27640 1 Processor -- start 2024-04-25T19:15:23.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.550+0000 7f686de27640 1 -- start start 2024-04-25T19:15:23.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.552+0000 7f686de27640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f686806de20 0x7f686806d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:23.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.552+0000 7f686de27640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f686806e310 con 0x7f68680cc9e0 2024-04-25T19:15:23.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.552+0000 7f686de27640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f686815ccc0 con 0x7f6868156f10 2024-04-25T19:15:23.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.552+0000 7f686de27640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f686815d2f0 con 0x7f686806de20 2024-04-25T19:15:23.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.552+0000 7f6866ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f68680cc9e0 0x7f68680cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35182/0 (socket says 172.21.15.76:35182) 2024-04-25T19:15:23.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.552+0000 7f6866ffd640 1 -- 172.21.15.76:0/1567188030 learned_addr learned my addr 172.21.15.76:0/1567188030 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:23.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.552+0000 7f68677fe640 1 --2- 172.21.15.76:0/1567188030 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f686806de20 0x7f686806d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:23.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.552+0000 7f68667fc640 1 -- 172.21.15.76:0/1567188030 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2130604446 0 0) 0x7f686806e310 con 0x7f68680cc9e0 2024-04-25T19:15:23.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.552+0000 7f68667fc640 1 -- 172.21.15.76:0/1567188030 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6850003520 con 0x7f68680cc9e0 2024-04-25T19:15:23.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.552+0000 7f68667fc640 1 -- 172.21.15.76:0/1567188030 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2870835750 0 0) 0x7f6850003520 con 0x7f68680cc9e0 2024-04-25T19:15:23.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.552+0000 7f68667fc640 1 -- 172.21.15.76:0/1567188030 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f686806de20 msgr2=0x7f686806d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:23.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.552+0000 7f68667fc640 1 --2- 172.21.15.76:0/1567188030 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f686806de20 0x7f686806d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:23.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.552+0000 7f68667fc640 1 -- 172.21.15.76:0/1567188030 >> v1:172.21.15.105:6789/0 conn(0x7f6868156f10 legacy=0x7f6868167640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:23.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.552+0000 7f68667fc640 1 -- 172.21.15.76:0/1567188030 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68680cb6a0 con 0x7f68680cc9e0 2024-04-25T19:15:23.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.552+0000 7f68667fc640 1 -- 172.21.15.76:0/1567188030 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f685c001670 con 0x7f68680cc9e0 2024-04-25T19:15:23.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.552+0000 7f68667fc640 1 -- 172.21.15.76:0/1567188030 >> v1:172.21.15.76:6789/0 conn(0x7f68680cc9e0 legacy=0x7f68680cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:23.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.552+0000 7f68667fc640 1 --2- 172.21.15.76:0/1567188030 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f686806de20 0x7f6850003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:23.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.552+0000 7f68667fc640 1 --2- 172.21.15.76:0/1567188030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6850003f80 0x7f6850004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:23.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.552+0000 7f68667fc640 1 --2- 172.21.15.76:0/1567188030 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6850004d40 0x7f6850005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:23.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.552+0000 7f68667fc640 1 -- 172.21.15.76:0/1567188030 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f686815d2f0 con 0x7f6850003f80 2024-04-25T19:15:23.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.552+0000 7f68667fc640 1 -- 172.21.15.76:0/1567188030 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f685000bc00 con 0x7f6850004d40 2024-04-25T19:15:23.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.552+0000 7f68667fc640 1 -- 172.21.15.76:0/1567188030 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f685000be00 con 0x7f686806de20 2024-04-25T19:15:23.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.553+0000 7f6866ffd640 1 --2- 172.21.15.76:0/1567188030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6850003f80 0x7f6850004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:23.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.553+0000 7f68677fe640 1 --2- 172.21.15.76:0/1567188030 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f686806de20 0x7f6850003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:23.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.553+0000 7f6867fff640 1 --2- 172.21.15.76:0/1567188030 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6850004d40 0x7f6850005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:23.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.553+0000 7f6866ffd640 1 -- 172.21.15.76:0/1567188030 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f686806de20 msgr2=0x7f6850003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:23.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.553+0000 7f6866ffd640 1 --2- 172.21.15.76:0/1567188030 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f686806de20 0x7f6850003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:23.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.553+0000 7f6866ffd640 1 -- 172.21.15.76:0/1567188030 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6850004d40 msgr2=0x7f6850005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:23.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.553+0000 7f6866ffd640 1 --2- 172.21.15.76:0/1567188030 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6850004d40 0x7f6850005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:23.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.553+0000 7f6866ffd640 1 -- 172.21.15.76:0/1567188030 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6850003520 con 0x7f6850003f80 2024-04-25T19:15:23.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.553+0000 7f6866ffd640 1 --2- 172.21.15.76:0/1567188030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6850003f80 0x7f6850004800 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f685c00c240 tx=0x7f6850005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf8bd72b0d8a98cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:23.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.554+0000 7f68667fc640 1 -- 172.21.15.76:0/1567188030 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c007cf0 con 0x7f6850003f80 2024-04-25T19:15:23.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.554+0000 7f68667fc640 1 -- 172.21.15.76:0/1567188030 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f685c0058d0 con 0x7f6850003f80 2024-04-25T19:15:23.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.554+0000 7f68667fc640 1 -- 172.21.15.76:0/1567188030 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c005ca0 con 0x7f6850003f80 2024-04-25T19:15:23.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.554+0000 7f686de27640 1 -- 172.21.15.76:0/1567188030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6850003f80 msgr2=0x7f6850004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:23.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.554+0000 7f686de27640 1 --2- 172.21.15.76:0/1567188030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6850003f80 0x7f6850004800 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f685c00c240 tx=0x7f6850005830 comp rx=0 tx=0).stop 2024-04-25T19:15:23.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.554+0000 7f6867fff640 1 -- 172.21.15.76:0/1567188030 reap_dead start 2024-04-25T19:15:23.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.554+0000 7f686de27640 1 -- 172.21.15.76:0/1567188030 shutdown_connections 2024-04-25T19:15:23.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.554+0000 7f686de27640 1 -- 172.21.15.76:0/1567188030 >> 172.21.15.76:0/1567188030 conn(0x7f68680684e0 msgr2=0x7f68680688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:23.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.554+0000 7f686de27640 1 -- 172.21.15.76:0/1567188030 shutdown_connections 2024-04-25T19:15:23.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.554+0000 7f686de27640 1 -- 172.21.15.76:0/1567188030 wait complete. 2024-04-25T19:15:23.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.556+0000 7f686de27640 1 Processor -- start 2024-04-25T19:15:23.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.556+0000 7f686de27640 1 -- start start 2024-04-25T19:15:23.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.557+0000 7f686de27640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6850003f80 0x7f686815f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:23.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.557+0000 7f686de27640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6868151050 0x7f686814d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:23.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.557+0000 7f686de27640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6868151460 0x7f686814de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:23.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.557+0000 7f686de27640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6868157320 con 0x7f6850003f80 2024-04-25T19:15:23.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.557+0000 7f686de27640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f686815cf80 con 0x7f6868151050 2024-04-25T19:15:23.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.557+0000 7f686de27640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f68680cbda0 con 0x7f6868151460 2024-04-25T19:15:23.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.558+0000 7f68677fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6850003f80 0x7f686815f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:23.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.558+0000 7f68677fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6850003f80 0x7f686815f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58952/0 (socket says 172.21.15.76:58952) 2024-04-25T19:15:23.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.558+0000 7f68677fe640 1 -- 172.21.15.76:0/1701443086 learned_addr learned my addr 172.21.15.76:0/1701443086 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:23.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.558+0000 7f6866ffd640 1 --2- 172.21.15.76:0/1701443086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6868151050 0x7f686814d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:23.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.558+0000 7f68677fe640 1 -- 172.21.15.76:0/1701443086 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6868151460 msgr2=0x7f686814de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:23.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.558+0000 7f68677fe640 1 --2- 172.21.15.76:0/1701443086 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6868151460 0x7f686814de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:23.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.558+0000 7f68677fe640 1 -- 172.21.15.76:0/1701443086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6868151050 msgr2=0x7f686814d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:23.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.558+0000 7f68677fe640 1 --2- 172.21.15.76:0/1701443086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6868151050 0x7f686814d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:23.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.558+0000 7f68677fe640 1 -- 172.21.15.76:0/1701443086 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68680cb000 con 0x7f6850003f80 2024-04-25T19:15:23.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.558+0000 7f68677fe640 1 --2- 172.21.15.76:0/1701443086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6850003f80 0x7f686815f380 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6858011920 tx=0x7f6858011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:23.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.558+0000 7f684ffff640 1 -- 172.21.15.76:0/1701443086 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685800f9d0 con 0x7f6850003f80 2024-04-25T19:15:23.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.558+0000 7f684ffff640 1 -- 172.21.15.76:0/1701443086 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f685800fb70 con 0x7f6850003f80 2024-04-25T19:15:23.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.558+0000 7f686de27640 1 -- 172.21.15.76:0/1701443086 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f686815a4e0 con 0x7f6850003f80 2024-04-25T19:15:23.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.558+0000 7f684ffff640 1 -- 172.21.15.76:0/1701443086 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6858014440 con 0x7f6850003f80 2024-04-25T19:15:23.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.558+0000 7f686de27640 1 -- 172.21.15.76:0/1701443086 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f686814ee90 con 0x7f6850003f80 2024-04-25T19:15:23.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.559+0000 7f686de27640 1 -- 172.21.15.76:0/1701443086 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f686806a390 con 0x7f6850003f80 2024-04-25T19:15:23.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.560+0000 7f684ffff640 1 -- 172.21.15.76:0/1701443086 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6858012040 con 0x7f6850003f80 2024-04-25T19:15:23.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.560+0000 7f684ffff640 1 --2- 172.21.15.76:0/1701443086 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f68540744e0 0x7f68540769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:23.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.560+0000 7f684ffff640 1 -- 172.21.15.76:0/1701443086 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f68580956e0 con 0x7f6850003f80 2024-04-25T19:15:23.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.561+0000 7f6866ffd640 1 --2- 172.21.15.76:0/1701443086 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f68540744e0 0x7f68540769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:23.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.561+0000 7f6866ffd640 1 --2- 172.21.15.76:0/1701443086 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f68540744e0 0x7f68540769f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f685c033000 tx=0x7f685c009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:23.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.565+0000 7f684ffff640 1 -- 172.21.15.76:0/1701443086 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6858019030 con 0x7f6850003f80 2024-04-25T19:15:23.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:23.780+0000 7f686de27640 1 -- 172.21.15.76:0/1701443086 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f68680cc0b0 con 0x7f6850003f80 2024-04-25T19:15:24.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.430+0000 7f684ffff640 1 -- 172.21.15.76:0/1701443086 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v8) ==== 115+0+0 (secure 0 0 0) 0x7f685805ff90 con 0x7f6850003f80 2024-04-25T19:15:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.433+0000 7f686de27640 1 -- 172.21.15.76:0/1701443086 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f68540744e0 msgr2=0x7f68540769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.433+0000 7f686de27640 1 --2- 172.21.15.76:0/1701443086 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f68540744e0 0x7f68540769f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f685c033000 tx=0x7f685c009030 comp rx=0 tx=0).stop 2024-04-25T19:15:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.433+0000 7f686de27640 1 -- 172.21.15.76:0/1701443086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6850003f80 msgr2=0x7f686815f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.433+0000 7f686de27640 1 --2- 172.21.15.76:0/1701443086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6850003f80 0x7f686815f380 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6858011920 tx=0x7f6858011de0 comp rx=0 tx=0).stop 2024-04-25T19:15:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.433+0000 7f686de27640 1 -- 172.21.15.76:0/1701443086 shutdown_connections 2024-04-25T19:15:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.433+0000 7f686de27640 1 --2- 172.21.15.76:0/1701443086 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f68540744e0 0x7f68540769f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.433+0000 7f686de27640 1 --2- 172.21.15.76:0/1701443086 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6868151460 0x7f686814de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.433+0000 7f686de27640 1 --2- 172.21.15.76:0/1701443086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6868151050 0x7f686814d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.433+0000 7f686de27640 1 --2- 172.21.15.76:0/1701443086 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6850003f80 0x7f686815f380 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.433+0000 7f686de27640 1 -- 172.21.15.76:0/1701443086 >> 172.21.15.76:0/1701443086 conn(0x7f68680684e0 msgr2=0x7f686806e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.433+0000 7f686de27640 1 -- 172.21.15.76:0/1701443086 shutdown_connections 2024-04-25T19:15:24.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.433+0000 7f686de27640 1 -- 172.21.15.76:0/1701443086 wait complete. 2024-04-25T19:15:24.447 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T19:15:24.447 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:15:24.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.552+0000 7f7ee009f640 1 Processor -- start 2024-04-25T19:15:24.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.553+0000 7f7ee009f640 1 -- start start 2024-04-25T19:15:24.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.554+0000 7f7ee009f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7ed80c96c0 0x7f7ed815cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:24.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.554+0000 7f7ee009f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7ed8074650 0x7f7ed8160a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:24.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.554+0000 7f7ee009f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7ed806c5c0 0x7f7ed806c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:24.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.554+0000 7f7ee009f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f7ed815d1c0 con 0x7f7ed8074650 2024-04-25T19:15:24.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.554+0000 7f7ee009f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f7ed815d4a0 con 0x7f7ed80c96c0 2024-04-25T19:15:24.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.554+0000 7f7ee009f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed815d780 con 0x7f7ed806c5c0 2024-04-25T19:15:24.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.554+0000 7f7edd63a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7ed8074650 0x7f7ed8160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:24.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.554+0000 7f7edd63a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7ed8074650 0x7f7ed8160a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58962/0 (socket says 172.21.15.76:58962) 2024-04-25T19:15:24.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.554+0000 7f7edd63a640 1 -- 172.21.15.76:0/1498737038 learned_addr learned my addr 172.21.15.76:0/1498737038 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:24.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.554+0000 7f7edde3b640 1 --2- 172.21.15.76:0/1498737038 >> v2:172.21.15.105:3300/0 conn(0x7f7ed80c96c0 0x7f7ed815cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:24.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.554+0000 7f7ede63c640 1 --2- 172.21.15.76:0/1498737038 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7ed806c5c0 0x7f7ed806c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:24.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.554+0000 7f7edd63a640 1 -- 172.21.15.76:0/1498737038 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7ed806c5c0 msgr2=0x7f7ed806c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:24.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.554+0000 7f7edd63a640 1 --2- 172.21.15.76:0/1498737038 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7ed806c5c0 0x7f7ed806c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:24.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.555+0000 7f7edd63a640 1 -- 172.21.15.76:0/1498737038 >> v2:172.21.15.105:3300/0 conn(0x7f7ed80c96c0 msgr2=0x7f7ed815cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:24.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.555+0000 7f7edd63a640 1 --2- 172.21.15.76:0/1498737038 >> v2:172.21.15.105:3300/0 conn(0x7f7ed80c96c0 0x7f7ed815cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:24.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.555+0000 7f7edd63a640 1 -- 172.21.15.76:0/1498737038 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed8160210 con 0x7f7ed8074650 2024-04-25T19:15:24.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.555+0000 7f7edd63a640 1 --2- 172.21.15.76:0/1498737038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ed8074650 0x7f7ed8160a10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7ed806d5e0 tx=0x7f7ec802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ca65b2288d1cb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:24.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.555+0000 7f7edce39640 1 -- 172.21.15.76:0/1498737038 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8037070 con 0x7f7ed8074650 2024-04-25T19:15:24.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.555+0000 7f7edce39640 1 -- 172.21.15.76:0/1498737038 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec8032440 con 0x7f7ed8074650 2024-04-25T19:15:24.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.555+0000 7f7edce39640 1 -- 172.21.15.76:0/1498737038 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8032810 con 0x7f7ed8074650 2024-04-25T19:15:24.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.556+0000 7f7ee009f640 1 -- 172.21.15.76:0/1498737038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ed8074650 msgr2=0x7f7ed8160a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:24.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.556+0000 7f7ee009f640 1 --2- 172.21.15.76:0/1498737038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ed8074650 0x7f7ed8160a10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7ed806d5e0 tx=0x7f7ec802a270 comp rx=0 tx=0).stop 2024-04-25T19:15:24.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.556+0000 7f7ee009f640 1 -- 172.21.15.76:0/1498737038 shutdown_connections 2024-04-25T19:15:24.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.556+0000 7f7ee009f640 1 --2- 172.21.15.76:0/1498737038 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7ed806c5c0 0x7f7ed806c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:24.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.556+0000 7f7ee009f640 1 --2- 172.21.15.76:0/1498737038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ed8074650 0x7f7ed8160a10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:24.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.556+0000 7f7ee009f640 1 --2- 172.21.15.76:0/1498737038 >> v2:172.21.15.105:3300/0 conn(0x7f7ed80c96c0 0x7f7ed815cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:24.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.556+0000 7f7ee009f640 1 -- 172.21.15.76:0/1498737038 >> 172.21.15.76:0/1498737038 conn(0x7f7ed80cecc0 msgr2=0x7f7ed80cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:24.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.556+0000 7f7ee009f640 1 -- 172.21.15.76:0/1498737038 shutdown_connections 2024-04-25T19:15:24.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.556+0000 7f7ee009f640 1 -- 172.21.15.76:0/1498737038 wait complete. 2024-04-25T19:15:24.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.558+0000 7f7ee009f640 1 Processor -- start 2024-04-25T19:15:24.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.558+0000 7f7ee009f640 1 -- start start 2024-04-25T19:15:24.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.559+0000 7f7ee009f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ed806c5c0 0x7f7ed81548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:24.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.559+0000 7f7ee009f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7ed8074650 0x7f7ed81f2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:24.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.559+0000 7f7ee009f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ed80c96c0 0x7f7ed814f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:24.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.559+0000 7f7ee009f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed80ca060 con 0x7f7ed80c96c0 2024-04-25T19:15:24.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.559+0000 7f7ee009f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed80c7330 con 0x7f7ed806c5c0 2024-04-25T19:15:24.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.559+0000 7f7ee009f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed815da60 con 0x7f7ed8074650 2024-04-25T19:15:24.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.559+0000 7f7ede63c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ed80c96c0 0x7f7ed814f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:24.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.559+0000 7f7ede63c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ed80c96c0 0x7f7ed814f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58966/0 (socket says 172.21.15.76:58966) 2024-04-25T19:15:24.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.559+0000 7f7ede63c640 1 -- 172.21.15.76:0/4187566979 learned_addr learned my addr 172.21.15.76:0/4187566979 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:24.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.559+0000 7f7edd63a640 1 --2- 172.21.15.76:0/4187566979 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7ed8074650 0x7f7ed81f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:24.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.559+0000 7f7edde3b640 1 --2- 172.21.15.76:0/4187566979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ed806c5c0 0x7f7ed81548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:24.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.559+0000 7f7ede63c640 1 -- 172.21.15.76:0/4187566979 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7ed8074650 msgr2=0x7f7ed81f2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:24.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.559+0000 7f7ede63c640 1 --2- 172.21.15.76:0/4187566979 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7ed8074650 0x7f7ed81f2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:24.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.559+0000 7f7ede63c640 1 -- 172.21.15.76:0/4187566979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ed806c5c0 msgr2=0x7f7ed81548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:24.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.559+0000 7f7ede63c640 1 --2- 172.21.15.76:0/4187566979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ed806c5c0 0x7f7ed81548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:24.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.559+0000 7f7ede63c640 1 -- 172.21.15.76:0/4187566979 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed815e9f0 con 0x7f7ed80c96c0 2024-04-25T19:15:24.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.560+0000 7f7ede63c640 1 --2- 172.21.15.76:0/4187566979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ed80c96c0 0x7f7ed814f500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000ef00 tx=0x7f7ed000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:24.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.560+0000 7f7ece7fc640 1 -- 172.21.15.76:0/4187566979 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0018020 con 0x7f7ed80c96c0 2024-04-25T19:15:24.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.560+0000 7f7ece7fc640 1 -- 172.21.15.76:0/4187566979 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed000bd30 con 0x7f7ed80c96c0 2024-04-25T19:15:24.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.560+0000 7f7ece7fc640 1 -- 172.21.15.76:0/4187566979 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0014640 con 0x7f7ed80c96c0 2024-04-25T19:15:24.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.560+0000 7f7ee009f640 1 -- 172.21.15.76:0/4187566979 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed815f750 con 0x7f7ed80c96c0 2024-04-25T19:15:24.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.560+0000 7f7ee009f640 1 -- 172.21.15.76:0/4187566979 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed8155b50 con 0x7f7ed80c96c0 2024-04-25T19:15:24.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.561+0000 7f7ee009f640 1 -- 172.21.15.76:0/4187566979 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed806a390 con 0x7f7ed80c96c0 2024-04-25T19:15:24.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.561+0000 7f7ece7fc640 1 -- 172.21.15.76:0/4187566979 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7ed0005d10 con 0x7f7ed80c96c0 2024-04-25T19:15:24.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.562+0000 7f7ece7fc640 1 --2- 172.21.15.76:0/4187566979 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f7ebc0744e0 0x7f7ebc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:24.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.562+0000 7f7ece7fc640 1 -- 172.21.15.76:0/4187566979 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7ed00958a0 con 0x7f7ed80c96c0 2024-04-25T19:15:24.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.562+0000 7f7edde3b640 1 --2- 172.21.15.76:0/4187566979 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f7ebc0744e0 0x7f7ebc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:24.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.563+0000 7f7edde3b640 1 --2- 172.21.15.76:0/4187566979 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f7ebc0744e0 0x7f7ebc0769f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4009aa0 tx=0x7f7ed4009190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:24.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.566+0000 7f7ece7fc640 1 -- 172.21.15.76:0/4187566979 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ed0022030 con 0x7f7ed80c96c0 2024-04-25T19:15:24.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.777+0000 7f7ee009f640 1 -- 172.21.15.76:0/4187566979 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ed8074430 con 0x7f7ed80c96c0 2024-04-25T19:15:24.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.778+0000 7f7ece7fc640 1 -- 172.21.15.76:0/4187566979 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+6849 (secure 0 0 0) 0x7f7ed815e9f0 con 0x7f7ed80c96c0 2024-04-25T19:15:24.780 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:24.780 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/3003966182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3003966182},{"type":"v1","addr":"172.21.15.76:6836","nonce":3003966182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6836/1809877606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1809877606},{"type":"v1","addr":"172.21.15.105:6836","nonce":1809877606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4238,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6838/2724142282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2724142282},{"type":"v1","addr":"172.21.15.105:6838","nonce":2724142282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6839/1798232582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6837","nonce":1798232582},{"type":"v1","addr":"172.21.15.105:6839","nonce":1798232582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T19:15:18.769449+0000","modified":"2024-04-25T19:15:24.437727+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1],"up":{"mds_0":4284,"mds_1":4278},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":9,"state":"up:creating","state_seq":1,"addr":"172.21.15.76:6833/2101494685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2101494685},{"type":"v1","addr":"172.21.15.76:6833","nonce":2101494685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4284":{"gid":4284,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6837/4226277412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":4226277412},{"type":"v1","addr":"172.21.15.76:6837","nonce":4226277412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-04-25T19:15:24.780 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 9 2024-04-25T19:15:24.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.780+0000 7f7ee009f640 1 -- 172.21.15.76:0/4187566979 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f7ebc0744e0 msgr2=0x7f7ebc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:24.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.780+0000 7f7ee009f640 1 --2- 172.21.15.76:0/4187566979 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f7ebc0744e0 0x7f7ebc0769f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4009aa0 tx=0x7f7ed4009190 comp rx=0 tx=0).stop 2024-04-25T19:15:24.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.780+0000 7f7ee009f640 1 -- 172.21.15.76:0/4187566979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ed80c96c0 msgr2=0x7f7ed814f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:24.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.780+0000 7f7ee009f640 1 --2- 172.21.15.76:0/4187566979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ed80c96c0 0x7f7ed814f500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000ef00 tx=0x7f7ed000b480 comp rx=0 tx=0).stop 2024-04-25T19:15:24.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.780+0000 7f7ee009f640 1 -- 172.21.15.76:0/4187566979 shutdown_connections 2024-04-25T19:15:24.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.780+0000 7f7ee009f640 1 --2- 172.21.15.76:0/4187566979 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f7ebc0744e0 0x7f7ebc0769f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:24.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.780+0000 7f7ee009f640 1 --2- 172.21.15.76:0/4187566979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ed80c96c0 0x7f7ed814f500 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:24.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.780+0000 7f7ee009f640 1 --2- 172.21.15.76:0/4187566979 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7ed8074650 0x7f7ed81f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:24.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.780+0000 7f7ee009f640 1 --2- 172.21.15.76:0/4187566979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7ed806c5c0 0x7f7ed81548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:24.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.780+0000 7f7ee009f640 1 -- 172.21.15.76:0/4187566979 >> 172.21.15.76:0/4187566979 conn(0x7f7ed80cecc0 msgr2=0x7f7ed81529d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:24.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.780+0000 7f7ee009f640 1 -- 172.21.15.76:0/4187566979 shutdown_connections 2024-04-25T19:15:24.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.780+0000 7f7ee009f640 1 -- 172.21.15.76:0/4187566979 wait complete. 2024-04-25T19:15:24.793 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:15:24.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.897+0000 7fb9388e7640 1 Processor -- start 2024-04-25T19:15:24.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.897+0000 7fb9388e7640 1 -- start start 2024-04-25T19:15:24.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.899+0000 7fb9388e7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb93406de20 0x7fb93406d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:24.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.899+0000 7fb9388e7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9340cc9e0 0x7fb934156f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:24.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.899+0000 7fb9388e7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb93415d780 0x7fb9340cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:24.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.899+0000 7fb9388e7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb934157450 con 0x7fb93406de20 2024-04-25T19:15:24.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.899+0000 7fb9388e7640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fb93415d2f0 con 0x7fb93415d780 2024-04-25T19:15:24.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.899+0000 7fb9388e7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb93406e310 con 0x7fb9340cc9e0 2024-04-25T19:15:24.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.899+0000 7fb93259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb93406de20 0x7fb93406d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:24.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.899+0000 7fb93259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb93406de20 0x7fb93406d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58970/0 (socket says 172.21.15.76:58970) 2024-04-25T19:15:24.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.899+0000 7fb93259c640 1 -- 172.21.15.76:0/1659915209 learned_addr learned my addr 172.21.15.76:0/1659915209 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:24.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.899+0000 7fb931d9b640 1 --2- 172.21.15.76:0/1659915209 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9340cc9e0 0x7fb934156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:24.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.899+0000 7fb932d9d640 1 --2- 172.21.15.76:0/1659915209 >> v2:172.21.15.105:3300/0 conn(0x7fb93415d780 0x7fb9340cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:24.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.899+0000 7fb93259c640 1 -- 172.21.15.76:0/1659915209 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9340cc9e0 msgr2=0x7fb934156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:24.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.899+0000 7fb93259c640 1 --2- 172.21.15.76:0/1659915209 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9340cc9e0 0x7fb934156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:24.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.899+0000 7fb93259c640 1 -- 172.21.15.76:0/1659915209 >> v2:172.21.15.105:3300/0 conn(0x7fb93415d780 msgr2=0x7fb9340cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:24.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.899+0000 7fb93259c640 1 --2- 172.21.15.76:0/1659915209 >> v2:172.21.15.105:3300/0 conn(0x7fb93415d780 0x7fb9340cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:24.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.899+0000 7fb93259c640 1 -- 172.21.15.76:0/1659915209 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93415ccc0 con 0x7fb93406de20 2024-04-25T19:15:24.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.900+0000 7fb93259c640 1 --2- 172.21.15.76:0/1659915209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb93406de20 0x7fb93406d6d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb934161350 tx=0x7fb92402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b103a8e20ce2a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:24.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.900+0000 7fb93159a640 1 -- 172.21.15.76:0/1659915209 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924037070 con 0x7fb93406de20 2024-04-25T19:15:24.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.900+0000 7fb93159a640 1 -- 172.21.15.76:0/1659915209 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb924032440 con 0x7fb93406de20 2024-04-25T19:15:24.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.900+0000 7fb93159a640 1 -- 172.21.15.76:0/1659915209 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924032810 con 0x7fb93406de20 2024-04-25T19:15:24.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.901+0000 7fb9388e7640 1 -- 172.21.15.76:0/1659915209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb93406de20 msgr2=0x7fb93406d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:24.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.901+0000 7fb9388e7640 1 --2- 172.21.15.76:0/1659915209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb93406de20 0x7fb93406d6d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb934161350 tx=0x7fb92402a140 comp rx=0 tx=0).stop 2024-04-25T19:15:24.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.901+0000 7fb9388e7640 1 -- 172.21.15.76:0/1659915209 shutdown_connections 2024-04-25T19:15:24.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.901+0000 7fb9388e7640 1 --2- 172.21.15.76:0/1659915209 >> v2:172.21.15.105:3300/0 conn(0x7fb93415d780 0x7fb9340cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:24.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.901+0000 7fb9388e7640 1 --2- 172.21.15.76:0/1659915209 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb9340cc9e0 0x7fb934156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:24.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.901+0000 7fb9388e7640 1 --2- 172.21.15.76:0/1659915209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb93406de20 0x7fb93406d6d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:24.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.901+0000 7fb9388e7640 1 -- 172.21.15.76:0/1659915209 >> 172.21.15.76:0/1659915209 conn(0x7fb9340684e0 msgr2=0x7fb9340688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:24.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.901+0000 7fb9388e7640 1 -- 172.21.15.76:0/1659915209 shutdown_connections 2024-04-25T19:15:24.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.901+0000 7fb9388e7640 1 -- 172.21.15.76:0/1659915209 wait complete. 2024-04-25T19:15:24.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.902+0000 7fb9388e7640 1 Processor -- start 2024-04-25T19:15:24.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.903+0000 7fb9388e7640 1 -- start start 2024-04-25T19:15:24.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.904+0000 7fb9388e7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb93406de20 0x7fb9341f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:24.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.904+0000 7fb9388e7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9340cc9e0 0x7fb934150660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:24.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.904+0000 7fb9388e7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb93415d780 0x7fb934152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:24.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.904+0000 7fb9388e7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb93415cf80 con 0x7fb9340cc9e0 2024-04-25T19:15:24.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.904+0000 7fb9388e7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb9340cb980 con 0x7fb93406de20 2024-04-25T19:15:24.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.904+0000 7fb9388e7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb9340cbda0 con 0x7fb93415d780 2024-04-25T19:15:24.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.904+0000 7fb931d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9340cc9e0 0x7fb934150660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:24.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.904+0000 7fb931d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9340cc9e0 0x7fb934150660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58982/0 (socket says 172.21.15.76:58982) 2024-04-25T19:15:24.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.904+0000 7fb931d9b640 1 -- 172.21.15.76:0/98923735 learned_addr learned my addr 172.21.15.76:0/98923735 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:24.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.904+0000 7fb93259c640 1 --2- 172.21.15.76:0/98923735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb93406de20 0x7fb9341f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:24.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.904+0000 7fb932d9d640 1 --2- 172.21.15.76:0/98923735 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb93415d780 0x7fb934152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:24.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.904+0000 7fb931d9b640 1 -- 172.21.15.76:0/98923735 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb93415d780 msgr2=0x7fb934152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:24.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.904+0000 7fb931d9b640 1 --2- 172.21.15.76:0/98923735 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb93415d780 0x7fb934152f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:24.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.904+0000 7fb931d9b640 1 -- 172.21.15.76:0/98923735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb93406de20 msgr2=0x7fb9341f2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:24.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.904+0000 7fb931d9b640 1 --2- 172.21.15.76:0/98923735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb93406de20 0x7fb9341f2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:24.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.904+0000 7fb931d9b640 1 -- 172.21.15.76:0/98923735 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb934150ba0 con 0x7fb9340cc9e0 2024-04-25T19:15:24.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.904+0000 7fb932d9d640 1 --2- 172.21.15.76:0/98923735 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb93415d780 0x7fb934152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:24.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.904+0000 7fb931d9b640 1 --2- 172.21.15.76:0/98923735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9340cc9e0 0x7fb934150660 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb92800d830 tx=0x7fb92800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:24.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.904+0000 7fb922ffd640 1 -- 172.21.15.76:0/98923735 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb928018030 con 0x7fb9340cc9e0 2024-04-25T19:15:24.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.904+0000 7fb922ffd640 1 -- 172.21.15.76:0/98923735 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb92800be10 con 0x7fb9340cc9e0 2024-04-25T19:15:24.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.905+0000 7fb9388e7640 1 -- 172.21.15.76:0/98923735 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb93414d760 con 0x7fb9340cc9e0 2024-04-25T19:15:24.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.905+0000 7fb922ffd640 1 -- 172.21.15.76:0/98923735 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb928009850 con 0x7fb9340cc9e0 2024-04-25T19:15:24.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.905+0000 7fb9388e7640 1 -- 172.21.15.76:0/98923735 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb93414c730 con 0x7fb9340cc9e0 2024-04-25T19:15:24.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.906+0000 7fb9388e7640 1 -- 172.21.15.76:0/98923735 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb93406a390 con 0x7fb9340cc9e0 2024-04-25T19:15:24.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.906+0000 7fb922ffd640 1 -- 172.21.15.76:0/98923735 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb92801b070 con 0x7fb9340cc9e0 2024-04-25T19:15:24.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.907+0000 7fb922ffd640 1 --2- 172.21.15.76:0/98923735 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb9140744e0 0x7fb9140769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:24.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.907+0000 7fb922ffd640 1 -- 172.21.15.76:0/98923735 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb928095200 con 0x7fb9340cc9e0 2024-04-25T19:15:24.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.907+0000 7fb93259c640 1 --2- 172.21.15.76:0/98923735 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb9140744e0 0x7fb9140769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:24.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.907+0000 7fb93259c640 1 --2- 172.21.15.76:0/98923735 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb9140744e0 0x7fb9140769f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb924034000 tx=0x7fb92402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:24.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:24.911+0000 7fb922ffd640 1 -- 172.21.15.76:0/98923735 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb92805fab0 con 0x7fb9340cc9e0 2024-04-25T19:15:25.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.087+0000 7fb9388e7640 1 -- 172.21.15.76:0/98923735 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9340cc0b0 con 0x7fb9340cc9e0 2024-04-25T19:15:25.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.089+0000 7fb922ffd640 1 -- 172.21.15.76:0/98923735 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7fb92805f3b0 con 0x7fb9340cc9e0 2024-04-25T19:15:25.091 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:25.091 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":21,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","created":"2024-04-25T19:15:01.693137+0000","modified":"2024-04-25T19:15:20.401211+0000","last_up_change":"2024-04-25T19:15:11.891681+0000","last_in_change":"2024-04-25T19:15:03.898065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:15:13.846514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:15:16.606533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:15:17.719197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72a16c02-48f9-45aa-9447-fcf755ebf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409574998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409574998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409574998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409574998}]},"public_addr":"172.21.15.76:6817/1409574998","cluster_addr":"172.21.15.76:6819/1409574998","heartbeat_back_addr":"172.21.15.76:6823/1409574998","heartbeat_front_addr":"172.21.15.76:6821/1409574998","state":["exists","up"]},{"osd":1,"uuid":"818d50f0-76c6-45f8-bf79-5e304f14c5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6801","nonce":1302663821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6803","nonce":1302663821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6807","nonce":1302663821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6805","nonce":1302663821}]},"public_addr":"172.21.15.76:6801/1302663821","cluster_addr":"172.21.15.76:6803/1302663821","heartbeat_back_addr":"172.21.15.76:6807/1302663821","heartbeat_front_addr":"172.21.15.76:6805/1302663821","state":["exists","up"]},{"osd":2,"uuid":"5714c8f5-ae23-473c-8cf1-db73811acded","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6809","nonce":1992938181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6811","nonce":1992938181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6815","nonce":1992938181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6813","nonce":1992938181}]},"public_addr":"172.21.15.76:6809/1992938181","cluster_addr":"172.21.15.76:6811/1992938181","heartbeat_back_addr":"172.21.15.76:6815/1992938181","heartbeat_front_addr":"172.21.15.76:6813/1992938181","state":["exists","up"]},{"osd":3,"uuid":"13204cdb-d8e6-4f97-bc6e-9561f442decf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6825","nonce":3418257163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6827","nonce":3418257163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6831","nonce":3418257163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6829","nonce":3418257163}]},"public_addr":"172.21.15.76:6825/3418257163","cluster_addr":"172.21.15.76:6827/3418257163","heartbeat_back_addr":"172.21.15.76:6831/3418257163","heartbeat_front_addr":"172.21.15.76:6829/3418257163","state":["exists","up"]},{"osd":4,"uuid":"5bea5166-1986-4d56-b5f5-ecc8cbe7f3ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6801","nonce":2125002468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6803","nonce":2125002468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6807","nonce":2125002468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6805","nonce":2125002468}]},"public_addr":"172.21.15.105:6801/2125002468","cluster_addr":"172.21.15.105:6803/2125002468","heartbeat_back_addr":"172.21.15.105:6807/2125002468","heartbeat_front_addr":"172.21.15.105:6805/2125002468","state":["exists","up"]},{"osd":5,"uuid":"96f89dcc-270d-4421-b8e8-3b92ba5f9844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6819","nonce":3413103121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6821","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6823","nonce":3413103121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6831","nonce":3413103121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6828","nonce":3413103121}]},"public_addr":"172.21.15.105:6819/3413103121","cluster_addr":"172.21.15.105:6823/3413103121","heartbeat_back_addr":"172.21.15.105:6831/3413103121","heartbeat_front_addr":"172.21.15.105:6828/3413103121","state":["exists","up"]},{"osd":6,"uuid":"93eb087a-d9ba-4096-b85d-b74d96e819a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6809","nonce":2721832954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6811","nonce":2721832954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6815","nonce":2721832954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6813","nonce":2721832954}]},"public_addr":"172.21.15.105:6809/2721832954","cluster_addr":"172.21.15.105:6811/2721832954","heartbeat_back_addr":"172.21.15.105:6815/2721832954","heartbeat_front_addr":"172.21.15.105:6813/2721832954","state":["exists","up"]},{"osd":7,"uuid":"2f80f790-ad2d-460e-a37d-47bf1771ef43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6818","nonce":548350430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6822","nonce":548350430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6829","nonce":548350430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6825","nonce":548350430}]},"public_addr":"172.21.15.105:6818/548350430","cluster_addr":"172.21.15.105:6822/548350430","heartbeat_back_addr":"172.21.15.105:6829/548350430","heartbeat_front_addr":"172.21.15.105:6825/548350430","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.106901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.072819+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.126865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.053464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:08.982614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.104825+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.112714+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.107387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:15:25.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.091+0000 7fb9388e7640 1 -- 172.21.15.76:0/98923735 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb9140744e0 msgr2=0x7fb9140769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:25.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.091+0000 7fb9388e7640 1 --2- 172.21.15.76:0/98923735 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb9140744e0 0x7fb9140769f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb924034000 tx=0x7fb92402e000 comp rx=0 tx=0).stop 2024-04-25T19:15:25.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.091+0000 7fb9388e7640 1 -- 172.21.15.76:0/98923735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9340cc9e0 msgr2=0x7fb934150660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:25.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.091+0000 7fb9388e7640 1 --2- 172.21.15.76:0/98923735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9340cc9e0 0x7fb934150660 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb92800d830 tx=0x7fb92800dcf0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.091+0000 7fb9388e7640 1 -- 172.21.15.76:0/98923735 shutdown_connections 2024-04-25T19:15:25.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.091+0000 7fb9388e7640 1 --2- 172.21.15.76:0/98923735 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb9140744e0 0x7fb9140769f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.091+0000 7fb9388e7640 1 --2- 172.21.15.76:0/98923735 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb93415d780 0x7fb934152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.091+0000 7fb9388e7640 1 --2- 172.21.15.76:0/98923735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9340cc9e0 0x7fb934150660 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.091+0000 7fb9388e7640 1 --2- 172.21.15.76:0/98923735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb93406de20 0x7fb9341f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.091+0000 7fb9388e7640 1 -- 172.21.15.76:0/98923735 >> 172.21.15.76:0/98923735 conn(0x7fb9340684e0 msgr2=0x7fb9340cec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:25.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.091+0000 7fb9388e7640 1 -- 172.21.15.76:0/98923735 shutdown_connections 2024-04-25T19:15:25.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.091+0000 7fb9388e7640 1 -- 172.21.15.76:0/98923735 wait complete. 2024-04-25T19:15:25.103 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T19:15:25.104 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T19:15:25.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.209+0000 7f22f8bb7640 1 Processor -- start 2024-04-25T19:15:25.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.209+0000 7f22f8bb7640 1 -- start start 2024-04-25T19:15:25.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.211+0000 7f22f8bb7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22f415cd10 0x7f22f4156c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:25.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.211+0000 7f22f8bb7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f22f406e4d0 0x7f22f4072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:25.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.211+0000 7f22f8bb7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22f41571c0 con 0x7f22f406e0c0 2024-04-25T19:15:25.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.211+0000 7f22f8bb7640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f22f406e8e0 con 0x7f22f406e4d0 2024-04-25T19:15:25.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.211+0000 7f22f8bb7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f22f415d250 con 0x7f22f415cd10 2024-04-25T19:15:25.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.211+0000 7f22f1d9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f22f406e0c0 0x7f22f41609c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35186/0 (socket says 172.21.15.76:35186) 2024-04-25T19:15:25.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.211+0000 7f22f1d9b640 1 -- 172.21.15.76:0/2266385658 learned_addr learned my addr 172.21.15.76:0/2266385658 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:25.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.211+0000 7f22f259c640 1 --2- 172.21.15.76:0/2266385658 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22f415cd10 0x7f22f4156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:25.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.211+0000 7f22f2d9d640 1 --2- 172.21.15.76:0/2266385658 >> v2:172.21.15.105:3300/0 conn(0x7f22f406e4d0 0x7f22f4072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:25.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.211+0000 7f22f159a640 1 -- 172.21.15.76:0/2266385658 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3987071593 0 0) 0x7f22f41571c0 con 0x7f22f406e0c0 2024-04-25T19:15:25.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.211+0000 7f22f159a640 1 -- 172.21.15.76:0/2266385658 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22dc003520 con 0x7f22f406e0c0 2024-04-25T19:15:25.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.211+0000 7f22f259c640 1 -- 172.21.15.76:0/2266385658 >> v2:172.21.15.105:3300/0 conn(0x7f22f406e4d0 msgr2=0x7f22f4072ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:25.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.211+0000 7f22f259c640 1 --2- 172.21.15.76:0/2266385658 >> v2:172.21.15.105:3300/0 conn(0x7f22f406e4d0 0x7f22f4072ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.211+0000 7f22f259c640 1 -- 172.21.15.76:0/2266385658 >> v1:172.21.15.76:6789/0 conn(0x7f22f406e0c0 legacy=0x7f22f41609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:25.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.211+0000 7f22f259c640 1 -- 172.21.15.76:0/2266385658 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f415d510 con 0x7f22f415cd10 2024-04-25T19:15:25.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.211+0000 7f22f2d9d640 1 --2- 172.21.15.76:0/2266385658 >> v2:172.21.15.105:3300/0 conn(0x7f22f406e4d0 0x7f22f4072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:25.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.212+0000 7f22f259c640 1 --2- 172.21.15.76:0/2266385658 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22f415cd10 0x7f22f4156c80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009d90 tx=0x7f22e402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b66a728d339efb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:25.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.212+0000 7f22f159a640 1 -- 172.21.15.76:0/2266385658 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4007740 con 0x7f22f415cd10 2024-04-25T19:15:25.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.212+0000 7f22f159a640 1 -- 172.21.15.76:0/2266385658 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22e40078e0 con 0x7f22f415cd10 2024-04-25T19:15:25.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.212+0000 7f22f159a640 1 -- 172.21.15.76:0/2266385658 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4007c90 con 0x7f22f415cd10 2024-04-25T19:15:25.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.213+0000 7f22f8bb7640 1 -- 172.21.15.76:0/2266385658 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22f415cd10 msgr2=0x7f22f4156c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:25.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.213+0000 7f22f8bb7640 1 --2- 172.21.15.76:0/2266385658 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22f415cd10 0x7f22f4156c80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009d90 tx=0x7f22e402a660 comp rx=0 tx=0).stop 2024-04-25T19:15:25.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.213+0000 7f22f8bb7640 1 -- 172.21.15.76:0/2266385658 shutdown_connections 2024-04-25T19:15:25.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.213+0000 7f22f8bb7640 1 --2- 172.21.15.76:0/2266385658 >> v2:172.21.15.105:3300/0 conn(0x7f22f406e4d0 0x7f22f4072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.213+0000 7f22f8bb7640 1 --2- 172.21.15.76:0/2266385658 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22f415cd10 0x7f22f4156c80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.213+0000 7f22f8bb7640 1 -- 172.21.15.76:0/2266385658 >> 172.21.15.76:0/2266385658 conn(0x7f22f4070b00 msgr2=0x7f22f4070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:25.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.213+0000 7f22f8bb7640 1 -- 172.21.15.76:0/2266385658 shutdown_connections 2024-04-25T19:15:25.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.213+0000 7f22f8bb7640 1 -- 172.21.15.76:0/2266385658 wait complete. 2024-04-25T19:15:25.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.215+0000 7f22f8bb7640 1 Processor -- start 2024-04-25T19:15:25.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.215+0000 7f22f8bb7640 1 -- start start 2024-04-25T19:15:25.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.216+0000 7f22f8bb7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22f406e0c0 0x7f22f40cf620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:25.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.216+0000 7f22f8bb7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22f406e4d0 0x7f22f41e8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:25.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.216+0000 7f22f8bb7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22f415cd10 0x7f22f40c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:25.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.216+0000 7f22f8bb7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f22f41604a0 con 0x7f22f406e4d0 2024-04-25T19:15:25.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.216+0000 7f22f8bb7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f22f415dad0 con 0x7f22f406e0c0 2024-04-25T19:15:25.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.216+0000 7f22f8bb7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f22f41610d0 con 0x7f22f415cd10 2024-04-25T19:15:25.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.216+0000 7f22f1d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22f406e4d0 0x7f22f41e8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:25.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.216+0000 7f22f1d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22f406e4d0 0x7f22f41e8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58996/0 (socket says 172.21.15.76:58996) 2024-04-25T19:15:25.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.216+0000 7f22f1d9b640 1 -- 172.21.15.76:0/90312708 learned_addr learned my addr 172.21.15.76:0/90312708 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:25.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.216+0000 7f22f259c640 1 --2- 172.21.15.76:0/90312708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22f406e0c0 0x7f22f40cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:25.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.216+0000 7f22f2d9d640 1 --2- 172.21.15.76:0/90312708 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22f415cd10 0x7f22f40c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:25.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.217+0000 7f22f1d9b640 1 -- 172.21.15.76:0/90312708 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22f415cd10 msgr2=0x7f22f40c8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:25.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.217+0000 7f22f1d9b640 1 --2- 172.21.15.76:0/90312708 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22f415cd10 0x7f22f40c8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.217+0000 7f22f1d9b640 1 -- 172.21.15.76:0/90312708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22f406e0c0 msgr2=0x7f22f40cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:25.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.217+0000 7f22f1d9b640 1 --2- 172.21.15.76:0/90312708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22f406e0c0 0x7f22f40cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.217+0000 7f22f1d9b640 1 -- 172.21.15.76:0/90312708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f40ce510 con 0x7f22f406e4d0 2024-04-25T19:15:25.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.217+0000 7f22f1d9b640 1 --2- 172.21.15.76:0/90312708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22f406e4d0 0x7f22f41e8a00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f22e800c390 tx=0x7f22e800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:25.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.217+0000 7f22e2ffd640 1 -- 172.21.15.76:0/90312708 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e8017070 con 0x7f22f406e4d0 2024-04-25T19:15:25.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.217+0000 7f22e2ffd640 1 -- 172.21.15.76:0/90312708 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22e8005ce0 con 0x7f22f406e4d0 2024-04-25T19:15:25.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.217+0000 7f22f8bb7640 1 -- 172.21.15.76:0/90312708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f40ce7f0 con 0x7f22f406e4d0 2024-04-25T19:15:25.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.217+0000 7f22e2ffd640 1 -- 172.21.15.76:0/90312708 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e80072f0 con 0x7f22f406e4d0 2024-04-25T19:15:25.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.217+0000 7f22f8bb7640 1 -- 172.21.15.76:0/90312708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f40cded0 con 0x7f22f406e4d0 2024-04-25T19:15:25.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.218+0000 7f22f8bb7640 1 -- 172.21.15.76:0/90312708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f415bb50 con 0x7f22f406e4d0 2024-04-25T19:15:25.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.218+0000 7f22e2ffd640 1 -- 172.21.15.76:0/90312708 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f22e801a070 con 0x7f22f406e4d0 2024-04-25T19:15:25.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.219+0000 7f22e2ffd640 1 --2- 172.21.15.76:0/90312708 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f22d40744e0 0x7f22d40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:25.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.219+0000 7f22e2ffd640 1 -- 172.21.15.76:0/90312708 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f22e8094590 con 0x7f22f406e4d0 2024-04-25T19:15:25.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.219+0000 7f22f259c640 1 --2- 172.21.15.76:0/90312708 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f22d40744e0 0x7f22d40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:25.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.220+0000 7f22f259c640 1 --2- 172.21.15.76:0/90312708 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f22d40744e0 0x7f22d40769f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f22e4034000 tx=0x7f22e402f000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:25.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.223+0000 7f22e2ffd640 1 -- 172.21.15.76:0/90312708 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f22f415bb50 con 0x7f22f406e4d0 2024-04-25T19:15:25.303 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:15:25.306 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:15:25.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.399+0000 7f22f8bb7640 1 -- 172.21.15.76:0/90312708 --> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f22f40cea10 con 0x7f22d40744e0 2024-04-25T19:15:25.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.406+0000 7f22e2ffd640 1 -- 172.21.15.76:0/90312708 <== mgr.4109 v2:172.21.15.105:6832/2585775531 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251914 (secure 0 0 0) 0x7f22e4003b60 con 0x7f22d40744e0 2024-04-25T19:15:25.411 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:25.412 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-04-25T19:15:25.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.409+0000 7f22f8bb7640 1 -- 172.21.15.76:0/90312708 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f22d40744e0 msgr2=0x7f22d40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:25.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.409+0000 7f22f8bb7640 1 --2- 172.21.15.76:0/90312708 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f22d40744e0 0x7f22d40769f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f22e4034000 tx=0x7f22e402f000 comp rx=0 tx=0).stop 2024-04-25T19:15:25.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.409+0000 7f22f8bb7640 1 -- 172.21.15.76:0/90312708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22f406e4d0 msgr2=0x7f22f41e8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:25.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.409+0000 7f22f8bb7640 1 --2- 172.21.15.76:0/90312708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22f406e4d0 0x7f22f41e8a00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f22e800c390 tx=0x7f22e800c850 comp rx=0 tx=0).stop 2024-04-25T19:15:25.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.409+0000 7f22f8bb7640 1 -- 172.21.15.76:0/90312708 shutdown_connections 2024-04-25T19:15:25.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.409+0000 7f22f8bb7640 1 --2- 172.21.15.76:0/90312708 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f22d40744e0 0x7f22d40769f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.409+0000 7f22f8bb7640 1 --2- 172.21.15.76:0/90312708 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f22f415cd10 0x7f22f40c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.409+0000 7f22f8bb7640 1 --2- 172.21.15.76:0/90312708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22f406e4d0 0x7f22f41e8a00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.409+0000 7f22f8bb7640 1 --2- 172.21.15.76:0/90312708 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22f406e0c0 0x7f22f40cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.409+0000 7f22f8bb7640 1 -- 172.21.15.76:0/90312708 >> 172.21.15.76:0/90312708 conn(0x7f22f4070b00 msgr2=0x7f22f4072e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:25.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.409+0000 7f22f8bb7640 1 -- 172.21.15.76:0/90312708 shutdown_connections 2024-04-25T19:15:25.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.409+0000 7f22f8bb7640 1 -- 172.21.15.76:0/90312708 wait complete. 2024-04-25T19:15:25.422 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-25T19:15:23.829568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218008,"kb_used_data":1824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756712,"statfs":{"total":773094113280,"available":772870873088,"internally_reserved":0,"allocated":1867776,"data_stored":1785576,"data_compressed":64838,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.426469"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425589+0000","last_change":"2024-04-25T19:15:19.433569+0000","last_active":"2024-04-25T19:15:20.425589+0000","last_peered":"2024-04-25T19:15:20.425589+0000","last_clean":"2024-04-25T19:15:20.425589+0000","last_became_active":"2024-04-25T19:15:19.431332+0000","last_became_peered":"2024-04-25T19:15:19.431332+0000","last_unstale":"2024-04-25T19:15:20.425589+0000","last_undegraded":"2024-04-25T19:15:20.425589+0000","last_fullsized":"2024-04-25T19:15:20.425589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:57:01.213834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404926+0000","last_change":"2024-04-25T19:15:19.418695+0000","last_active":"2024-04-25T19:15:20.404926+0000","last_peered":"2024-04-25T19:15:20.404926+0000","last_clean":"2024-04-25T19:15:20.404926+0000","last_became_active":"2024-04-25T19:15:19.418235+0000","last_became_peered":"2024-04-25T19:15:19.418235+0000","last_unstale":"2024-04-25T19:15:20.404926+0000","last_undegraded":"2024-04-25T19:15:20.404926+0000","last_fullsized":"2024-04-25T19:15:20.404926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:30.516472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804491+0000","last_change":"2024-04-25T19:15:19.416983+0000","last_active":"2024-04-25T19:15:20.804491+0000","last_peered":"2024-04-25T19:15:20.804491+0000","last_clean":"2024-04-25T19:15:20.804491+0000","last_became_active":"2024-04-25T19:15:19.416443+0000","last_became_peered":"2024-04-25T19:15:19.416443+0000","last_unstale":"2024-04-25T19:15:20.804491+0000","last_undegraded":"2024-04-25T19:15:20.804491+0000","last_fullsized":"2024-04-25T19:15:20.804491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:02.794164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848308+0000","last_change":"2024-04-25T19:15:19.429058+0000","last_active":"2024-04-25T19:15:20.848308+0000","last_peered":"2024-04-25T19:15:20.848308+0000","last_clean":"2024-04-25T19:15:20.848308+0000","last_became_active":"2024-04-25T19:15:19.428419+0000","last_became_peered":"2024-04-25T19:15:19.428419+0000","last_unstale":"2024-04-25T19:15:20.848308+0000","last_undegraded":"2024-04-25T19:15:20.848308+0000","last_fullsized":"2024-04-25T19:15:20.848308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:23:56.817616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425402+0000","last_change":"2024-04-25T19:15:19.427615+0000","last_active":"2024-04-25T19:15:20.425402+0000","last_peered":"2024-04-25T19:15:20.425402+0000","last_clean":"2024-04-25T19:15:20.425402+0000","last_became_active":"2024-04-25T19:15:19.426968+0000","last_became_peered":"2024-04-25T19:15:19.426968+0000","last_unstale":"2024-04-25T19:15:20.425402+0000","last_undegraded":"2024-04-25T19:15:20.425402+0000","last_fullsized":"2024-04-25T19:15:20.425402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:42:00.686712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847989+0000","last_change":"2024-04-25T19:15:19.417602+0000","last_active":"2024-04-25T19:15:20.847989+0000","last_peered":"2024-04-25T19:15:20.847989+0000","last_clean":"2024-04-25T19:15:20.847989+0000","last_became_active":"2024-04-25T19:15:19.417292+0000","last_became_peered":"2024-04-25T19:15:19.417292+0000","last_unstale":"2024-04-25T19:15:20.847989+0000","last_undegraded":"2024-04-25T19:15:20.847989+0000","last_fullsized":"2024-04-25T19:15:20.847989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:37:43.583809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425114+0000","last_change":"2024-04-25T19:15:19.428180+0000","last_active":"2024-04-25T19:15:20.425114+0000","last_peered":"2024-04-25T19:15:20.425114+0000","last_clean":"2024-04-25T19:15:20.425114+0000","last_became_active":"2024-04-25T19:15:19.427852+0000","last_became_peered":"2024-04-25T19:15:19.427852+0000","last_unstale":"2024-04-25T19:15:20.425114+0000","last_undegraded":"2024-04-25T19:15:20.425114+0000","last_fullsized":"2024-04-25T19:15:20.425114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:51.300880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405446+0000","last_change":"2024-04-25T19:15:19.418253+0000","last_active":"2024-04-25T19:15:20.405446+0000","last_peered":"2024-04-25T19:15:20.405446+0000","last_clean":"2024-04-25T19:15:20.405446+0000","last_became_active":"2024-04-25T19:15:19.417183+0000","last_became_peered":"2024-04-25T19:15:19.417183+0000","last_unstale":"2024-04-25T19:15:20.405446+0000","last_undegraded":"2024-04-25T19:15:20.405446+0000","last_fullsized":"2024-04-25T19:15:20.405446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:05:13.181646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425578+0000","last_change":"2024-04-25T19:15:19.431671+0000","last_active":"2024-04-25T19:15:20.425578+0000","last_peered":"2024-04-25T19:15:20.425578+0000","last_clean":"2024-04-25T19:15:20.425578+0000","last_became_active":"2024-04-25T19:15:19.431240+0000","last_became_peered":"2024-04-25T19:15:19.431240+0000","last_unstale":"2024-04-25T19:15:20.425578+0000","last_undegraded":"2024-04-25T19:15:20.425578+0000","last_fullsized":"2024-04-25T19:15:20.425578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:22.759462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.406080+0000","last_change":"2024-04-25T19:15:19.416304+0000","last_active":"2024-04-25T19:15:20.406080+0000","last_peered":"2024-04-25T19:15:20.406080+0000","last_clean":"2024-04-25T19:15:20.406080+0000","last_became_active":"2024-04-25T19:15:19.415967+0000","last_became_peered":"2024-04-25T19:15:19.415967+0000","last_unstale":"2024-04-25T19:15:20.406080+0000","last_undegraded":"2024-04-25T19:15:20.406080+0000","last_fullsized":"2024-04-25T19:15:20.406080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:30:52.958731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405773+0000","last_change":"2024-04-25T19:15:19.417596+0000","last_active":"2024-04-25T19:15:20.405773+0000","last_peered":"2024-04-25T19:15:20.405773+0000","last_clean":"2024-04-25T19:15:20.405773+0000","last_became_active":"2024-04-25T19:15:19.417230+0000","last_became_peered":"2024-04-25T19:15:19.417230+0000","last_unstale":"2024-04-25T19:15:20.405773+0000","last_undegraded":"2024-04-25T19:15:20.405773+0000","last_fullsized":"2024-04-25T19:15:20.405773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:28:12.387651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822843+0000","last_change":"2024-04-25T19:15:19.420564+0000","last_active":"2024-04-25T19:15:20.822843+0000","last_peered":"2024-04-25T19:15:20.822843+0000","last_clean":"2024-04-25T19:15:20.822843+0000","last_became_active":"2024-04-25T19:15:19.420044+0000","last_became_peered":"2024-04-25T19:15:19.420044+0000","last_unstale":"2024-04-25T19:15:20.822843+0000","last_undegraded":"2024-04-25T19:15:20.822843+0000","last_fullsized":"2024-04-25T19:15:20.822843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:04.066030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405269+0000","last_change":"2024-04-25T19:15:19.415500+0000","last_active":"2024-04-25T19:15:20.405269+0000","last_peered":"2024-04-25T19:15:20.405269+0000","last_clean":"2024-04-25T19:15:20.405269+0000","last_became_active":"2024-04-25T19:15:19.414007+0000","last_became_peered":"2024-04-25T19:15:19.414007+0000","last_unstale":"2024-04-25T19:15:20.405269+0000","last_undegraded":"2024-04-25T19:15:20.405269+0000","last_fullsized":"2024-04-25T19:15:20.405269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:56:12.320555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804235+0000","last_change":"2024-04-25T19:15:19.423884+0000","last_active":"2024-04-25T19:15:20.804235+0000","last_peered":"2024-04-25T19:15:20.804235+0000","last_clean":"2024-04-25T19:15:20.804235+0000","last_became_active":"2024-04-25T19:15:19.422575+0000","last_became_peered":"2024-04-25T19:15:19.422575+0000","last_unstale":"2024-04-25T19:15:20.804235+0000","last_undegraded":"2024-04-25T19:15:20.804235+0000","last_fullsized":"2024-04-25T19:15:20.804235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:30.988950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404396+0000","last_change":"2024-04-25T19:15:19.424267+0000","last_active":"2024-04-25T19:15:20.404396+0000","last_peered":"2024-04-25T19:15:20.404396+0000","last_clean":"2024-04-25T19:15:20.404396+0000","last_became_active":"2024-04-25T19:15:19.422953+0000","last_became_peered":"2024-04-25T19:15:19.422953+0000","last_unstale":"2024-04-25T19:15:20.404396+0000","last_undegraded":"2024-04-25T19:15:20.404396+0000","last_fullsized":"2024-04-25T19:15:20.404396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:42.823233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847983+0000","last_change":"2024-04-25T19:15:19.420374+0000","last_active":"2024-04-25T19:15:20.847983+0000","last_peered":"2024-04-25T19:15:20.847983+0000","last_clean":"2024-04-25T19:15:20.847983+0000","last_became_active":"2024-04-25T19:15:19.420015+0000","last_became_peered":"2024-04-25T19:15:19.420015+0000","last_unstale":"2024-04-25T19:15:20.847983+0000","last_undegraded":"2024-04-25T19:15:20.847983+0000","last_fullsized":"2024-04-25T19:15:20.847983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:04:26.719359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848705+0000","last_change":"2024-04-25T19:15:19.434476+0000","last_active":"2024-04-25T19:15:20.848705+0000","last_peered":"2024-04-25T19:15:20.848705+0000","last_clean":"2024-04-25T19:15:20.848705+0000","last_became_active":"2024-04-25T19:15:19.433836+0000","last_became_peered":"2024-04-25T19:15:19.433836+0000","last_unstale":"2024-04-25T19:15:20.848705+0000","last_undegraded":"2024-04-25T19:15:20.848705+0000","last_fullsized":"2024-04-25T19:15:20.848705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:13.234829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847359+0000","last_change":"2024-04-25T19:15:19.415521+0000","last_active":"2024-04-25T19:15:20.847359+0000","last_peered":"2024-04-25T19:15:20.847359+0000","last_clean":"2024-04-25T19:15:20.847359+0000","last_became_active":"2024-04-25T19:15:19.415177+0000","last_became_peered":"2024-04-25T19:15:19.415177+0000","last_unstale":"2024-04-25T19:15:20.847359+0000","last_undegraded":"2024-04-25T19:15:20.847359+0000","last_fullsized":"2024-04-25T19:15:20.847359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:24.355619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425590+0000","last_change":"2024-04-25T19:15:19.430291+0000","last_active":"2024-04-25T19:15:20.425590+0000","last_peered":"2024-04-25T19:15:20.425590+0000","last_clean":"2024-04-25T19:15:20.425590+0000","last_became_active":"2024-04-25T19:15:19.427226+0000","last_became_peered":"2024-04-25T19:15:19.427226+0000","last_unstale":"2024-04-25T19:15:20.425590+0000","last_undegraded":"2024-04-25T19:15:20.425590+0000","last_fullsized":"2024-04-25T19:15:20.425590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:38.464211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405750+0000","last_change":"2024-04-25T19:15:19.421696+0000","last_active":"2024-04-25T19:15:20.405750+0000","last_peered":"2024-04-25T19:15:20.405750+0000","last_clean":"2024-04-25T19:15:20.405750+0000","last_became_active":"2024-04-25T19:15:19.421300+0000","last_became_peered":"2024-04-25T19:15:19.421300+0000","last_unstale":"2024-04-25T19:15:20.405750+0000","last_undegraded":"2024-04-25T19:15:20.405750+0000","last_fullsized":"2024-04-25T19:15:20.405750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:15.226398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804410+0000","last_change":"2024-04-25T19:15:19.427880+0000","last_active":"2024-04-25T19:15:20.804410+0000","last_peered":"2024-04-25T19:15:20.804410+0000","last_clean":"2024-04-25T19:15:20.804410+0000","last_became_active":"2024-04-25T19:15:19.427579+0000","last_became_peered":"2024-04-25T19:15:19.427579+0000","last_unstale":"2024-04-25T19:15:20.804410+0000","last_undegraded":"2024-04-25T19:15:20.804410+0000","last_fullsized":"2024-04-25T19:15:20.804410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:47:10.657715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848145+0000","last_change":"2024-04-25T19:15:19.430409+0000","last_active":"2024-04-25T19:15:20.848145+0000","last_peered":"2024-04-25T19:15:20.848145+0000","last_clean":"2024-04-25T19:15:20.848145+0000","last_became_active":"2024-04-25T19:15:19.428305+0000","last_became_peered":"2024-04-25T19:15:19.428305+0000","last_unstale":"2024-04-25T19:15:20.848145+0000","last_undegraded":"2024-04-25T19:15:20.848145+0000","last_fullsized":"2024-04-25T19:15:20.848145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:10:01.268963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405396+0000","last_change":"2024-04-25T19:15:19.424435+0000","last_active":"2024-04-25T19:15:20.405396+0000","last_peered":"2024-04-25T19:15:20.405396+0000","last_clean":"2024-04-25T19:15:20.405396+0000","last_became_active":"2024-04-25T19:15:19.422763+0000","last_became_peered":"2024-04-25T19:15:19.422763+0000","last_unstale":"2024-04-25T19:15:20.405396+0000","last_undegraded":"2024-04-25T19:15:20.405396+0000","last_fullsized":"2024-04-25T19:15:20.405396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:58.092412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848072+0000","last_change":"2024-04-25T19:15:19.420275+0000","last_active":"2024-04-25T19:15:20.848072+0000","last_peered":"2024-04-25T19:15:20.848072+0000","last_clean":"2024-04-25T19:15:20.848072+0000","last_became_active":"2024-04-25T19:15:19.419936+0000","last_became_peered":"2024-04-25T19:15:19.419936+0000","last_unstale":"2024-04-25T19:15:20.848072+0000","last_undegraded":"2024-04-25T19:15:20.848072+0000","last_fullsized":"2024-04-25T19:15:20.848072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:55.710620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405087+0000","last_change":"2024-04-25T19:15:19.423408+0000","last_active":"2024-04-25T19:15:20.405087+0000","last_peered":"2024-04-25T19:15:20.405087+0000","last_clean":"2024-04-25T19:15:20.405087+0000","last_became_active":"2024-04-25T19:15:19.423116+0000","last_became_peered":"2024-04-25T19:15:19.423116+0000","last_unstale":"2024-04-25T19:15:20.405087+0000","last_undegraded":"2024-04-25T19:15:20.405087+0000","last_fullsized":"2024-04-25T19:15:20.405087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:18:57.787013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.406015+0000","last_change":"2024-04-25T19:15:19.428826+0000","last_active":"2024-04-25T19:15:20.406015+0000","last_peered":"2024-04-25T19:15:20.406015+0000","last_clean":"2024-04-25T19:15:20.406015+0000","last_became_active":"2024-04-25T19:15:19.428474+0000","last_became_peered":"2024-04-25T19:15:19.428474+0000","last_unstale":"2024-04-25T19:15:20.406015+0000","last_undegraded":"2024-04-25T19:15:20.406015+0000","last_fullsized":"2024-04-25T19:15:20.406015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:19.974851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404739+0000","last_change":"2024-04-25T19:15:19.423353+0000","last_active":"2024-04-25T19:15:20.404739+0000","last_peered":"2024-04-25T19:15:20.404739+0000","last_clean":"2024-04-25T19:15:20.404739+0000","last_became_active":"2024-04-25T19:15:19.422795+0000","last_became_peered":"2024-04-25T19:15:19.422795+0000","last_unstale":"2024-04-25T19:15:20.404739+0000","last_undegraded":"2024-04-25T19:15:20.404739+0000","last_fullsized":"2024-04-25T19:15:20.404739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:34.119976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405076+0000","last_change":"2024-04-25T19:15:19.421809+0000","last_active":"2024-04-25T19:15:20.405076+0000","last_peered":"2024-04-25T19:15:20.405076+0000","last_clean":"2024-04-25T19:15:20.405076+0000","last_became_active":"2024-04-25T19:15:19.421515+0000","last_became_peered":"2024-04-25T19:15:19.421515+0000","last_unstale":"2024-04-25T19:15:20.405076+0000","last_undegraded":"2024-04-25T19:15:20.405076+0000","last_fullsized":"2024-04-25T19:15:20.405076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:22.603291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804477+0000","last_change":"2024-04-25T19:15:19.410183+0000","last_active":"2024-04-25T19:15:20.804477+0000","last_peered":"2024-04-25T19:15:20.804477+0000","last_clean":"2024-04-25T19:15:20.804477+0000","last_became_active":"2024-04-25T19:15:19.409857+0000","last_became_peered":"2024-04-25T19:15:19.409857+0000","last_unstale":"2024-04-25T19:15:20.804477+0000","last_undegraded":"2024-04-25T19:15:20.804477+0000","last_fullsized":"2024-04-25T19:15:20.804477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:30.180786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848239+0000","last_change":"2024-04-25T19:15:19.425524+0000","last_active":"2024-04-25T19:15:20.848239+0000","last_peered":"2024-04-25T19:15:20.848239+0000","last_clean":"2024-04-25T19:15:20.848239+0000","last_became_active":"2024-04-25T19:15:19.424578+0000","last_became_peered":"2024-04-25T19:15:19.424578+0000","last_unstale":"2024-04-25T19:15:20.848239+0000","last_undegraded":"2024-04-25T19:15:20.848239+0000","last_fullsized":"2024-04-25T19:15:20.848239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:02.103879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847243+0000","last_change":"2024-04-25T19:15:19.426720+0000","last_active":"2024-04-25T19:15:20.847243+0000","last_peered":"2024-04-25T19:15:20.847243+0000","last_clean":"2024-04-25T19:15:20.847243+0000","last_became_active":"2024-04-25T19:15:19.426139+0000","last_became_peered":"2024-04-25T19:15:19.426139+0000","last_unstale":"2024-04-25T19:15:20.847243+0000","last_undegraded":"2024-04-25T19:15:20.847243+0000","last_fullsized":"2024-04-25T19:15:20.847243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:31.608993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405390+0000","last_change":"2024-04-25T19:15:19.413805+0000","last_active":"2024-04-25T19:15:20.405390+0000","last_peered":"2024-04-25T19:15:20.405390+0000","last_clean":"2024-04-25T19:15:20.405390+0000","last_became_active":"2024-04-25T19:15:19.413455+0000","last_became_peered":"2024-04-25T19:15:19.413455+0000","last_unstale":"2024-04-25T19:15:20.405390+0000","last_undegraded":"2024-04-25T19:15:20.405390+0000","last_fullsized":"2024-04-25T19:15:20.405390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:05:45.008076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822709+0000","last_change":"2024-04-25T19:15:19.423770+0000","last_active":"2024-04-25T19:15:20.822709+0000","last_peered":"2024-04-25T19:15:20.822709+0000","last_clean":"2024-04-25T19:15:20.822709+0000","last_became_active":"2024-04-25T19:15:19.423406+0000","last_became_peered":"2024-04-25T19:15:19.423406+0000","last_unstale":"2024-04-25T19:15:20.822709+0000","last_undegraded":"2024-04-25T19:15:20.822709+0000","last_fullsized":"2024-04-25T19:15:20.822709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:41:23.669832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405343+0000","last_change":"2024-04-25T19:15:18.422035+0000","last_active":"2024-04-25T19:15:20.405343+0000","last_peered":"2024-04-25T19:15:20.405343+0000","last_clean":"2024-04-25T19:15:20.405343+0000","last_became_active":"2024-04-25T19:15:18.421658+0000","last_became_peered":"2024-04-25T19:15:18.421658+0000","last_unstale":"2024-04-25T19:15:20.405343+0000","last_undegraded":"2024-04-25T19:15:20.405343+0000","last_fullsized":"2024-04-25T19:15:20.405343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:23.349126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804571+0000","last_change":"2024-04-25T19:15:19.421805+0000","last_active":"2024-04-25T19:15:20.804571+0000","last_peered":"2024-04-25T19:15:20.804571+0000","last_clean":"2024-04-25T19:15:20.804571+0000","last_became_active":"2024-04-25T19:15:19.421497+0000","last_became_peered":"2024-04-25T19:15:19.421497+0000","last_unstale":"2024-04-25T19:15:20.804571+0000","last_undegraded":"2024-04-25T19:15:20.804571+0000","last_fullsized":"2024-04-25T19:15:20.804571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:29:29.949508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405485+0000","last_change":"2024-04-25T19:15:18.416894+0000","last_active":"2024-04-25T19:15:20.405485+0000","last_peered":"2024-04-25T19:15:20.405485+0000","last_clean":"2024-04-25T19:15:20.405485+0000","last_became_active":"2024-04-25T19:15:18.416539+0000","last_became_peered":"2024-04-25T19:15:18.416539+0000","last_unstale":"2024-04-25T19:15:20.405485+0000","last_undegraded":"2024-04-25T19:15:20.405485+0000","last_fullsized":"2024-04-25T19:15:20.405485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:30.388606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405329+0000","last_change":"2024-04-25T19:15:18.420129+0000","last_active":"2024-04-25T19:15:20.405329+0000","last_peered":"2024-04-25T19:15:20.405329+0000","last_clean":"2024-04-25T19:15:20.405329+0000","last_became_active":"2024-04-25T19:15:18.419732+0000","last_became_peered":"2024-04-25T19:15:18.419732+0000","last_unstale":"2024-04-25T19:15:20.405329+0000","last_undegraded":"2024-04-25T19:15:20.405329+0000","last_fullsized":"2024-04-25T19:15:20.405329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:41:48.673989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405615+0000","last_change":"2024-04-25T19:15:19.415655+0000","last_active":"2024-04-25T19:15:20.405615+0000","last_peered":"2024-04-25T19:15:20.405615+0000","last_clean":"2024-04-25T19:15:20.405615+0000","last_became_active":"2024-04-25T19:15:19.415287+0000","last_became_peered":"2024-04-25T19:15:19.415287+0000","last_unstale":"2024-04-25T19:15:20.405615+0000","last_undegraded":"2024-04-25T19:15:20.405615+0000","last_fullsized":"2024-04-25T19:15:20.405615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:49.155289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848076+0000","last_change":"2024-04-25T19:15:18.420711+0000","last_active":"2024-04-25T19:15:20.848076+0000","last_peered":"2024-04-25T19:15:20.848076+0000","last_clean":"2024-04-25T19:15:20.848076+0000","last_became_active":"2024-04-25T19:15:18.420362+0000","last_became_peered":"2024-04-25T19:15:18.420362+0000","last_unstale":"2024-04-25T19:15:20.848076+0000","last_undegraded":"2024-04-25T19:15:20.848076+0000","last_fullsized":"2024-04-25T19:15:20.848076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:48.245193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404375+0000","last_change":"2024-04-25T19:15:19.424356+0000","last_active":"2024-04-25T19:15:20.404375+0000","last_peered":"2024-04-25T19:15:20.404375+0000","last_clean":"2024-04-25T19:15:20.404375+0000","last_became_active":"2024-04-25T19:15:19.423556+0000","last_became_peered":"2024-04-25T19:15:19.423556+0000","last_unstale":"2024-04-25T19:15:20.404375+0000","last_undegraded":"2024-04-25T19:15:20.404375+0000","last_fullsized":"2024-04-25T19:15:20.404375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:19:35.427261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405514+0000","last_change":"2024-04-25T19:15:18.419250+0000","last_active":"2024-04-25T19:15:20.405514+0000","last_peered":"2024-04-25T19:15:20.405514+0000","last_clean":"2024-04-25T19:15:20.405514+0000","last_became_active":"2024-04-25T19:15:18.418927+0000","last_became_peered":"2024-04-25T19:15:18.418927+0000","last_unstale":"2024-04-25T19:15:20.405514+0000","last_undegraded":"2024-04-25T19:15:20.405514+0000","last_fullsized":"2024-04-25T19:15:20.405514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:26.012965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848606+0000","last_change":"2024-04-25T19:15:19.430117+0000","last_active":"2024-04-25T19:15:20.848606+0000","last_peered":"2024-04-25T19:15:20.848606+0000","last_clean":"2024-04-25T19:15:20.848606+0000","last_became_active":"2024-04-25T19:15:19.428985+0000","last_became_peered":"2024-04-25T19:15:19.428985+0000","last_unstale":"2024-04-25T19:15:20.848606+0000","last_undegraded":"2024-04-25T19:15:20.848606+0000","last_fullsized":"2024-04-25T19:15:20.848606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:36:29.069541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848490+0000","last_change":"2024-04-25T19:15:18.422931+0000","last_active":"2024-04-25T19:15:20.848490+0000","last_peered":"2024-04-25T19:15:20.848490+0000","last_clean":"2024-04-25T19:15:20.848490+0000","last_became_active":"2024-04-25T19:15:18.422610+0000","last_became_peered":"2024-04-25T19:15:18.422610+0000","last_unstale":"2024-04-25T19:15:20.848490+0000","last_undegraded":"2024-04-25T19:15:20.848490+0000","last_fullsized":"2024-04-25T19:15:20.848490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:18.489484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804598+0000","last_change":"2024-04-25T19:15:19.416109+0000","last_active":"2024-04-25T19:15:20.804598+0000","last_peered":"2024-04-25T19:15:20.804598+0000","last_clean":"2024-04-25T19:15:20.804598+0000","last_became_active":"2024-04-25T19:15:19.415734+0000","last_became_peered":"2024-04-25T19:15:19.415734+0000","last_unstale":"2024-04-25T19:15:20.804598+0000","last_undegraded":"2024-04-25T19:15:20.804598+0000","last_fullsized":"2024-04-25T19:15:20.804598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:57:31.406303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404631+0000","last_change":"2024-04-25T19:15:18.428677+0000","last_active":"2024-04-25T19:15:20.404631+0000","last_peered":"2024-04-25T19:15:20.404631+0000","last_clean":"2024-04-25T19:15:20.404631+0000","last_became_active":"2024-04-25T19:15:18.428432+0000","last_became_peered":"2024-04-25T19:15:18.428432+0000","last_unstale":"2024-04-25T19:15:20.404631+0000","last_undegraded":"2024-04-25T19:15:20.404631+0000","last_fullsized":"2024-04-25T19:15:20.404631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:38.478595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822814+0000","last_change":"2024-04-25T19:15:19.428245+0000","last_active":"2024-04-25T19:15:20.822814+0000","last_peered":"2024-04-25T19:15:20.822814+0000","last_clean":"2024-04-25T19:15:20.822814+0000","last_became_active":"2024-04-25T19:15:19.427941+0000","last_became_peered":"2024-04-25T19:15:19.427941+0000","last_unstale":"2024-04-25T19:15:20.822814+0000","last_undegraded":"2024-04-25T19:15:20.822814+0000","last_fullsized":"2024-04-25T19:15:20.822814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:09.947923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405735+0000","last_change":"2024-04-25T19:15:18.420881+0000","last_active":"2024-04-25T19:15:20.405735+0000","last_peered":"2024-04-25T19:15:20.405735+0000","last_clean":"2024-04-25T19:15:20.405735+0000","last_became_active":"2024-04-25T19:15:18.420303+0000","last_became_peered":"2024-04-25T19:15:18.420303+0000","last_unstale":"2024-04-25T19:15:20.405735+0000","last_undegraded":"2024-04-25T19:15:20.405735+0000","last_fullsized":"2024-04-25T19:15:20.405735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:39:24.356298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425356+0000","last_change":"2024-04-25T19:15:19.432735+0000","last_active":"2024-04-25T19:15:20.425356+0000","last_peered":"2024-04-25T19:15:20.425356+0000","last_clean":"2024-04-25T19:15:20.425356+0000","last_became_active":"2024-04-25T19:15:19.431828+0000","last_became_peered":"2024-04-25T19:15:19.431828+0000","last_unstale":"2024-04-25T19:15:20.425356+0000","last_undegraded":"2024-04-25T19:15:20.425356+0000","last_fullsized":"2024-04-25T19:15:20.425356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:20.189230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804550+0000","last_change":"2024-04-25T19:15:18.421008+0000","last_active":"2024-04-25T19:15:20.804550+0000","last_peered":"2024-04-25T19:15:20.804550+0000","last_clean":"2024-04-25T19:15:20.804550+0000","last_became_active":"2024-04-25T19:15:18.420402+0000","last_became_peered":"2024-04-25T19:15:18.420402+0000","last_unstale":"2024-04-25T19:15:20.804550+0000","last_undegraded":"2024-04-25T19:15:20.804550+0000","last_fullsized":"2024-04-25T19:15:20.804550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:05:28.723958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804304+0000","last_change":"2024-04-25T19:15:19.424805+0000","last_active":"2024-04-25T19:15:20.804304+0000","last_peered":"2024-04-25T19:15:20.804304+0000","last_clean":"2024-04-25T19:15:20.804304+0000","last_became_active":"2024-04-25T19:15:19.424512+0000","last_became_peered":"2024-04-25T19:15:19.424512+0000","last_unstale":"2024-04-25T19:15:20.804304+0000","last_undegraded":"2024-04-25T19:15:20.804304+0000","last_fullsized":"2024-04-25T19:15:20.804304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:50.423186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848213+0000","last_change":"2024-04-25T19:15:18.426805+0000","last_active":"2024-04-25T19:15:20.848213+0000","last_peered":"2024-04-25T19:15:20.848213+0000","last_clean":"2024-04-25T19:15:20.848213+0000","last_became_active":"2024-04-25T19:15:18.426472+0000","last_became_peered":"2024-04-25T19:15:18.426472+0000","last_unstale":"2024-04-25T19:15:20.848213+0000","last_undegraded":"2024-04-25T19:15:20.848213+0000","last_fullsized":"2024-04-25T19:15:20.848213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:29.373363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425459+0000","last_change":"2024-04-25T19:15:19.428780+0000","last_active":"2024-04-25T19:15:20.425459+0000","last_peered":"2024-04-25T19:15:20.425459+0000","last_clean":"2024-04-25T19:15:20.425459+0000","last_became_active":"2024-04-25T19:15:19.428450+0000","last_became_peered":"2024-04-25T19:15:19.428450+0000","last_unstale":"2024-04-25T19:15:20.425459+0000","last_undegraded":"2024-04-25T19:15:20.425459+0000","last_fullsized":"2024-04-25T19:15:20.425459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:01.203880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405399+0000","last_change":"2024-04-25T19:15:18.417740+0000","last_active":"2024-04-25T19:15:20.405399+0000","last_peered":"2024-04-25T19:15:20.405399+0000","last_clean":"2024-04-25T19:15:20.405399+0000","last_became_active":"2024-04-25T19:15:18.417404+0000","last_became_peered":"2024-04-25T19:15:18.417404+0000","last_unstale":"2024-04-25T19:15:20.405399+0000","last_undegraded":"2024-04-25T19:15:20.405399+0000","last_fullsized":"2024-04-25T19:15:20.405399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:21.737283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405391+0000","last_change":"2024-04-25T19:15:19.411214+0000","last_active":"2024-04-25T19:15:20.405391+0000","last_peered":"2024-04-25T19:15:20.405391+0000","last_clean":"2024-04-25T19:15:20.405391+0000","last_became_active":"2024-04-25T19:15:19.410814+0000","last_became_peered":"2024-04-25T19:15:19.410814+0000","last_unstale":"2024-04-25T19:15:20.405391+0000","last_undegraded":"2024-04-25T19:15:20.405391+0000","last_fullsized":"2024-04-25T19:15:20.405391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:42.430277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847063+0000","last_change":"2024-04-25T19:15:18.422226+0000","last_active":"2024-04-25T19:15:20.847063+0000","last_peered":"2024-04-25T19:15:20.847063+0000","last_clean":"2024-04-25T19:15:20.847063+0000","last_became_active":"2024-04-25T19:15:18.421900+0000","last_became_peered":"2024-04-25T19:15:18.421900+0000","last_unstale":"2024-04-25T19:15:20.847063+0000","last_undegraded":"2024-04-25T19:15:20.847063+0000","last_fullsized":"2024-04-25T19:15:20.847063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:18.040114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425347+0000","last_change":"2024-04-25T19:15:19.428885+0000","last_active":"2024-04-25T19:15:20.425347+0000","last_peered":"2024-04-25T19:15:20.425347+0000","last_clean":"2024-04-25T19:15:20.425347+0000","last_became_active":"2024-04-25T19:15:19.428563+0000","last_became_peered":"2024-04-25T19:15:19.428563+0000","last_unstale":"2024-04-25T19:15:20.425347+0000","last_undegraded":"2024-04-25T19:15:20.425347+0000","last_fullsized":"2024-04-25T19:15:20.425347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:43.469189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822767+0000","last_change":"2024-04-25T19:15:18.422125+0000","last_active":"2024-04-25T19:15:20.822767+0000","last_peered":"2024-04-25T19:15:20.822767+0000","last_clean":"2024-04-25T19:15:20.822767+0000","last_became_active":"2024-04-25T19:15:18.421777+0000","last_became_peered":"2024-04-25T19:15:18.421777+0000","last_unstale":"2024-04-25T19:15:20.822767+0000","last_undegraded":"2024-04-25T19:15:20.822767+0000","last_fullsized":"2024-04-25T19:15:20.822767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:24.289010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425489+0000","last_change":"2024-04-25T19:15:19.429675+0000","last_active":"2024-04-25T19:15:20.425489+0000","last_peered":"2024-04-25T19:15:20.425489+0000","last_clean":"2024-04-25T19:15:20.425489+0000","last_became_active":"2024-04-25T19:15:19.429316+0000","last_became_peered":"2024-04-25T19:15:19.429316+0000","last_unstale":"2024-04-25T19:15:20.425489+0000","last_undegraded":"2024-04-25T19:15:20.425489+0000","last_fullsized":"2024-04-25T19:15:20.425489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:09:55.521944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847203+0000","last_change":"2024-04-25T19:15:18.418661+0000","last_active":"2024-04-25T19:15:20.847203+0000","last_peered":"2024-04-25T19:15:20.847203+0000","last_clean":"2024-04-25T19:15:20.847203+0000","last_became_active":"2024-04-25T19:15:18.418359+0000","last_became_peered":"2024-04-25T19:15:18.418359+0000","last_unstale":"2024-04-25T19:15:20.847203+0000","last_undegraded":"2024-04-25T19:15:20.847203+0000","last_fullsized":"2024-04-25T19:15:20.847203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:49:22.023652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425687+0000","last_change":"2024-04-25T19:15:19.434853+0000","last_active":"2024-04-25T19:15:20.425687+0000","last_peered":"2024-04-25T19:15:20.425687+0000","last_clean":"2024-04-25T19:15:20.425687+0000","last_became_active":"2024-04-25T19:15:19.434508+0000","last_became_peered":"2024-04-25T19:15:19.434508+0000","last_unstale":"2024-04-25T19:15:20.425687+0000","last_undegraded":"2024-04-25T19:15:20.425687+0000","last_fullsized":"2024-04-25T19:15:20.425687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:52.965864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404798+0000","last_change":"2024-04-25T19:15:18.428626+0000","last_active":"2024-04-25T19:15:20.404798+0000","last_peered":"2024-04-25T19:15:20.404798+0000","last_clean":"2024-04-25T19:15:20.404798+0000","last_became_active":"2024-04-25T19:15:18.428332+0000","last_became_peered":"2024-04-25T19:15:18.428332+0000","last_unstale":"2024-04-25T19:15:20.404798+0000","last_undegraded":"2024-04-25T19:15:20.404798+0000","last_fullsized":"2024-04-25T19:15:20.404798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:15.595305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848535+0000","last_change":"2024-04-25T19:15:19.428754+0000","last_active":"2024-04-25T19:15:20.848535+0000","last_peered":"2024-04-25T19:15:20.848535+0000","last_clean":"2024-04-25T19:15:20.848535+0000","last_became_active":"2024-04-25T19:15:19.427992+0000","last_became_peered":"2024-04-25T19:15:19.427992+0000","last_unstale":"2024-04-25T19:15:20.848535+0000","last_undegraded":"2024-04-25T19:15:20.848535+0000","last_fullsized":"2024-04-25T19:15:20.848535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:43:48.443923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847234+0000","last_change":"2024-04-25T19:15:18.419645+0000","last_active":"2024-04-25T19:15:20.847234+0000","last_peered":"2024-04-25T19:15:20.847234+0000","last_clean":"2024-04-25T19:15:20.847234+0000","last_became_active":"2024-04-25T19:15:18.419286+0000","last_became_peered":"2024-04-25T19:15:18.419286+0000","last_unstale":"2024-04-25T19:15:20.847234+0000","last_undegraded":"2024-04-25T19:15:20.847234+0000","last_fullsized":"2024-04-25T19:15:20.847234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:08.295208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425261+0000","last_change":"2024-04-25T19:15:19.432395+0000","last_active":"2024-04-25T19:15:20.425261+0000","last_peered":"2024-04-25T19:15:20.425261+0000","last_clean":"2024-04-25T19:15:20.425261+0000","last_became_active":"2024-04-25T19:15:19.431739+0000","last_became_peered":"2024-04-25T19:15:19.431739+0000","last_unstale":"2024-04-25T19:15:20.425261+0000","last_undegraded":"2024-04-25T19:15:20.425261+0000","last_fullsized":"2024-04-25T19:15:20.425261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:50:03.964030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":110,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425087+0000","last_change":"2024-04-25T19:15:15.604825+0000","last_active":"2024-04-25T19:15:20.425087+0000","last_peered":"2024-04-25T19:15:20.425087+0000","last_clean":"2024-04-25T19:15:20.425087+0000","last_became_active":"2024-04-25T19:15:15.603730+0000","last_became_peered":"2024-04-25T19:15:15.603730+0000","last_unstale":"2024-04-25T19:15:20.425087+0000","last_undegraded":"2024-04-25T19:15:20.425087+0000","last_fullsized":"2024-04-25T19:15:20.425087+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:14.367890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:14.367890+0000","last_clean_scrub_stamp":"2024-04-25T19:15:14.367890+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:02.411899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847915+0000","last_change":"2024-04-25T19:15:18.426331+0000","last_active":"2024-04-25T19:15:20.847915+0000","last_peered":"2024-04-25T19:15:20.847915+0000","last_clean":"2024-04-25T19:15:20.847915+0000","last_became_active":"2024-04-25T19:15:18.425993+0000","last_became_peered":"2024-04-25T19:15:18.425993+0000","last_unstale":"2024-04-25T19:15:20.847915+0000","last_undegraded":"2024-04-25T19:15:20.847915+0000","last_fullsized":"2024-04-25T19:15:20.847915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:17.440484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405021+0000","last_change":"2024-04-25T19:15:19.417337+0000","last_active":"2024-04-25T19:15:20.405021+0000","last_peered":"2024-04-25T19:15:20.405021+0000","last_clean":"2024-04-25T19:15:20.405021+0000","last_became_active":"2024-04-25T19:15:19.416808+0000","last_became_peered":"2024-04-25T19:15:19.416808+0000","last_unstale":"2024-04-25T19:15:20.405021+0000","last_undegraded":"2024-04-25T19:15:20.405021+0000","last_fullsized":"2024-04-25T19:15:20.405021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:57.797904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.424926+0000","last_change":"2024-04-25T19:15:18.426358+0000","last_active":"2024-04-25T19:15:20.424926+0000","last_peered":"2024-04-25T19:15:20.424926+0000","last_clean":"2024-04-25T19:15:20.424926+0000","last_became_active":"2024-04-25T19:15:18.426037+0000","last_became_peered":"2024-04-25T19:15:18.426037+0000","last_unstale":"2024-04-25T19:15:20.424926+0000","last_undegraded":"2024-04-25T19:15:20.424926+0000","last_fullsized":"2024-04-25T19:15:20.424926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:26:28.945667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804206+0000","last_change":"2024-04-25T19:15:19.426219+0000","last_active":"2024-04-25T19:15:20.804206+0000","last_peered":"2024-04-25T19:15:20.804206+0000","last_clean":"2024-04-25T19:15:20.804206+0000","last_became_active":"2024-04-25T19:15:19.425626+0000","last_became_peered":"2024-04-25T19:15:19.425626+0000","last_unstale":"2024-04-25T19:15:20.804206+0000","last_undegraded":"2024-04-25T19:15:20.804206+0000","last_fullsized":"2024-04-25T19:15:20.804206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:58.372454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847232+0000","last_change":"2024-04-25T19:15:18.424560+0000","last_active":"2024-04-25T19:15:20.847232+0000","last_peered":"2024-04-25T19:15:20.847232+0000","last_clean":"2024-04-25T19:15:20.847232+0000","last_became_active":"2024-04-25T19:15:18.424179+0000","last_became_peered":"2024-04-25T19:15:18.424179+0000","last_unstale":"2024-04-25T19:15:20.847232+0000","last_undegraded":"2024-04-25T19:15:20.847232+0000","last_fullsized":"2024-04-25T19:15:20.847232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:06.184399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404401+0000","last_change":"2024-04-25T19:15:19.415219+0000","last_active":"2024-04-25T19:15:20.404401+0000","last_peered":"2024-04-25T19:15:20.404401+0000","last_clean":"2024-04-25T19:15:20.404401+0000","last_became_active":"2024-04-25T19:15:19.414877+0000","last_became_peered":"2024-04-25T19:15:19.414877+0000","last_unstale":"2024-04-25T19:15:20.404401+0000","last_undegraded":"2024-04-25T19:15:20.404401+0000","last_fullsized":"2024-04-25T19:15:20.404401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:15.063419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848341+0000","last_change":"2024-04-25T19:15:18.423402+0000","last_active":"2024-04-25T19:15:20.848341+0000","last_peered":"2024-04-25T19:15:20.848341+0000","last_clean":"2024-04-25T19:15:20.848341+0000","last_became_active":"2024-04-25T19:15:18.422330+0000","last_became_peered":"2024-04-25T19:15:18.422330+0000","last_unstale":"2024-04-25T19:15:20.848341+0000","last_undegraded":"2024-04-25T19:15:20.848341+0000","last_fullsized":"2024-04-25T19:15:20.848341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:39:17.512502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822708+0000","last_change":"2024-04-25T19:15:19.421096+0000","last_active":"2024-04-25T19:15:20.822708+0000","last_peered":"2024-04-25T19:15:20.822708+0000","last_clean":"2024-04-25T19:15:20.822708+0000","last_became_active":"2024-04-25T19:15:19.420598+0000","last_became_peered":"2024-04-25T19:15:19.420598+0000","last_unstale":"2024-04-25T19:15:20.822708+0000","last_undegraded":"2024-04-25T19:15:20.822708+0000","last_fullsized":"2024-04-25T19:15:20.822708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:20.733888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404629+0000","last_change":"2024-04-25T19:15:18.422979+0000","last_active":"2024-04-25T19:15:20.404629+0000","last_peered":"2024-04-25T19:15:20.404629+0000","last_clean":"2024-04-25T19:15:20.404629+0000","last_became_active":"2024-04-25T19:15:18.422621+0000","last_became_peered":"2024-04-25T19:15:18.422621+0000","last_unstale":"2024-04-25T19:15:20.404629+0000","last_undegraded":"2024-04-25T19:15:20.404629+0000","last_fullsized":"2024-04-25T19:15:20.404629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:47:56.160615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848468+0000","last_change":"2024-04-25T19:15:19.426379+0000","last_active":"2024-04-25T19:15:20.848468+0000","last_peered":"2024-04-25T19:15:20.848468+0000","last_clean":"2024-04-25T19:15:20.848468+0000","last_became_active":"2024-04-25T19:15:19.425997+0000","last_became_peered":"2024-04-25T19:15:19.425997+0000","last_unstale":"2024-04-25T19:15:20.848468+0000","last_undegraded":"2024-04-25T19:15:20.848468+0000","last_fullsized":"2024-04-25T19:15:20.848468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:32.972274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425571+0000","last_change":"2024-04-25T19:15:18.427748+0000","last_active":"2024-04-25T19:15:20.425571+0000","last_peered":"2024-04-25T19:15:20.425571+0000","last_clean":"2024-04-25T19:15:20.425571+0000","last_became_active":"2024-04-25T19:15:18.427374+0000","last_became_peered":"2024-04-25T19:15:18.427374+0000","last_unstale":"2024-04-25T19:15:20.425571+0000","last_undegraded":"2024-04-25T19:15:20.425571+0000","last_fullsized":"2024-04-25T19:15:20.425571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:47:41.469558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404710+0000","last_change":"2024-04-25T19:15:19.422196+0000","last_active":"2024-04-25T19:15:20.404710+0000","last_peered":"2024-04-25T19:15:20.404710+0000","last_clean":"2024-04-25T19:15:20.404710+0000","last_became_active":"2024-04-25T19:15:19.421835+0000","last_became_peered":"2024-04-25T19:15:19.421835+0000","last_unstale":"2024-04-25T19:15:20.404710+0000","last_undegraded":"2024-04-25T19:15:20.404710+0000","last_fullsized":"2024-04-25T19:15:20.404710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:38.500369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404858+0000","last_change":"2024-04-25T19:15:18.419086+0000","last_active":"2024-04-25T19:15:20.404858+0000","last_peered":"2024-04-25T19:15:20.404858+0000","last_clean":"2024-04-25T19:15:20.404858+0000","last_became_active":"2024-04-25T19:15:18.418775+0000","last_became_peered":"2024-04-25T19:15:18.418775+0000","last_unstale":"2024-04-25T19:15:20.404858+0000","last_undegraded":"2024-04-25T19:15:20.404858+0000","last_fullsized":"2024-04-25T19:15:20.404858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:20.074753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405244+0000","last_change":"2024-04-25T19:15:19.432155+0000","last_active":"2024-04-25T19:15:20.405244+0000","last_peered":"2024-04-25T19:15:20.405244+0000","last_clean":"2024-04-25T19:15:20.405244+0000","last_became_active":"2024-04-25T19:15:19.431905+0000","last_became_peered":"2024-04-25T19:15:19.431905+0000","last_unstale":"2024-04-25T19:15:20.405244+0000","last_undegraded":"2024-04-25T19:15:20.405244+0000","last_fullsized":"2024-04-25T19:15:20.405244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:52.752469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405624+0000","last_change":"2024-04-25T19:15:18.423144+0000","last_active":"2024-04-25T19:15:20.405624+0000","last_peered":"2024-04-25T19:15:20.405624+0000","last_clean":"2024-04-25T19:15:20.405624+0000","last_became_active":"2024-04-25T19:15:18.422838+0000","last_became_peered":"2024-04-25T19:15:18.422838+0000","last_unstale":"2024-04-25T19:15:20.405624+0000","last_undegraded":"2024-04-25T19:15:20.405624+0000","last_fullsized":"2024-04-25T19:15:20.405624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:28.026912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804520+0000","last_change":"2024-04-25T19:15:19.420582+0000","last_active":"2024-04-25T19:15:20.804520+0000","last_peered":"2024-04-25T19:15:20.804520+0000","last_clean":"2024-04-25T19:15:20.804520+0000","last_became_active":"2024-04-25T19:15:19.420270+0000","last_became_peered":"2024-04-25T19:15:19.420270+0000","last_unstale":"2024-04-25T19:15:20.804520+0000","last_undegraded":"2024-04-25T19:15:20.804520+0000","last_fullsized":"2024-04-25T19:15:20.804520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:06.827819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.424926+0000","last_change":"2024-04-25T19:15:18.425469+0000","last_active":"2024-04-25T19:15:20.424926+0000","last_peered":"2024-04-25T19:15:20.424926+0000","last_clean":"2024-04-25T19:15:20.424926+0000","last_became_active":"2024-04-25T19:15:18.425084+0000","last_became_peered":"2024-04-25T19:15:18.425084+0000","last_unstale":"2024-04-25T19:15:20.424926+0000","last_undegraded":"2024-04-25T19:15:20.424926+0000","last_fullsized":"2024-04-25T19:15:20.424926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:14.363158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822624+0000","last_change":"2024-04-25T19:15:19.425159+0000","last_active":"2024-04-25T19:15:20.822624+0000","last_peered":"2024-04-25T19:15:20.822624+0000","last_clean":"2024-04-25T19:15:20.822624+0000","last_became_active":"2024-04-25T19:15:19.424847+0000","last_became_peered":"2024-04-25T19:15:19.424847+0000","last_unstale":"2024-04-25T19:15:20.822624+0000","last_undegraded":"2024-04-25T19:15:20.822624+0000","last_fullsized":"2024-04-25T19:15:20.822624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:28.715286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404461+0000","last_change":"2024-04-25T19:15:18.424645+0000","last_active":"2024-04-25T19:15:20.404461+0000","last_peered":"2024-04-25T19:15:20.404461+0000","last_clean":"2024-04-25T19:15:20.404461+0000","last_became_active":"2024-04-25T19:15:18.424241+0000","last_became_peered":"2024-04-25T19:15:18.424241+0000","last_unstale":"2024-04-25T19:15:20.404461+0000","last_undegraded":"2024-04-25T19:15:20.404461+0000","last_fullsized":"2024-04-25T19:15:20.404461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:29.373363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405640+0000","last_change":"2024-04-25T19:15:19.421095+0000","last_active":"2024-04-25T19:15:20.405640+0000","last_peered":"2024-04-25T19:15:20.405640+0000","last_clean":"2024-04-25T19:15:20.405640+0000","last_became_active":"2024-04-25T19:15:19.420795+0000","last_became_peered":"2024-04-25T19:15:19.420795+0000","last_unstale":"2024-04-25T19:15:20.405640+0000","last_undegraded":"2024-04-25T19:15:20.405640+0000","last_fullsized":"2024-04-25T19:15:20.405640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:13.061506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404579+0000","last_change":"2024-04-25T19:15:18.423841+0000","last_active":"2024-04-25T19:15:20.404579+0000","last_peered":"2024-04-25T19:15:20.404579+0000","last_clean":"2024-04-25T19:15:20.404579+0000","last_became_active":"2024-04-25T19:15:18.423500+0000","last_became_peered":"2024-04-25T19:15:18.423500+0000","last_unstale":"2024-04-25T19:15:20.404579+0000","last_undegraded":"2024-04-25T19:15:20.404579+0000","last_fullsized":"2024-04-25T19:15:20.404579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:58.933085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405564+0000","last_change":"2024-04-25T19:15:19.416232+0000","last_active":"2024-04-25T19:15:20.405564+0000","last_peered":"2024-04-25T19:15:20.405564+0000","last_clean":"2024-04-25T19:15:20.405564+0000","last_became_active":"2024-04-25T19:15:19.415828+0000","last_became_peered":"2024-04-25T19:15:19.415828+0000","last_unstale":"2024-04-25T19:15:20.405564+0000","last_undegraded":"2024-04-25T19:15:20.405564+0000","last_fullsized":"2024-04-25T19:15:20.405564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:59.751840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425530+0000","last_change":"2024-04-25T19:15:18.423918+0000","last_active":"2024-04-25T19:15:20.425530+0000","last_peered":"2024-04-25T19:15:20.425530+0000","last_clean":"2024-04-25T19:15:20.425530+0000","last_became_active":"2024-04-25T19:15:18.423588+0000","last_became_peered":"2024-04-25T19:15:18.423588+0000","last_unstale":"2024-04-25T19:15:20.425530+0000","last_undegraded":"2024-04-25T19:15:20.425530+0000","last_fullsized":"2024-04-25T19:15:20.425530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:16.378285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405402+0000","last_change":"2024-04-25T19:15:19.422984+0000","last_active":"2024-04-25T19:15:20.405402+0000","last_peered":"2024-04-25T19:15:20.405402+0000","last_clean":"2024-04-25T19:15:20.405402+0000","last_became_active":"2024-04-25T19:15:19.422651+0000","last_became_peered":"2024-04-25T19:15:19.422651+0000","last_unstale":"2024-04-25T19:15:20.405402+0000","last_undegraded":"2024-04-25T19:15:20.405402+0000","last_fullsized":"2024-04-25T19:15:20.405402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:54.984914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848417+0000","last_change":"2024-04-25T19:15:19.423982+0000","last_active":"2024-04-25T19:15:20.848417+0000","last_peered":"2024-04-25T19:15:20.848417+0000","last_clean":"2024-04-25T19:15:20.848417+0000","last_became_active":"2024-04-25T19:15:19.423341+0000","last_became_peered":"2024-04-25T19:15:19.423341+0000","last_unstale":"2024-04-25T19:15:20.848417+0000","last_undegraded":"2024-04-25T19:15:20.848417+0000","last_fullsized":"2024-04-25T19:15:20.848417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:51.808352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425216+0000","last_change":"2024-04-25T19:15:18.424455+0000","last_active":"2024-04-25T19:15:20.425216+0000","last_peered":"2024-04-25T19:15:20.425216+0000","last_clean":"2024-04-25T19:15:20.425216+0000","last_became_active":"2024-04-25T19:15:18.424119+0000","last_became_peered":"2024-04-25T19:15:18.424119+0000","last_unstale":"2024-04-25T19:15:20.425216+0000","last_undegraded":"2024-04-25T19:15:20.425216+0000","last_fullsized":"2024-04-25T19:15:20.425216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:43.135567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848336+0000","last_change":"2024-04-25T19:15:19.424064+0000","last_active":"2024-04-25T19:15:20.848336+0000","last_peered":"2024-04-25T19:15:20.848336+0000","last_clean":"2024-04-25T19:15:20.848336+0000","last_became_active":"2024-04-25T19:15:19.423187+0000","last_became_peered":"2024-04-25T19:15:19.423187+0000","last_unstale":"2024-04-25T19:15:20.848336+0000","last_undegraded":"2024-04-25T19:15:20.848336+0000","last_fullsized":"2024-04-25T19:15:20.848336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:43:31.590700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425443+0000","last_change":"2024-04-25T19:15:18.423151+0000","last_active":"2024-04-25T19:15:20.425443+0000","last_peered":"2024-04-25T19:15:20.425443+0000","last_clean":"2024-04-25T19:15:20.425443+0000","last_became_active":"2024-04-25T19:15:18.422831+0000","last_became_peered":"2024-04-25T19:15:18.422831+0000","last_unstale":"2024-04-25T19:15:20.425443+0000","last_undegraded":"2024-04-25T19:15:20.425443+0000","last_fullsized":"2024-04-25T19:15:20.425443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:43.244963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804534+0000","last_change":"2024-04-25T19:15:19.416717+0000","last_active":"2024-04-25T19:15:20.804534+0000","last_peered":"2024-04-25T19:15:20.804534+0000","last_clean":"2024-04-25T19:15:20.804534+0000","last_became_active":"2024-04-25T19:15:19.416163+0000","last_became_peered":"2024-04-25T19:15:19.416163+0000","last_unstale":"2024-04-25T19:15:20.804534+0000","last_undegraded":"2024-04-25T19:15:20.804534+0000","last_fullsized":"2024-04-25T19:15:20.804534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:46.955159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405366+0000","last_change":"2024-04-25T19:15:18.422875+0000","last_active":"2024-04-25T19:15:20.405366+0000","last_peered":"2024-04-25T19:15:20.405366+0000","last_clean":"2024-04-25T19:15:20.405366+0000","last_became_active":"2024-04-25T19:15:18.422489+0000","last_became_peered":"2024-04-25T19:15:18.422489+0000","last_unstale":"2024-04-25T19:15:20.405366+0000","last_undegraded":"2024-04-25T19:15:20.405366+0000","last_fullsized":"2024-04-25T19:15:20.405366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:30.602500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405477+0000","last_change":"2024-04-25T19:15:19.416987+0000","last_active":"2024-04-25T19:15:20.405477+0000","last_peered":"2024-04-25T19:15:20.405477+0000","last_clean":"2024-04-25T19:15:20.405477+0000","last_became_active":"2024-04-25T19:15:19.414173+0000","last_became_peered":"2024-04-25T19:15:19.414173+0000","last_unstale":"2024-04-25T19:15:20.405477+0000","last_undegraded":"2024-04-25T19:15:20.405477+0000","last_fullsized":"2024-04-25T19:15:20.405477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:32:43.188653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804581+0000","last_change":"2024-04-25T19:15:18.423867+0000","last_active":"2024-04-25T19:15:20.804581+0000","last_peered":"2024-04-25T19:15:20.804581+0000","last_clean":"2024-04-25T19:15:20.804581+0000","last_became_active":"2024-04-25T19:15:18.423532+0000","last_became_peered":"2024-04-25T19:15:18.423532+0000","last_unstale":"2024-04-25T19:15:20.804581+0000","last_undegraded":"2024-04-25T19:15:20.804581+0000","last_fullsized":"2024-04-25T19:15:20.804581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:04.531338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64838,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":665973,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":665973,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T19:15:25.426 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T19:15:25.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.568+0000 7fb001184640 1 Processor -- start 2024-04-25T19:15:25.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.568+0000 7fb001184640 1 -- start start 2024-04-25T19:15:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.569+0000 7fb001184640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faffc069a50 0x7faffc06ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.569+0000 7fb001184640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faffc074540 0x7faffc156b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.569+0000 7fb001184640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7faffc072e40 0x7faffc073250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.569+0000 7fb001184640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7faffc15d580 con 0x7faffc069a50 2024-04-25T19:15:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.569+0000 7fb001184640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7faffc15d820 con 0x7faffc072e40 2024-04-25T19:15:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.569+0000 7fb001184640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7faffc15dac0 con 0x7faffc074540 2024-04-25T19:15:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.569+0000 7faffad9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faffc069a50 0x7faffc06ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.569+0000 7faffad9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faffc069a50 0x7faffc06ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59000/0 (socket says 172.21.15.76:59000) 2024-04-25T19:15:25.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.569+0000 7faffad9d640 1 -- 172.21.15.76:0/1502601550 learned_addr learned my addr 172.21.15.76:0/1502601550 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:25.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.570+0000 7faffb59e640 1 --2- 172.21.15.76:0/1502601550 >> v2:172.21.15.105:3300/0 conn(0x7faffc072e40 0x7faffc073250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:25.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.570+0000 7faff3fff640 1 --2- 172.21.15.76:0/1502601550 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faffc074540 0x7faffc156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:25.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.570+0000 7faffad9d640 1 -- 172.21.15.76:0/1502601550 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faffc074540 msgr2=0x7faffc156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:25.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.570+0000 7faffad9d640 1 --2- 172.21.15.76:0/1502601550 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faffc074540 0x7faffc156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.570+0000 7faffad9d640 1 -- 172.21.15.76:0/1502601550 >> v2:172.21.15.105:3300/0 conn(0x7faffc072e40 msgr2=0x7faffc073250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:25.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.570+0000 7faffad9d640 1 --2- 172.21.15.76:0/1502601550 >> v2:172.21.15.105:3300/0 conn(0x7faffc072e40 0x7faffc073250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.570+0000 7faffad9d640 1 -- 172.21.15.76:0/1502601550 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faffc0c4230 con 0x7faffc069a50 2024-04-25T19:15:25.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.570+0000 7faffad9d640 1 --2- 172.21.15.76:0/1502601550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faffc069a50 0x7faffc06ebc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faffc0cc1c0 tx=0x7fafec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19743e78d79255bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:25.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.571+0000 7faffa59c640 1 -- 172.21.15.76:0/1502601550 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec037070 con 0x7faffc069a50 2024-04-25T19:15:25.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.571+0000 7faffa59c640 1 -- 172.21.15.76:0/1502601550 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafec032440 con 0x7faffc069a50 2024-04-25T19:15:25.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.571+0000 7faffa59c640 1 -- 172.21.15.76:0/1502601550 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec032810 con 0x7faffc069a50 2024-04-25T19:15:25.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.571+0000 7fb001184640 1 -- 172.21.15.76:0/1502601550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faffc069a50 msgr2=0x7faffc06ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:25.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.571+0000 7fb001184640 1 --2- 172.21.15.76:0/1502601550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faffc069a50 0x7faffc06ebc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faffc0cc1c0 tx=0x7fafec02a270 comp rx=0 tx=0).stop 2024-04-25T19:15:25.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.571+0000 7fb001184640 1 -- 172.21.15.76:0/1502601550 shutdown_connections 2024-04-25T19:15:25.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.571+0000 7fb001184640 1 --2- 172.21.15.76:0/1502601550 >> v2:172.21.15.105:3300/0 conn(0x7faffc072e40 0x7faffc073250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.571+0000 7fb001184640 1 --2- 172.21.15.76:0/1502601550 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faffc074540 0x7faffc156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.571+0000 7fb001184640 1 --2- 172.21.15.76:0/1502601550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faffc069a50 0x7faffc06ebc0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.571+0000 7fb001184640 1 -- 172.21.15.76:0/1502601550 >> 172.21.15.76:0/1502601550 conn(0x7faffc0cae10 msgr2=0x7faffc0cb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:25.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.571+0000 7fb001184640 1 -- 172.21.15.76:0/1502601550 shutdown_connections 2024-04-25T19:15:25.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.571+0000 7fb001184640 1 -- 172.21.15.76:0/1502601550 wait complete. 2024-04-25T19:15:25.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.573+0000 7fb001184640 1 Processor -- start 2024-04-25T19:15:25.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.573+0000 7fb001184640 1 -- start start 2024-04-25T19:15:25.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.574+0000 7fb001184640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faffc069a50 0x7faffc1e3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:25.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.574+0000 7fb001184640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faffc072e40 0x7faffc0c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:25.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.574+0000 7fb001184640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faffc074540 0x7faffc0c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:25.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.574+0000 7fb001184640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faffc0c4ab0 con 0x7faffc072e40 2024-04-25T19:15:25.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.574+0000 7fb001184640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faffc0c47d0 con 0x7faffc069a50 2024-04-25T19:15:25.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.574+0000 7fb001184640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7faffc06f100 con 0x7faffc074540 2024-04-25T19:15:25.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.574+0000 7faff3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faffc072e40 0x7faffc0c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:25.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.575+0000 7faff3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faffc072e40 0x7faffc0c7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59010/0 (socket says 172.21.15.76:59010) 2024-04-25T19:15:25.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.575+0000 7faff3fff640 1 -- 172.21.15.76:0/2162189156 learned_addr learned my addr 172.21.15.76:0/2162189156 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:25.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.575+0000 7faffb59e640 1 --2- 172.21.15.76:0/2162189156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faffc074540 0x7faffc0c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:25.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.575+0000 7faffad9d640 1 --2- 172.21.15.76:0/2162189156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faffc069a50 0x7faffc1e3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:25.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.575+0000 7faff3fff640 1 -- 172.21.15.76:0/2162189156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faffc074540 msgr2=0x7faffc0c5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:25.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.575+0000 7faff3fff640 1 --2- 172.21.15.76:0/2162189156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faffc074540 0x7faffc0c5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.575+0000 7faff3fff640 1 -- 172.21.15.76:0/2162189156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faffc069a50 msgr2=0x7faffc1e3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:25.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.575+0000 7faff3fff640 1 --2- 172.21.15.76:0/2162189156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faffc069a50 0x7faffc1e3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.575+0000 7faff3fff640 1 -- 172.21.15.76:0/2162189156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faffc1e6b50 con 0x7faffc072e40 2024-04-25T19:15:25.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.575+0000 7faff3fff640 1 --2- 172.21.15.76:0/2162189156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faffc072e40 0x7faffc0c7ce0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fafe400d5f0 tx=0x7fafe400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:25.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.575+0000 7faff37fe640 1 -- 172.21.15.76:0/2162189156 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe40125f0 con 0x7faffc072e40 2024-04-25T19:15:25.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.575+0000 7faff37fe640 1 -- 172.21.15.76:0/2162189156 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafe4019070 con 0x7faffc072e40 2024-04-25T19:15:25.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.575+0000 7faff37fe640 1 -- 172.21.15.76:0/2162189156 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe40129d0 con 0x7faffc072e40 2024-04-25T19:15:25.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.575+0000 7fb001184640 1 -- 172.21.15.76:0/2162189156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faffc0c8220 con 0x7faffc072e40 2024-04-25T19:15:25.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.575+0000 7fb001184640 1 -- 172.21.15.76:0/2162189156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faffc0c51d0 con 0x7faffc072e40 2024-04-25T19:15:25.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.576+0000 7fb001184640 1 -- 172.21.15.76:0/2162189156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faffc06a390 con 0x7faffc072e40 2024-04-25T19:15:25.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.577+0000 7faff37fe640 1 -- 172.21.15.76:0/2162189156 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fafe4012b70 con 0x7faffc072e40 2024-04-25T19:15:25.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.577+0000 7faff37fe640 1 --2- 172.21.15.76:0/2162189156 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fafdc0744e0 0x7fafdc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:25.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.577+0000 7faff37fe640 1 -- 172.21.15.76:0/2162189156 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fafe40954a0 con 0x7faffc072e40 2024-04-25T19:15:25.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.578+0000 7faffad9d640 1 --2- 172.21.15.76:0/2162189156 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fafdc0744e0 0x7fafdc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:25.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.578+0000 7faffad9d640 1 --2- 172.21.15.76:0/2162189156 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fafdc0744e0 0x7fafdc0769f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fafec034000 tx=0x7fafec02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:25.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.582+0000 7faff37fe640 1 -- 172.21.15.76:0/2162189156 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fafe405fd50 con 0x7faffc072e40 2024-04-25T19:15:25.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.756+0000 7fb001184640 1 -- 172.21.15.76:0/2162189156 --> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faffc1f3550 con 0x7fafdc0744e0 2024-04-25T19:15:25.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.763+0000 7faff37fe640 1 -- 172.21.15.76:0/2162189156 <== mgr.4109 v2:172.21.15.105:6832/2585775531 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251914 (secure 0 0 0) 0x7faffc1f3550 con 0x7fafdc0744e0 2024-04-25T19:15:25.766 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:25.768 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-04-25T19:15:25.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.766+0000 7fb001184640 1 -- 172.21.15.76:0/2162189156 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fafdc0744e0 msgr2=0x7fafdc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:25.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.766+0000 7fb001184640 1 --2- 172.21.15.76:0/2162189156 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fafdc0744e0 0x7fafdc0769f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fafec034000 tx=0x7fafec02e000 comp rx=0 tx=0).stop 2024-04-25T19:15:25.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.766+0000 7fb001184640 1 -- 172.21.15.76:0/2162189156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faffc072e40 msgr2=0x7faffc0c7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:25.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.766+0000 7fb001184640 1 --2- 172.21.15.76:0/2162189156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faffc072e40 0x7faffc0c7ce0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fafe400d5f0 tx=0x7fafe400dab0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.766+0000 7fb001184640 1 -- 172.21.15.76:0/2162189156 shutdown_connections 2024-04-25T19:15:25.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.766+0000 7fb001184640 1 --2- 172.21.15.76:0/2162189156 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fafdc0744e0 0x7fafdc0769f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.766+0000 7fb001184640 1 --2- 172.21.15.76:0/2162189156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faffc074540 0x7faffc0c5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.766+0000 7fb001184640 1 --2- 172.21.15.76:0/2162189156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faffc072e40 0x7faffc0c7ce0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.766+0000 7fb001184640 1 --2- 172.21.15.76:0/2162189156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faffc069a50 0x7faffc1e3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.766+0000 7fb001184640 1 -- 172.21.15.76:0/2162189156 >> 172.21.15.76:0/2162189156 conn(0x7faffc0cae10 msgr2=0x7faffc0ccf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:25.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.766+0000 7fb001184640 1 -- 172.21.15.76:0/2162189156 shutdown_connections 2024-04-25T19:15:25.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.766+0000 7fb001184640 1 -- 172.21.15.76:0/2162189156 wait complete. 2024-04-25T19:15:25.778 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-25T19:15:23.829568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218008,"kb_used_data":1824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756712,"statfs":{"total":773094113280,"available":772870873088,"internally_reserved":0,"allocated":1867776,"data_stored":1785576,"data_compressed":64838,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.426469"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425589+0000","last_change":"2024-04-25T19:15:19.433569+0000","last_active":"2024-04-25T19:15:20.425589+0000","last_peered":"2024-04-25T19:15:20.425589+0000","last_clean":"2024-04-25T19:15:20.425589+0000","last_became_active":"2024-04-25T19:15:19.431332+0000","last_became_peered":"2024-04-25T19:15:19.431332+0000","last_unstale":"2024-04-25T19:15:20.425589+0000","last_undegraded":"2024-04-25T19:15:20.425589+0000","last_fullsized":"2024-04-25T19:15:20.425589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:57:01.213834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404926+0000","last_change":"2024-04-25T19:15:19.418695+0000","last_active":"2024-04-25T19:15:20.404926+0000","last_peered":"2024-04-25T19:15:20.404926+0000","last_clean":"2024-04-25T19:15:20.404926+0000","last_became_active":"2024-04-25T19:15:19.418235+0000","last_became_peered":"2024-04-25T19:15:19.418235+0000","last_unstale":"2024-04-25T19:15:20.404926+0000","last_undegraded":"2024-04-25T19:15:20.404926+0000","last_fullsized":"2024-04-25T19:15:20.404926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:30.516472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804491+0000","last_change":"2024-04-25T19:15:19.416983+0000","last_active":"2024-04-25T19:15:20.804491+0000","last_peered":"2024-04-25T19:15:20.804491+0000","last_clean":"2024-04-25T19:15:20.804491+0000","last_became_active":"2024-04-25T19:15:19.416443+0000","last_became_peered":"2024-04-25T19:15:19.416443+0000","last_unstale":"2024-04-25T19:15:20.804491+0000","last_undegraded":"2024-04-25T19:15:20.804491+0000","last_fullsized":"2024-04-25T19:15:20.804491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:02.794164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848308+0000","last_change":"2024-04-25T19:15:19.429058+0000","last_active":"2024-04-25T19:15:20.848308+0000","last_peered":"2024-04-25T19:15:20.848308+0000","last_clean":"2024-04-25T19:15:20.848308+0000","last_became_active":"2024-04-25T19:15:19.428419+0000","last_became_peered":"2024-04-25T19:15:19.428419+0000","last_unstale":"2024-04-25T19:15:20.848308+0000","last_undegraded":"2024-04-25T19:15:20.848308+0000","last_fullsized":"2024-04-25T19:15:20.848308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:23:56.817616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425402+0000","last_change":"2024-04-25T19:15:19.427615+0000","last_active":"2024-04-25T19:15:20.425402+0000","last_peered":"2024-04-25T19:15:20.425402+0000","last_clean":"2024-04-25T19:15:20.425402+0000","last_became_active":"2024-04-25T19:15:19.426968+0000","last_became_peered":"2024-04-25T19:15:19.426968+0000","last_unstale":"2024-04-25T19:15:20.425402+0000","last_undegraded":"2024-04-25T19:15:20.425402+0000","last_fullsized":"2024-04-25T19:15:20.425402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:42:00.686712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847989+0000","last_change":"2024-04-25T19:15:19.417602+0000","last_active":"2024-04-25T19:15:20.847989+0000","last_peered":"2024-04-25T19:15:20.847989+0000","last_clean":"2024-04-25T19:15:20.847989+0000","last_became_active":"2024-04-25T19:15:19.417292+0000","last_became_peered":"2024-04-25T19:15:19.417292+0000","last_unstale":"2024-04-25T19:15:20.847989+0000","last_undegraded":"2024-04-25T19:15:20.847989+0000","last_fullsized":"2024-04-25T19:15:20.847989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:37:43.583809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425114+0000","last_change":"2024-04-25T19:15:19.428180+0000","last_active":"2024-04-25T19:15:20.425114+0000","last_peered":"2024-04-25T19:15:20.425114+0000","last_clean":"2024-04-25T19:15:20.425114+0000","last_became_active":"2024-04-25T19:15:19.427852+0000","last_became_peered":"2024-04-25T19:15:19.427852+0000","last_unstale":"2024-04-25T19:15:20.425114+0000","last_undegraded":"2024-04-25T19:15:20.425114+0000","last_fullsized":"2024-04-25T19:15:20.425114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:51.300880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405446+0000","last_change":"2024-04-25T19:15:19.418253+0000","last_active":"2024-04-25T19:15:20.405446+0000","last_peered":"2024-04-25T19:15:20.405446+0000","last_clean":"2024-04-25T19:15:20.405446+0000","last_became_active":"2024-04-25T19:15:19.417183+0000","last_became_peered":"2024-04-25T19:15:19.417183+0000","last_unstale":"2024-04-25T19:15:20.405446+0000","last_undegraded":"2024-04-25T19:15:20.405446+0000","last_fullsized":"2024-04-25T19:15:20.405446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:05:13.181646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425578+0000","last_change":"2024-04-25T19:15:19.431671+0000","last_active":"2024-04-25T19:15:20.425578+0000","last_peered":"2024-04-25T19:15:20.425578+0000","last_clean":"2024-04-25T19:15:20.425578+0000","last_became_active":"2024-04-25T19:15:19.431240+0000","last_became_peered":"2024-04-25T19:15:19.431240+0000","last_unstale":"2024-04-25T19:15:20.425578+0000","last_undegraded":"2024-04-25T19:15:20.425578+0000","last_fullsized":"2024-04-25T19:15:20.425578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:22.759462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.406080+0000","last_change":"2024-04-25T19:15:19.416304+0000","last_active":"2024-04-25T19:15:20.406080+0000","last_peered":"2024-04-25T19:15:20.406080+0000","last_clean":"2024-04-25T19:15:20.406080+0000","last_became_active":"2024-04-25T19:15:19.415967+0000","last_became_peered":"2024-04-25T19:15:19.415967+0000","last_unstale":"2024-04-25T19:15:20.406080+0000","last_undegraded":"2024-04-25T19:15:20.406080+0000","last_fullsized":"2024-04-25T19:15:20.406080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:30:52.958731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405773+0000","last_change":"2024-04-25T19:15:19.417596+0000","last_active":"2024-04-25T19:15:20.405773+0000","last_peered":"2024-04-25T19:15:20.405773+0000","last_clean":"2024-04-25T19:15:20.405773+0000","last_became_active":"2024-04-25T19:15:19.417230+0000","last_became_peered":"2024-04-25T19:15:19.417230+0000","last_unstale":"2024-04-25T19:15:20.405773+0000","last_undegraded":"2024-04-25T19:15:20.405773+0000","last_fullsized":"2024-04-25T19:15:20.405773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:28:12.387651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822843+0000","last_change":"2024-04-25T19:15:19.420564+0000","last_active":"2024-04-25T19:15:20.822843+0000","last_peered":"2024-04-25T19:15:20.822843+0000","last_clean":"2024-04-25T19:15:20.822843+0000","last_became_active":"2024-04-25T19:15:19.420044+0000","last_became_peered":"2024-04-25T19:15:19.420044+0000","last_unstale":"2024-04-25T19:15:20.822843+0000","last_undegraded":"2024-04-25T19:15:20.822843+0000","last_fullsized":"2024-04-25T19:15:20.822843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:04.066030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405269+0000","last_change":"2024-04-25T19:15:19.415500+0000","last_active":"2024-04-25T19:15:20.405269+0000","last_peered":"2024-04-25T19:15:20.405269+0000","last_clean":"2024-04-25T19:15:20.405269+0000","last_became_active":"2024-04-25T19:15:19.414007+0000","last_became_peered":"2024-04-25T19:15:19.414007+0000","last_unstale":"2024-04-25T19:15:20.405269+0000","last_undegraded":"2024-04-25T19:15:20.405269+0000","last_fullsized":"2024-04-25T19:15:20.405269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:56:12.320555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804235+0000","last_change":"2024-04-25T19:15:19.423884+0000","last_active":"2024-04-25T19:15:20.804235+0000","last_peered":"2024-04-25T19:15:20.804235+0000","last_clean":"2024-04-25T19:15:20.804235+0000","last_became_active":"2024-04-25T19:15:19.422575+0000","last_became_peered":"2024-04-25T19:15:19.422575+0000","last_unstale":"2024-04-25T19:15:20.804235+0000","last_undegraded":"2024-04-25T19:15:20.804235+0000","last_fullsized":"2024-04-25T19:15:20.804235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:30.988950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404396+0000","last_change":"2024-04-25T19:15:19.424267+0000","last_active":"2024-04-25T19:15:20.404396+0000","last_peered":"2024-04-25T19:15:20.404396+0000","last_clean":"2024-04-25T19:15:20.404396+0000","last_became_active":"2024-04-25T19:15:19.422953+0000","last_became_peered":"2024-04-25T19:15:19.422953+0000","last_unstale":"2024-04-25T19:15:20.404396+0000","last_undegraded":"2024-04-25T19:15:20.404396+0000","last_fullsized":"2024-04-25T19:15:20.404396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:42.823233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847983+0000","last_change":"2024-04-25T19:15:19.420374+0000","last_active":"2024-04-25T19:15:20.847983+0000","last_peered":"2024-04-25T19:15:20.847983+0000","last_clean":"2024-04-25T19:15:20.847983+0000","last_became_active":"2024-04-25T19:15:19.420015+0000","last_became_peered":"2024-04-25T19:15:19.420015+0000","last_unstale":"2024-04-25T19:15:20.847983+0000","last_undegraded":"2024-04-25T19:15:20.847983+0000","last_fullsized":"2024-04-25T19:15:20.847983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:04:26.719359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848705+0000","last_change":"2024-04-25T19:15:19.434476+0000","last_active":"2024-04-25T19:15:20.848705+0000","last_peered":"2024-04-25T19:15:20.848705+0000","last_clean":"2024-04-25T19:15:20.848705+0000","last_became_active":"2024-04-25T19:15:19.433836+0000","last_became_peered":"2024-04-25T19:15:19.433836+0000","last_unstale":"2024-04-25T19:15:20.848705+0000","last_undegraded":"2024-04-25T19:15:20.848705+0000","last_fullsized":"2024-04-25T19:15:20.848705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:13.234829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847359+0000","last_change":"2024-04-25T19:15:19.415521+0000","last_active":"2024-04-25T19:15:20.847359+0000","last_peered":"2024-04-25T19:15:20.847359+0000","last_clean":"2024-04-25T19:15:20.847359+0000","last_became_active":"2024-04-25T19:15:19.415177+0000","last_became_peered":"2024-04-25T19:15:19.415177+0000","last_unstale":"2024-04-25T19:15:20.847359+0000","last_undegraded":"2024-04-25T19:15:20.847359+0000","last_fullsized":"2024-04-25T19:15:20.847359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:24.355619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425590+0000","last_change":"2024-04-25T19:15:19.430291+0000","last_active":"2024-04-25T19:15:20.425590+0000","last_peered":"2024-04-25T19:15:20.425590+0000","last_clean":"2024-04-25T19:15:20.425590+0000","last_became_active":"2024-04-25T19:15:19.427226+0000","last_became_peered":"2024-04-25T19:15:19.427226+0000","last_unstale":"2024-04-25T19:15:20.425590+0000","last_undegraded":"2024-04-25T19:15:20.425590+0000","last_fullsized":"2024-04-25T19:15:20.425590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:38.464211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405750+0000","last_change":"2024-04-25T19:15:19.421696+0000","last_active":"2024-04-25T19:15:20.405750+0000","last_peered":"2024-04-25T19:15:20.405750+0000","last_clean":"2024-04-25T19:15:20.405750+0000","last_became_active":"2024-04-25T19:15:19.421300+0000","last_became_peered":"2024-04-25T19:15:19.421300+0000","last_unstale":"2024-04-25T19:15:20.405750+0000","last_undegraded":"2024-04-25T19:15:20.405750+0000","last_fullsized":"2024-04-25T19:15:20.405750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:15.226398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804410+0000","last_change":"2024-04-25T19:15:19.427880+0000","last_active":"2024-04-25T19:15:20.804410+0000","last_peered":"2024-04-25T19:15:20.804410+0000","last_clean":"2024-04-25T19:15:20.804410+0000","last_became_active":"2024-04-25T19:15:19.427579+0000","last_became_peered":"2024-04-25T19:15:19.427579+0000","last_unstale":"2024-04-25T19:15:20.804410+0000","last_undegraded":"2024-04-25T19:15:20.804410+0000","last_fullsized":"2024-04-25T19:15:20.804410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:47:10.657715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848145+0000","last_change":"2024-04-25T19:15:19.430409+0000","last_active":"2024-04-25T19:15:20.848145+0000","last_peered":"2024-04-25T19:15:20.848145+0000","last_clean":"2024-04-25T19:15:20.848145+0000","last_became_active":"2024-04-25T19:15:19.428305+0000","last_became_peered":"2024-04-25T19:15:19.428305+0000","last_unstale":"2024-04-25T19:15:20.848145+0000","last_undegraded":"2024-04-25T19:15:20.848145+0000","last_fullsized":"2024-04-25T19:15:20.848145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:10:01.268963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405396+0000","last_change":"2024-04-25T19:15:19.424435+0000","last_active":"2024-04-25T19:15:20.405396+0000","last_peered":"2024-04-25T19:15:20.405396+0000","last_clean":"2024-04-25T19:15:20.405396+0000","last_became_active":"2024-04-25T19:15:19.422763+0000","last_became_peered":"2024-04-25T19:15:19.422763+0000","last_unstale":"2024-04-25T19:15:20.405396+0000","last_undegraded":"2024-04-25T19:15:20.405396+0000","last_fullsized":"2024-04-25T19:15:20.405396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:58.092412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848072+0000","last_change":"2024-04-25T19:15:19.420275+0000","last_active":"2024-04-25T19:15:20.848072+0000","last_peered":"2024-04-25T19:15:20.848072+0000","last_clean":"2024-04-25T19:15:20.848072+0000","last_became_active":"2024-04-25T19:15:19.419936+0000","last_became_peered":"2024-04-25T19:15:19.419936+0000","last_unstale":"2024-04-25T19:15:20.848072+0000","last_undegraded":"2024-04-25T19:15:20.848072+0000","last_fullsized":"2024-04-25T19:15:20.848072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:55.710620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405087+0000","last_change":"2024-04-25T19:15:19.423408+0000","last_active":"2024-04-25T19:15:20.405087+0000","last_peered":"2024-04-25T19:15:20.405087+0000","last_clean":"2024-04-25T19:15:20.405087+0000","last_became_active":"2024-04-25T19:15:19.423116+0000","last_became_peered":"2024-04-25T19:15:19.423116+0000","last_unstale":"2024-04-25T19:15:20.405087+0000","last_undegraded":"2024-04-25T19:15:20.405087+0000","last_fullsized":"2024-04-25T19:15:20.405087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:18:57.787013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.406015+0000","last_change":"2024-04-25T19:15:19.428826+0000","last_active":"2024-04-25T19:15:20.406015+0000","last_peered":"2024-04-25T19:15:20.406015+0000","last_clean":"2024-04-25T19:15:20.406015+0000","last_became_active":"2024-04-25T19:15:19.428474+0000","last_became_peered":"2024-04-25T19:15:19.428474+0000","last_unstale":"2024-04-25T19:15:20.406015+0000","last_undegraded":"2024-04-25T19:15:20.406015+0000","last_fullsized":"2024-04-25T19:15:20.406015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:19.974851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404739+0000","last_change":"2024-04-25T19:15:19.423353+0000","last_active":"2024-04-25T19:15:20.404739+0000","last_peered":"2024-04-25T19:15:20.404739+0000","last_clean":"2024-04-25T19:15:20.404739+0000","last_became_active":"2024-04-25T19:15:19.422795+0000","last_became_peered":"2024-04-25T19:15:19.422795+0000","last_unstale":"2024-04-25T19:15:20.404739+0000","last_undegraded":"2024-04-25T19:15:20.404739+0000","last_fullsized":"2024-04-25T19:15:20.404739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:34.119976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405076+0000","last_change":"2024-04-25T19:15:19.421809+0000","last_active":"2024-04-25T19:15:20.405076+0000","last_peered":"2024-04-25T19:15:20.405076+0000","last_clean":"2024-04-25T19:15:20.405076+0000","last_became_active":"2024-04-25T19:15:19.421515+0000","last_became_peered":"2024-04-25T19:15:19.421515+0000","last_unstale":"2024-04-25T19:15:20.405076+0000","last_undegraded":"2024-04-25T19:15:20.405076+0000","last_fullsized":"2024-04-25T19:15:20.405076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:22.603291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804477+0000","last_change":"2024-04-25T19:15:19.410183+0000","last_active":"2024-04-25T19:15:20.804477+0000","last_peered":"2024-04-25T19:15:20.804477+0000","last_clean":"2024-04-25T19:15:20.804477+0000","last_became_active":"2024-04-25T19:15:19.409857+0000","last_became_peered":"2024-04-25T19:15:19.409857+0000","last_unstale":"2024-04-25T19:15:20.804477+0000","last_undegraded":"2024-04-25T19:15:20.804477+0000","last_fullsized":"2024-04-25T19:15:20.804477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:30.180786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848239+0000","last_change":"2024-04-25T19:15:19.425524+0000","last_active":"2024-04-25T19:15:20.848239+0000","last_peered":"2024-04-25T19:15:20.848239+0000","last_clean":"2024-04-25T19:15:20.848239+0000","last_became_active":"2024-04-25T19:15:19.424578+0000","last_became_peered":"2024-04-25T19:15:19.424578+0000","last_unstale":"2024-04-25T19:15:20.848239+0000","last_undegraded":"2024-04-25T19:15:20.848239+0000","last_fullsized":"2024-04-25T19:15:20.848239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:02.103879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847243+0000","last_change":"2024-04-25T19:15:19.426720+0000","last_active":"2024-04-25T19:15:20.847243+0000","last_peered":"2024-04-25T19:15:20.847243+0000","last_clean":"2024-04-25T19:15:20.847243+0000","last_became_active":"2024-04-25T19:15:19.426139+0000","last_became_peered":"2024-04-25T19:15:19.426139+0000","last_unstale":"2024-04-25T19:15:20.847243+0000","last_undegraded":"2024-04-25T19:15:20.847243+0000","last_fullsized":"2024-04-25T19:15:20.847243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:31.608993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405390+0000","last_change":"2024-04-25T19:15:19.413805+0000","last_active":"2024-04-25T19:15:20.405390+0000","last_peered":"2024-04-25T19:15:20.405390+0000","last_clean":"2024-04-25T19:15:20.405390+0000","last_became_active":"2024-04-25T19:15:19.413455+0000","last_became_peered":"2024-04-25T19:15:19.413455+0000","last_unstale":"2024-04-25T19:15:20.405390+0000","last_undegraded":"2024-04-25T19:15:20.405390+0000","last_fullsized":"2024-04-25T19:15:20.405390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:05:45.008076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822709+0000","last_change":"2024-04-25T19:15:19.423770+0000","last_active":"2024-04-25T19:15:20.822709+0000","last_peered":"2024-04-25T19:15:20.822709+0000","last_clean":"2024-04-25T19:15:20.822709+0000","last_became_active":"2024-04-25T19:15:19.423406+0000","last_became_peered":"2024-04-25T19:15:19.423406+0000","last_unstale":"2024-04-25T19:15:20.822709+0000","last_undegraded":"2024-04-25T19:15:20.822709+0000","last_fullsized":"2024-04-25T19:15:20.822709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:41:23.669832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405343+0000","last_change":"2024-04-25T19:15:18.422035+0000","last_active":"2024-04-25T19:15:20.405343+0000","last_peered":"2024-04-25T19:15:20.405343+0000","last_clean":"2024-04-25T19:15:20.405343+0000","last_became_active":"2024-04-25T19:15:18.421658+0000","last_became_peered":"2024-04-25T19:15:18.421658+0000","last_unstale":"2024-04-25T19:15:20.405343+0000","last_undegraded":"2024-04-25T19:15:20.405343+0000","last_fullsized":"2024-04-25T19:15:20.405343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:23.349126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804571+0000","last_change":"2024-04-25T19:15:19.421805+0000","last_active":"2024-04-25T19:15:20.804571+0000","last_peered":"2024-04-25T19:15:20.804571+0000","last_clean":"2024-04-25T19:15:20.804571+0000","last_became_active":"2024-04-25T19:15:19.421497+0000","last_became_peered":"2024-04-25T19:15:19.421497+0000","last_unstale":"2024-04-25T19:15:20.804571+0000","last_undegraded":"2024-04-25T19:15:20.804571+0000","last_fullsized":"2024-04-25T19:15:20.804571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:29:29.949508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405485+0000","last_change":"2024-04-25T19:15:18.416894+0000","last_active":"2024-04-25T19:15:20.405485+0000","last_peered":"2024-04-25T19:15:20.405485+0000","last_clean":"2024-04-25T19:15:20.405485+0000","last_became_active":"2024-04-25T19:15:18.416539+0000","last_became_peered":"2024-04-25T19:15:18.416539+0000","last_unstale":"2024-04-25T19:15:20.405485+0000","last_undegraded":"2024-04-25T19:15:20.405485+0000","last_fullsized":"2024-04-25T19:15:20.405485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:30.388606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405329+0000","last_change":"2024-04-25T19:15:18.420129+0000","last_active":"2024-04-25T19:15:20.405329+0000","last_peered":"2024-04-25T19:15:20.405329+0000","last_clean":"2024-04-25T19:15:20.405329+0000","last_became_active":"2024-04-25T19:15:18.419732+0000","last_became_peered":"2024-04-25T19:15:18.419732+0000","last_unstale":"2024-04-25T19:15:20.405329+0000","last_undegraded":"2024-04-25T19:15:20.405329+0000","last_fullsized":"2024-04-25T19:15:20.405329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:41:48.673989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405615+0000","last_change":"2024-04-25T19:15:19.415655+0000","last_active":"2024-04-25T19:15:20.405615+0000","last_peered":"2024-04-25T19:15:20.405615+0000","last_clean":"2024-04-25T19:15:20.405615+0000","last_became_active":"2024-04-25T19:15:19.415287+0000","last_became_peered":"2024-04-25T19:15:19.415287+0000","last_unstale":"2024-04-25T19:15:20.405615+0000","last_undegraded":"2024-04-25T19:15:20.405615+0000","last_fullsized":"2024-04-25T19:15:20.405615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:49.155289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848076+0000","last_change":"2024-04-25T19:15:18.420711+0000","last_active":"2024-04-25T19:15:20.848076+0000","last_peered":"2024-04-25T19:15:20.848076+0000","last_clean":"2024-04-25T19:15:20.848076+0000","last_became_active":"2024-04-25T19:15:18.420362+0000","last_became_peered":"2024-04-25T19:15:18.420362+0000","last_unstale":"2024-04-25T19:15:20.848076+0000","last_undegraded":"2024-04-25T19:15:20.848076+0000","last_fullsized":"2024-04-25T19:15:20.848076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:48.245193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404375+0000","last_change":"2024-04-25T19:15:19.424356+0000","last_active":"2024-04-25T19:15:20.404375+0000","last_peered":"2024-04-25T19:15:20.404375+0000","last_clean":"2024-04-25T19:15:20.404375+0000","last_became_active":"2024-04-25T19:15:19.423556+0000","last_became_peered":"2024-04-25T19:15:19.423556+0000","last_unstale":"2024-04-25T19:15:20.404375+0000","last_undegraded":"2024-04-25T19:15:20.404375+0000","last_fullsized":"2024-04-25T19:15:20.404375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:19:35.427261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405514+0000","last_change":"2024-04-25T19:15:18.419250+0000","last_active":"2024-04-25T19:15:20.405514+0000","last_peered":"2024-04-25T19:15:20.405514+0000","last_clean":"2024-04-25T19:15:20.405514+0000","last_became_active":"2024-04-25T19:15:18.418927+0000","last_became_peered":"2024-04-25T19:15:18.418927+0000","last_unstale":"2024-04-25T19:15:20.405514+0000","last_undegraded":"2024-04-25T19:15:20.405514+0000","last_fullsized":"2024-04-25T19:15:20.405514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:26.012965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848606+0000","last_change":"2024-04-25T19:15:19.430117+0000","last_active":"2024-04-25T19:15:20.848606+0000","last_peered":"2024-04-25T19:15:20.848606+0000","last_clean":"2024-04-25T19:15:20.848606+0000","last_became_active":"2024-04-25T19:15:19.428985+0000","last_became_peered":"2024-04-25T19:15:19.428985+0000","last_unstale":"2024-04-25T19:15:20.848606+0000","last_undegraded":"2024-04-25T19:15:20.848606+0000","last_fullsized":"2024-04-25T19:15:20.848606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:36:29.069541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848490+0000","last_change":"2024-04-25T19:15:18.422931+0000","last_active":"2024-04-25T19:15:20.848490+0000","last_peered":"2024-04-25T19:15:20.848490+0000","last_clean":"2024-04-25T19:15:20.848490+0000","last_became_active":"2024-04-25T19:15:18.422610+0000","last_became_peered":"2024-04-25T19:15:18.422610+0000","last_unstale":"2024-04-25T19:15:20.848490+0000","last_undegraded":"2024-04-25T19:15:20.848490+0000","last_fullsized":"2024-04-25T19:15:20.848490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:18.489484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804598+0000","last_change":"2024-04-25T19:15:19.416109+0000","last_active":"2024-04-25T19:15:20.804598+0000","last_peered":"2024-04-25T19:15:20.804598+0000","last_clean":"2024-04-25T19:15:20.804598+0000","last_became_active":"2024-04-25T19:15:19.415734+0000","last_became_peered":"2024-04-25T19:15:19.415734+0000","last_unstale":"2024-04-25T19:15:20.804598+0000","last_undegraded":"2024-04-25T19:15:20.804598+0000","last_fullsized":"2024-04-25T19:15:20.804598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:57:31.406303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404631+0000","last_change":"2024-04-25T19:15:18.428677+0000","last_active":"2024-04-25T19:15:20.404631+0000","last_peered":"2024-04-25T19:15:20.404631+0000","last_clean":"2024-04-25T19:15:20.404631+0000","last_became_active":"2024-04-25T19:15:18.428432+0000","last_became_peered":"2024-04-25T19:15:18.428432+0000","last_unstale":"2024-04-25T19:15:20.404631+0000","last_undegraded":"2024-04-25T19:15:20.404631+0000","last_fullsized":"2024-04-25T19:15:20.404631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:38.478595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822814+0000","last_change":"2024-04-25T19:15:19.428245+0000","last_active":"2024-04-25T19:15:20.822814+0000","last_peered":"2024-04-25T19:15:20.822814+0000","last_clean":"2024-04-25T19:15:20.822814+0000","last_became_active":"2024-04-25T19:15:19.427941+0000","last_became_peered":"2024-04-25T19:15:19.427941+0000","last_unstale":"2024-04-25T19:15:20.822814+0000","last_undegraded":"2024-04-25T19:15:20.822814+0000","last_fullsized":"2024-04-25T19:15:20.822814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:09.947923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405735+0000","last_change":"2024-04-25T19:15:18.420881+0000","last_active":"2024-04-25T19:15:20.405735+0000","last_peered":"2024-04-25T19:15:20.405735+0000","last_clean":"2024-04-25T19:15:20.405735+0000","last_became_active":"2024-04-25T19:15:18.420303+0000","last_became_peered":"2024-04-25T19:15:18.420303+0000","last_unstale":"2024-04-25T19:15:20.405735+0000","last_undegraded":"2024-04-25T19:15:20.405735+0000","last_fullsized":"2024-04-25T19:15:20.405735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:39:24.356298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425356+0000","last_change":"2024-04-25T19:15:19.432735+0000","last_active":"2024-04-25T19:15:20.425356+0000","last_peered":"2024-04-25T19:15:20.425356+0000","last_clean":"2024-04-25T19:15:20.425356+0000","last_became_active":"2024-04-25T19:15:19.431828+0000","last_became_peered":"2024-04-25T19:15:19.431828+0000","last_unstale":"2024-04-25T19:15:20.425356+0000","last_undegraded":"2024-04-25T19:15:20.425356+0000","last_fullsized":"2024-04-25T19:15:20.425356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:20.189230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804550+0000","last_change":"2024-04-25T19:15:18.421008+0000","last_active":"2024-04-25T19:15:20.804550+0000","last_peered":"2024-04-25T19:15:20.804550+0000","last_clean":"2024-04-25T19:15:20.804550+0000","last_became_active":"2024-04-25T19:15:18.420402+0000","last_became_peered":"2024-04-25T19:15:18.420402+0000","last_unstale":"2024-04-25T19:15:20.804550+0000","last_undegraded":"2024-04-25T19:15:20.804550+0000","last_fullsized":"2024-04-25T19:15:20.804550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:05:28.723958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804304+0000","last_change":"2024-04-25T19:15:19.424805+0000","last_active":"2024-04-25T19:15:20.804304+0000","last_peered":"2024-04-25T19:15:20.804304+0000","last_clean":"2024-04-25T19:15:20.804304+0000","last_became_active":"2024-04-25T19:15:19.424512+0000","last_became_peered":"2024-04-25T19:15:19.424512+0000","last_unstale":"2024-04-25T19:15:20.804304+0000","last_undegraded":"2024-04-25T19:15:20.804304+0000","last_fullsized":"2024-04-25T19:15:20.804304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:50.423186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848213+0000","last_change":"2024-04-25T19:15:18.426805+0000","last_active":"2024-04-25T19:15:20.848213+0000","last_peered":"2024-04-25T19:15:20.848213+0000","last_clean":"2024-04-25T19:15:20.848213+0000","last_became_active":"2024-04-25T19:15:18.426472+0000","last_became_peered":"2024-04-25T19:15:18.426472+0000","last_unstale":"2024-04-25T19:15:20.848213+0000","last_undegraded":"2024-04-25T19:15:20.848213+0000","last_fullsized":"2024-04-25T19:15:20.848213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:29.373363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425459+0000","last_change":"2024-04-25T19:15:19.428780+0000","last_active":"2024-04-25T19:15:20.425459+0000","last_peered":"2024-04-25T19:15:20.425459+0000","last_clean":"2024-04-25T19:15:20.425459+0000","last_became_active":"2024-04-25T19:15:19.428450+0000","last_became_peered":"2024-04-25T19:15:19.428450+0000","last_unstale":"2024-04-25T19:15:20.425459+0000","last_undegraded":"2024-04-25T19:15:20.425459+0000","last_fullsized":"2024-04-25T19:15:20.425459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:01.203880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405399+0000","last_change":"2024-04-25T19:15:18.417740+0000","last_active":"2024-04-25T19:15:20.405399+0000","last_peered":"2024-04-25T19:15:20.405399+0000","last_clean":"2024-04-25T19:15:20.405399+0000","last_became_active":"2024-04-25T19:15:18.417404+0000","last_became_peered":"2024-04-25T19:15:18.417404+0000","last_unstale":"2024-04-25T19:15:20.405399+0000","last_undegraded":"2024-04-25T19:15:20.405399+0000","last_fullsized":"2024-04-25T19:15:20.405399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:21.737283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405391+0000","last_change":"2024-04-25T19:15:19.411214+0000","last_active":"2024-04-25T19:15:20.405391+0000","last_peered":"2024-04-25T19:15:20.405391+0000","last_clean":"2024-04-25T19:15:20.405391+0000","last_became_active":"2024-04-25T19:15:19.410814+0000","last_became_peered":"2024-04-25T19:15:19.410814+0000","last_unstale":"2024-04-25T19:15:20.405391+0000","last_undegraded":"2024-04-25T19:15:20.405391+0000","last_fullsized":"2024-04-25T19:15:20.405391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:42.430277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847063+0000","last_change":"2024-04-25T19:15:18.422226+0000","last_active":"2024-04-25T19:15:20.847063+0000","last_peered":"2024-04-25T19:15:20.847063+0000","last_clean":"2024-04-25T19:15:20.847063+0000","last_became_active":"2024-04-25T19:15:18.421900+0000","last_became_peered":"2024-04-25T19:15:18.421900+0000","last_unstale":"2024-04-25T19:15:20.847063+0000","last_undegraded":"2024-04-25T19:15:20.847063+0000","last_fullsized":"2024-04-25T19:15:20.847063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:18.040114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425347+0000","last_change":"2024-04-25T19:15:19.428885+0000","last_active":"2024-04-25T19:15:20.425347+0000","last_peered":"2024-04-25T19:15:20.425347+0000","last_clean":"2024-04-25T19:15:20.425347+0000","last_became_active":"2024-04-25T19:15:19.428563+0000","last_became_peered":"2024-04-25T19:15:19.428563+0000","last_unstale":"2024-04-25T19:15:20.425347+0000","last_undegraded":"2024-04-25T19:15:20.425347+0000","last_fullsized":"2024-04-25T19:15:20.425347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:43.469189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822767+0000","last_change":"2024-04-25T19:15:18.422125+0000","last_active":"2024-04-25T19:15:20.822767+0000","last_peered":"2024-04-25T19:15:20.822767+0000","last_clean":"2024-04-25T19:15:20.822767+0000","last_became_active":"2024-04-25T19:15:18.421777+0000","last_became_peered":"2024-04-25T19:15:18.421777+0000","last_unstale":"2024-04-25T19:15:20.822767+0000","last_undegraded":"2024-04-25T19:15:20.822767+0000","last_fullsized":"2024-04-25T19:15:20.822767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:24.289010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425489+0000","last_change":"2024-04-25T19:15:19.429675+0000","last_active":"2024-04-25T19:15:20.425489+0000","last_peered":"2024-04-25T19:15:20.425489+0000","last_clean":"2024-04-25T19:15:20.425489+0000","last_became_active":"2024-04-25T19:15:19.429316+0000","last_became_peered":"2024-04-25T19:15:19.429316+0000","last_unstale":"2024-04-25T19:15:20.425489+0000","last_undegraded":"2024-04-25T19:15:20.425489+0000","last_fullsized":"2024-04-25T19:15:20.425489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:09:55.521944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847203+0000","last_change":"2024-04-25T19:15:18.418661+0000","last_active":"2024-04-25T19:15:20.847203+0000","last_peered":"2024-04-25T19:15:20.847203+0000","last_clean":"2024-04-25T19:15:20.847203+0000","last_became_active":"2024-04-25T19:15:18.418359+0000","last_became_peered":"2024-04-25T19:15:18.418359+0000","last_unstale":"2024-04-25T19:15:20.847203+0000","last_undegraded":"2024-04-25T19:15:20.847203+0000","last_fullsized":"2024-04-25T19:15:20.847203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:49:22.023652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425687+0000","last_change":"2024-04-25T19:15:19.434853+0000","last_active":"2024-04-25T19:15:20.425687+0000","last_peered":"2024-04-25T19:15:20.425687+0000","last_clean":"2024-04-25T19:15:20.425687+0000","last_became_active":"2024-04-25T19:15:19.434508+0000","last_became_peered":"2024-04-25T19:15:19.434508+0000","last_unstale":"2024-04-25T19:15:20.425687+0000","last_undegraded":"2024-04-25T19:15:20.425687+0000","last_fullsized":"2024-04-25T19:15:20.425687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:52.965864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404798+0000","last_change":"2024-04-25T19:15:18.428626+0000","last_active":"2024-04-25T19:15:20.404798+0000","last_peered":"2024-04-25T19:15:20.404798+0000","last_clean":"2024-04-25T19:15:20.404798+0000","last_became_active":"2024-04-25T19:15:18.428332+0000","last_became_peered":"2024-04-25T19:15:18.428332+0000","last_unstale":"2024-04-25T19:15:20.404798+0000","last_undegraded":"2024-04-25T19:15:20.404798+0000","last_fullsized":"2024-04-25T19:15:20.404798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:15.595305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848535+0000","last_change":"2024-04-25T19:15:19.428754+0000","last_active":"2024-04-25T19:15:20.848535+0000","last_peered":"2024-04-25T19:15:20.848535+0000","last_clean":"2024-04-25T19:15:20.848535+0000","last_became_active":"2024-04-25T19:15:19.427992+0000","last_became_peered":"2024-04-25T19:15:19.427992+0000","last_unstale":"2024-04-25T19:15:20.848535+0000","last_undegraded":"2024-04-25T19:15:20.848535+0000","last_fullsized":"2024-04-25T19:15:20.848535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:43:48.443923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847234+0000","last_change":"2024-04-25T19:15:18.419645+0000","last_active":"2024-04-25T19:15:20.847234+0000","last_peered":"2024-04-25T19:15:20.847234+0000","last_clean":"2024-04-25T19:15:20.847234+0000","last_became_active":"2024-04-25T19:15:18.419286+0000","last_became_peered":"2024-04-25T19:15:18.419286+0000","last_unstale":"2024-04-25T19:15:20.847234+0000","last_undegraded":"2024-04-25T19:15:20.847234+0000","last_fullsized":"2024-04-25T19:15:20.847234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:08.295208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425261+0000","last_change":"2024-04-25T19:15:19.432395+0000","last_active":"2024-04-25T19:15:20.425261+0000","last_peered":"2024-04-25T19:15:20.425261+0000","last_clean":"2024-04-25T19:15:20.425261+0000","last_became_active":"2024-04-25T19:15:19.431739+0000","last_became_peered":"2024-04-25T19:15:19.431739+0000","last_unstale":"2024-04-25T19:15:20.425261+0000","last_undegraded":"2024-04-25T19:15:20.425261+0000","last_fullsized":"2024-04-25T19:15:20.425261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:50:03.964030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":110,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425087+0000","last_change":"2024-04-25T19:15:15.604825+0000","last_active":"2024-04-25T19:15:20.425087+0000","last_peered":"2024-04-25T19:15:20.425087+0000","last_clean":"2024-04-25T19:15:20.425087+0000","last_became_active":"2024-04-25T19:15:15.603730+0000","last_became_peered":"2024-04-25T19:15:15.603730+0000","last_unstale":"2024-04-25T19:15:20.425087+0000","last_undegraded":"2024-04-25T19:15:20.425087+0000","last_fullsized":"2024-04-25T19:15:20.425087+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:14.367890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:14.367890+0000","last_clean_scrub_stamp":"2024-04-25T19:15:14.367890+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:02.411899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847915+0000","last_change":"2024-04-25T19:15:18.426331+0000","last_active":"2024-04-25T19:15:20.847915+0000","last_peered":"2024-04-25T19:15:20.847915+0000","last_clean":"2024-04-25T19:15:20.847915+0000","last_became_active":"2024-04-25T19:15:18.425993+0000","last_became_peered":"2024-04-25T19:15:18.425993+0000","last_unstale":"2024-04-25T19:15:20.847915+0000","last_undegraded":"2024-04-25T19:15:20.847915+0000","last_fullsized":"2024-04-25T19:15:20.847915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:17.440484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405021+0000","last_change":"2024-04-25T19:15:19.417337+0000","last_active":"2024-04-25T19:15:20.405021+0000","last_peered":"2024-04-25T19:15:20.405021+0000","last_clean":"2024-04-25T19:15:20.405021+0000","last_became_active":"2024-04-25T19:15:19.416808+0000","last_became_peered":"2024-04-25T19:15:19.416808+0000","last_unstale":"2024-04-25T19:15:20.405021+0000","last_undegraded":"2024-04-25T19:15:20.405021+0000","last_fullsized":"2024-04-25T19:15:20.405021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:57.797904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.424926+0000","last_change":"2024-04-25T19:15:18.426358+0000","last_active":"2024-04-25T19:15:20.424926+0000","last_peered":"2024-04-25T19:15:20.424926+0000","last_clean":"2024-04-25T19:15:20.424926+0000","last_became_active":"2024-04-25T19:15:18.426037+0000","last_became_peered":"2024-04-25T19:15:18.426037+0000","last_unstale":"2024-04-25T19:15:20.424926+0000","last_undegraded":"2024-04-25T19:15:20.424926+0000","last_fullsized":"2024-04-25T19:15:20.424926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:26:28.945667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804206+0000","last_change":"2024-04-25T19:15:19.426219+0000","last_active":"2024-04-25T19:15:20.804206+0000","last_peered":"2024-04-25T19:15:20.804206+0000","last_clean":"2024-04-25T19:15:20.804206+0000","last_became_active":"2024-04-25T19:15:19.425626+0000","last_became_peered":"2024-04-25T19:15:19.425626+0000","last_unstale":"2024-04-25T19:15:20.804206+0000","last_undegraded":"2024-04-25T19:15:20.804206+0000","last_fullsized":"2024-04-25T19:15:20.804206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:58.372454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847232+0000","last_change":"2024-04-25T19:15:18.424560+0000","last_active":"2024-04-25T19:15:20.847232+0000","last_peered":"2024-04-25T19:15:20.847232+0000","last_clean":"2024-04-25T19:15:20.847232+0000","last_became_active":"2024-04-25T19:15:18.424179+0000","last_became_peered":"2024-04-25T19:15:18.424179+0000","last_unstale":"2024-04-25T19:15:20.847232+0000","last_undegraded":"2024-04-25T19:15:20.847232+0000","last_fullsized":"2024-04-25T19:15:20.847232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:06.184399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404401+0000","last_change":"2024-04-25T19:15:19.415219+0000","last_active":"2024-04-25T19:15:20.404401+0000","last_peered":"2024-04-25T19:15:20.404401+0000","last_clean":"2024-04-25T19:15:20.404401+0000","last_became_active":"2024-04-25T19:15:19.414877+0000","last_became_peered":"2024-04-25T19:15:19.414877+0000","last_unstale":"2024-04-25T19:15:20.404401+0000","last_undegraded":"2024-04-25T19:15:20.404401+0000","last_fullsized":"2024-04-25T19:15:20.404401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:15.063419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848341+0000","last_change":"2024-04-25T19:15:18.423402+0000","last_active":"2024-04-25T19:15:20.848341+0000","last_peered":"2024-04-25T19:15:20.848341+0000","last_clean":"2024-04-25T19:15:20.848341+0000","last_became_active":"2024-04-25T19:15:18.422330+0000","last_became_peered":"2024-04-25T19:15:18.422330+0000","last_unstale":"2024-04-25T19:15:20.848341+0000","last_undegraded":"2024-04-25T19:15:20.848341+0000","last_fullsized":"2024-04-25T19:15:20.848341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:39:17.512502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822708+0000","last_change":"2024-04-25T19:15:19.421096+0000","last_active":"2024-04-25T19:15:20.822708+0000","last_peered":"2024-04-25T19:15:20.822708+0000","last_clean":"2024-04-25T19:15:20.822708+0000","last_became_active":"2024-04-25T19:15:19.420598+0000","last_became_peered":"2024-04-25T19:15:19.420598+0000","last_unstale":"2024-04-25T19:15:20.822708+0000","last_undegraded":"2024-04-25T19:15:20.822708+0000","last_fullsized":"2024-04-25T19:15:20.822708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:20.733888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404629+0000","last_change":"2024-04-25T19:15:18.422979+0000","last_active":"2024-04-25T19:15:20.404629+0000","last_peered":"2024-04-25T19:15:20.404629+0000","last_clean":"2024-04-25T19:15:20.404629+0000","last_became_active":"2024-04-25T19:15:18.422621+0000","last_became_peered":"2024-04-25T19:15:18.422621+0000","last_unstale":"2024-04-25T19:15:20.404629+0000","last_undegraded":"2024-04-25T19:15:20.404629+0000","last_fullsized":"2024-04-25T19:15:20.404629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:47:56.160615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848468+0000","last_change":"2024-04-25T19:15:19.426379+0000","last_active":"2024-04-25T19:15:20.848468+0000","last_peered":"2024-04-25T19:15:20.848468+0000","last_clean":"2024-04-25T19:15:20.848468+0000","last_became_active":"2024-04-25T19:15:19.425997+0000","last_became_peered":"2024-04-25T19:15:19.425997+0000","last_unstale":"2024-04-25T19:15:20.848468+0000","last_undegraded":"2024-04-25T19:15:20.848468+0000","last_fullsized":"2024-04-25T19:15:20.848468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:32.972274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425571+0000","last_change":"2024-04-25T19:15:18.427748+0000","last_active":"2024-04-25T19:15:20.425571+0000","last_peered":"2024-04-25T19:15:20.425571+0000","last_clean":"2024-04-25T19:15:20.425571+0000","last_became_active":"2024-04-25T19:15:18.427374+0000","last_became_peered":"2024-04-25T19:15:18.427374+0000","last_unstale":"2024-04-25T19:15:20.425571+0000","last_undegraded":"2024-04-25T19:15:20.425571+0000","last_fullsized":"2024-04-25T19:15:20.425571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:47:41.469558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404710+0000","last_change":"2024-04-25T19:15:19.422196+0000","last_active":"2024-04-25T19:15:20.404710+0000","last_peered":"2024-04-25T19:15:20.404710+0000","last_clean":"2024-04-25T19:15:20.404710+0000","last_became_active":"2024-04-25T19:15:19.421835+0000","last_became_peered":"2024-04-25T19:15:19.421835+0000","last_unstale":"2024-04-25T19:15:20.404710+0000","last_undegraded":"2024-04-25T19:15:20.404710+0000","last_fullsized":"2024-04-25T19:15:20.404710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:38.500369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404858+0000","last_change":"2024-04-25T19:15:18.419086+0000","last_active":"2024-04-25T19:15:20.404858+0000","last_peered":"2024-04-25T19:15:20.404858+0000","last_clean":"2024-04-25T19:15:20.404858+0000","last_became_active":"2024-04-25T19:15:18.418775+0000","last_became_peered":"2024-04-25T19:15:18.418775+0000","last_unstale":"2024-04-25T19:15:20.404858+0000","last_undegraded":"2024-04-25T19:15:20.404858+0000","last_fullsized":"2024-04-25T19:15:20.404858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:20.074753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405244+0000","last_change":"2024-04-25T19:15:19.432155+0000","last_active":"2024-04-25T19:15:20.405244+0000","last_peered":"2024-04-25T19:15:20.405244+0000","last_clean":"2024-04-25T19:15:20.405244+0000","last_became_active":"2024-04-25T19:15:19.431905+0000","last_became_peered":"2024-04-25T19:15:19.431905+0000","last_unstale":"2024-04-25T19:15:20.405244+0000","last_undegraded":"2024-04-25T19:15:20.405244+0000","last_fullsized":"2024-04-25T19:15:20.405244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:52.752469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405624+0000","last_change":"2024-04-25T19:15:18.423144+0000","last_active":"2024-04-25T19:15:20.405624+0000","last_peered":"2024-04-25T19:15:20.405624+0000","last_clean":"2024-04-25T19:15:20.405624+0000","last_became_active":"2024-04-25T19:15:18.422838+0000","last_became_peered":"2024-04-25T19:15:18.422838+0000","last_unstale":"2024-04-25T19:15:20.405624+0000","last_undegraded":"2024-04-25T19:15:20.405624+0000","last_fullsized":"2024-04-25T19:15:20.405624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:28.026912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804520+0000","last_change":"2024-04-25T19:15:19.420582+0000","last_active":"2024-04-25T19:15:20.804520+0000","last_peered":"2024-04-25T19:15:20.804520+0000","last_clean":"2024-04-25T19:15:20.804520+0000","last_became_active":"2024-04-25T19:15:19.420270+0000","last_became_peered":"2024-04-25T19:15:19.420270+0000","last_unstale":"2024-04-25T19:15:20.804520+0000","last_undegraded":"2024-04-25T19:15:20.804520+0000","last_fullsized":"2024-04-25T19:15:20.804520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:06.827819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.424926+0000","last_change":"2024-04-25T19:15:18.425469+0000","last_active":"2024-04-25T19:15:20.424926+0000","last_peered":"2024-04-25T19:15:20.424926+0000","last_clean":"2024-04-25T19:15:20.424926+0000","last_became_active":"2024-04-25T19:15:18.425084+0000","last_became_peered":"2024-04-25T19:15:18.425084+0000","last_unstale":"2024-04-25T19:15:20.424926+0000","last_undegraded":"2024-04-25T19:15:20.424926+0000","last_fullsized":"2024-04-25T19:15:20.424926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:14.363158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822624+0000","last_change":"2024-04-25T19:15:19.425159+0000","last_active":"2024-04-25T19:15:20.822624+0000","last_peered":"2024-04-25T19:15:20.822624+0000","last_clean":"2024-04-25T19:15:20.822624+0000","last_became_active":"2024-04-25T19:15:19.424847+0000","last_became_peered":"2024-04-25T19:15:19.424847+0000","last_unstale":"2024-04-25T19:15:20.822624+0000","last_undegraded":"2024-04-25T19:15:20.822624+0000","last_fullsized":"2024-04-25T19:15:20.822624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:28.715286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404461+0000","last_change":"2024-04-25T19:15:18.424645+0000","last_active":"2024-04-25T19:15:20.404461+0000","last_peered":"2024-04-25T19:15:20.404461+0000","last_clean":"2024-04-25T19:15:20.404461+0000","last_became_active":"2024-04-25T19:15:18.424241+0000","last_became_peered":"2024-04-25T19:15:18.424241+0000","last_unstale":"2024-04-25T19:15:20.404461+0000","last_undegraded":"2024-04-25T19:15:20.404461+0000","last_fullsized":"2024-04-25T19:15:20.404461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:29.373363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405640+0000","last_change":"2024-04-25T19:15:19.421095+0000","last_active":"2024-04-25T19:15:20.405640+0000","last_peered":"2024-04-25T19:15:20.405640+0000","last_clean":"2024-04-25T19:15:20.405640+0000","last_became_active":"2024-04-25T19:15:19.420795+0000","last_became_peered":"2024-04-25T19:15:19.420795+0000","last_unstale":"2024-04-25T19:15:20.405640+0000","last_undegraded":"2024-04-25T19:15:20.405640+0000","last_fullsized":"2024-04-25T19:15:20.405640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:13.061506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404579+0000","last_change":"2024-04-25T19:15:18.423841+0000","last_active":"2024-04-25T19:15:20.404579+0000","last_peered":"2024-04-25T19:15:20.404579+0000","last_clean":"2024-04-25T19:15:20.404579+0000","last_became_active":"2024-04-25T19:15:18.423500+0000","last_became_peered":"2024-04-25T19:15:18.423500+0000","last_unstale":"2024-04-25T19:15:20.404579+0000","last_undegraded":"2024-04-25T19:15:20.404579+0000","last_fullsized":"2024-04-25T19:15:20.404579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:58.933085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405564+0000","last_change":"2024-04-25T19:15:19.416232+0000","last_active":"2024-04-25T19:15:20.405564+0000","last_peered":"2024-04-25T19:15:20.405564+0000","last_clean":"2024-04-25T19:15:20.405564+0000","last_became_active":"2024-04-25T19:15:19.415828+0000","last_became_peered":"2024-04-25T19:15:19.415828+0000","last_unstale":"2024-04-25T19:15:20.405564+0000","last_undegraded":"2024-04-25T19:15:20.405564+0000","last_fullsized":"2024-04-25T19:15:20.405564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:59.751840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425530+0000","last_change":"2024-04-25T19:15:18.423918+0000","last_active":"2024-04-25T19:15:20.425530+0000","last_peered":"2024-04-25T19:15:20.425530+0000","last_clean":"2024-04-25T19:15:20.425530+0000","last_became_active":"2024-04-25T19:15:18.423588+0000","last_became_peered":"2024-04-25T19:15:18.423588+0000","last_unstale":"2024-04-25T19:15:20.425530+0000","last_undegraded":"2024-04-25T19:15:20.425530+0000","last_fullsized":"2024-04-25T19:15:20.425530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:16.378285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405402+0000","last_change":"2024-04-25T19:15:19.422984+0000","last_active":"2024-04-25T19:15:20.405402+0000","last_peered":"2024-04-25T19:15:20.405402+0000","last_clean":"2024-04-25T19:15:20.405402+0000","last_became_active":"2024-04-25T19:15:19.422651+0000","last_became_peered":"2024-04-25T19:15:19.422651+0000","last_unstale":"2024-04-25T19:15:20.405402+0000","last_undegraded":"2024-04-25T19:15:20.405402+0000","last_fullsized":"2024-04-25T19:15:20.405402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:54.984914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848417+0000","last_change":"2024-04-25T19:15:19.423982+0000","last_active":"2024-04-25T19:15:20.848417+0000","last_peered":"2024-04-25T19:15:20.848417+0000","last_clean":"2024-04-25T19:15:20.848417+0000","last_became_active":"2024-04-25T19:15:19.423341+0000","last_became_peered":"2024-04-25T19:15:19.423341+0000","last_unstale":"2024-04-25T19:15:20.848417+0000","last_undegraded":"2024-04-25T19:15:20.848417+0000","last_fullsized":"2024-04-25T19:15:20.848417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:51.808352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425216+0000","last_change":"2024-04-25T19:15:18.424455+0000","last_active":"2024-04-25T19:15:20.425216+0000","last_peered":"2024-04-25T19:15:20.425216+0000","last_clean":"2024-04-25T19:15:20.425216+0000","last_became_active":"2024-04-25T19:15:18.424119+0000","last_became_peered":"2024-04-25T19:15:18.424119+0000","last_unstale":"2024-04-25T19:15:20.425216+0000","last_undegraded":"2024-04-25T19:15:20.425216+0000","last_fullsized":"2024-04-25T19:15:20.425216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:43.135567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848336+0000","last_change":"2024-04-25T19:15:19.424064+0000","last_active":"2024-04-25T19:15:20.848336+0000","last_peered":"2024-04-25T19:15:20.848336+0000","last_clean":"2024-04-25T19:15:20.848336+0000","last_became_active":"2024-04-25T19:15:19.423187+0000","last_became_peered":"2024-04-25T19:15:19.423187+0000","last_unstale":"2024-04-25T19:15:20.848336+0000","last_undegraded":"2024-04-25T19:15:20.848336+0000","last_fullsized":"2024-04-25T19:15:20.848336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:43:31.590700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425443+0000","last_change":"2024-04-25T19:15:18.423151+0000","last_active":"2024-04-25T19:15:20.425443+0000","last_peered":"2024-04-25T19:15:20.425443+0000","last_clean":"2024-04-25T19:15:20.425443+0000","last_became_active":"2024-04-25T19:15:18.422831+0000","last_became_peered":"2024-04-25T19:15:18.422831+0000","last_unstale":"2024-04-25T19:15:20.425443+0000","last_undegraded":"2024-04-25T19:15:20.425443+0000","last_fullsized":"2024-04-25T19:15:20.425443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:43.244963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804534+0000","last_change":"2024-04-25T19:15:19.416717+0000","last_active":"2024-04-25T19:15:20.804534+0000","last_peered":"2024-04-25T19:15:20.804534+0000","last_clean":"2024-04-25T19:15:20.804534+0000","last_became_active":"2024-04-25T19:15:19.416163+0000","last_became_peered":"2024-04-25T19:15:19.416163+0000","last_unstale":"2024-04-25T19:15:20.804534+0000","last_undegraded":"2024-04-25T19:15:20.804534+0000","last_fullsized":"2024-04-25T19:15:20.804534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:46.955159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405366+0000","last_change":"2024-04-25T19:15:18.422875+0000","last_active":"2024-04-25T19:15:20.405366+0000","last_peered":"2024-04-25T19:15:20.405366+0000","last_clean":"2024-04-25T19:15:20.405366+0000","last_became_active":"2024-04-25T19:15:18.422489+0000","last_became_peered":"2024-04-25T19:15:18.422489+0000","last_unstale":"2024-04-25T19:15:20.405366+0000","last_undegraded":"2024-04-25T19:15:20.405366+0000","last_fullsized":"2024-04-25T19:15:20.405366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:30.602500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405477+0000","last_change":"2024-04-25T19:15:19.416987+0000","last_active":"2024-04-25T19:15:20.405477+0000","last_peered":"2024-04-25T19:15:20.405477+0000","last_clean":"2024-04-25T19:15:20.405477+0000","last_became_active":"2024-04-25T19:15:19.414173+0000","last_became_peered":"2024-04-25T19:15:19.414173+0000","last_unstale":"2024-04-25T19:15:20.405477+0000","last_undegraded":"2024-04-25T19:15:20.405477+0000","last_fullsized":"2024-04-25T19:15:20.405477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:32:43.188653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804581+0000","last_change":"2024-04-25T19:15:18.423867+0000","last_active":"2024-04-25T19:15:20.804581+0000","last_peered":"2024-04-25T19:15:20.804581+0000","last_clean":"2024-04-25T19:15:20.804581+0000","last_became_active":"2024-04-25T19:15:18.423532+0000","last_became_peered":"2024-04-25T19:15:18.423532+0000","last_unstale":"2024-04-25T19:15:20.804581+0000","last_undegraded":"2024-04-25T19:15:20.804581+0000","last_fullsized":"2024-04-25T19:15:20.804581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:04.531338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64838,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":665973,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":665973,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T19:15:25.782 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T19:15:25.782 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T19:15:25.782 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T19:15:25.782 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T19:15:25.783 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T19:15:25.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.925+0000 7f893e70d640 1 Processor -- start 2024-04-25T19:15:25.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.925+0000 7f893e70d640 1 -- start start 2024-04-25T19:15:25.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.926+0000 7f893e70d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8938158060 0x7f89380c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:25.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.926+0000 7f893e70d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89380cab60 0x7f893815af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:25.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.926+0000 7f893e70d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89380c56a0 con 0x7f89380ca750 2024-04-25T19:15:25.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.926+0000 7f893e70d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f89380c5960 con 0x7f8938158060 2024-04-25T19:15:25.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.926+0000 7f893e70d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f89380c5c40 con 0x7f89380cab60 2024-04-25T19:15:25.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.926+0000 7f8937fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8938158060 0x7f89380c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:25.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.926+0000 7f8937fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8938158060 0x7f89380c4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59018/0 (socket says 172.21.15.76:59018) 2024-04-25T19:15:25.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.926+0000 7f8937fff640 1 -- 172.21.15.76:0/3495433953 learned_addr learned my addr 172.21.15.76:0/3495433953 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:25.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.926+0000 7f893ccaa640 1 --2- 172.21.15.76:0/3495433953 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89380cab60 0x7f893815af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:25.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.926+0000 7f8937fff640 1 -- 172.21.15.76:0/3495433953 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89380cab60 msgr2=0x7f893815af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:25.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.926+0000 7f8937fff640 1 --2- 172.21.15.76:0/3495433953 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89380cab60 0x7f893815af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.926+0000 7f8937fff640 1 -- 172.21.15.76:0/3495433953 >> v1:172.21.15.105:6789/0 conn(0x7f89380ca750 legacy=0x7f89380cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:25.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.926+0000 7f8937fff640 1 -- 172.21.15.76:0/3495433953 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89380c5f20 con 0x7f8938158060 2024-04-25T19:15:25.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.926+0000 7f893ccaa640 1 --2- 172.21.15.76:0/3495433953 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89380cab60 0x7f893815af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:15:25.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.927+0000 7f8937fff640 1 --2- 172.21.15.76:0/3495433953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8938158060 0x7f89380c4de0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f89380d0e10 tx=0x7f892802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1cd97eeda6b17a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:25.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.927+0000 7f8936ffd640 1 -- 172.21.15.76:0/3495433953 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928037070 con 0x7f8938158060 2024-04-25T19:15:25.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.927+0000 7f8936ffd640 1 -- 172.21.15.76:0/3495433953 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8928032440 con 0x7f8938158060 2024-04-25T19:15:25.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.927+0000 7f8936ffd640 1 -- 172.21.15.76:0/3495433953 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928032810 con 0x7f8938158060 2024-04-25T19:15:25.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.928+0000 7f893e70d640 1 -- 172.21.15.76:0/3495433953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8938158060 msgr2=0x7f89380c4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:25.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.928+0000 7f893e70d640 1 --2- 172.21.15.76:0/3495433953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8938158060 0x7f89380c4de0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f89380d0e10 tx=0x7f892802a270 comp rx=0 tx=0).stop 2024-04-25T19:15:25.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.928+0000 7f893e70d640 1 -- 172.21.15.76:0/3495433953 shutdown_connections 2024-04-25T19:15:25.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.928+0000 7f893e70d640 1 --2- 172.21.15.76:0/3495433953 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89380cab60 0x7f893815af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.928+0000 7f893e70d640 1 --2- 172.21.15.76:0/3495433953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8938158060 0x7f89380c4de0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.928+0000 7f893e70d640 1 -- 172.21.15.76:0/3495433953 >> 172.21.15.76:0/3495433953 conn(0x7f8938068b10 msgr2=0x7f8938072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:25.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.928+0000 7f893e70d640 1 -- 172.21.15.76:0/3495433953 shutdown_connections 2024-04-25T19:15:25.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.928+0000 7f893e70d640 1 -- 172.21.15.76:0/3495433953 wait complete. 2024-04-25T19:15:25.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.930+0000 7f893e70d640 1 Processor -- start 2024-04-25T19:15:25.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.930+0000 7f893e70d640 1 -- start start 2024-04-25T19:15:25.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.931+0000 7f893e70d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89380ca750 0x7f893815e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:25.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.931+0000 7f893e70d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89380cab60 0x7f8938153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:25.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.931+0000 7f893e70d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8938158060 0x7f8938151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:25.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.931+0000 7f893e70d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f89380c5320 con 0x7f89380ca750 2024-04-25T19:15:25.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.931+0000 7f893e70d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f89380c6200 con 0x7f8938158060 2024-04-25T19:15:25.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.931+0000 7f893e70d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f89380ccef0 con 0x7f89380cab60 2024-04-25T19:15:25.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.931+0000 7f8937fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89380ca750 0x7f893815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:25.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.931+0000 7f8937fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89380ca750 0x7f893815e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59026/0 (socket says 172.21.15.76:59026) 2024-04-25T19:15:25.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.931+0000 7f8937fff640 1 -- 172.21.15.76:0/2530462612 learned_addr learned my addr 172.21.15.76:0/2530462612 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:25.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.931+0000 7f893ccaa640 1 --2- 172.21.15.76:0/2530462612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8938158060 0x7f8938151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:25.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.931+0000 7f89377fe640 1 --2- 172.21.15.76:0/2530462612 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89380cab60 0x7f8938153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:25.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.932+0000 7f8937fff640 1 -- 172.21.15.76:0/2530462612 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89380cab60 msgr2=0x7f8938153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:25.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.932+0000 7f8937fff640 1 --2- 172.21.15.76:0/2530462612 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89380cab60 0x7f8938153360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.932+0000 7f8937fff640 1 -- 172.21.15.76:0/2530462612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8938158060 msgr2=0x7f8938151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:25.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.932+0000 7f8937fff640 1 --2- 172.21.15.76:0/2530462612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8938158060 0x7f8938151610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:25.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.932+0000 7f8937fff640 1 -- 172.21.15.76:0/2530462612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893815ea70 con 0x7f89380ca750 2024-04-25T19:15:25.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.932+0000 7f8937fff640 1 --2- 172.21.15.76:0/2530462612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89380ca750 0x7f893815e530 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8928000c00 tx=0x7f89280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:25.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.932+0000 7f8934ff9640 1 -- 172.21.15.76:0/2530462612 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928041070 con 0x7f89380ca750 2024-04-25T19:15:25.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.932+0000 7f8934ff9640 1 -- 172.21.15.76:0/2530462612 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f892803c870 con 0x7f89380ca750 2024-04-25T19:15:25.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.932+0000 7f8934ff9640 1 -- 172.21.15.76:0/2530462612 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928037040 con 0x7f89380ca750 2024-04-25T19:15:25.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.932+0000 7f893e70d640 1 -- 172.21.15.76:0/2530462612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89380cfcf0 con 0x7f89380ca750 2024-04-25T19:15:25.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.932+0000 7f893e70d640 1 -- 172.21.15.76:0/2530462612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f893806d990 con 0x7f89380ca750 2024-04-25T19:15:25.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.933+0000 7f893e70d640 1 -- 172.21.15.76:0/2530462612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f893806a390 con 0x7f89380ca750 2024-04-25T19:15:25.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.934+0000 7f8934ff9640 1 -- 172.21.15.76:0/2530462612 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8928044070 con 0x7f89380ca750 2024-04-25T19:15:25.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.934+0000 7f8934ff9640 1 --2- 172.21.15.76:0/2530462612 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f89240744b0 0x7f89240769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:25.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.934+0000 7f8934ff9640 1 -- 172.21.15.76:0/2530462612 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f89280b5cf0 con 0x7f89380ca750 2024-04-25T19:15:25.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.934+0000 7f89377fe640 1 --2- 172.21.15.76:0/2530462612 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f89240744b0 0x7f89240769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:25.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.935+0000 7f89377fe640 1 --2- 172.21.15.76:0/2530462612 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f89240744b0 0x7f89240769c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f892c006fd0 tx=0x7f892c009060 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:25.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:25.939+0000 7f8934ff9640 1 -- 172.21.15.76:0/2530462612 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f89280ba070 con 0x7f89380ca750 2024-04-25T19:15:26.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.161+0000 7f893e70d640 1 -- 172.21.15.76:0/2530462612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f89380cb1d0 con 0x7f89380ca750 2024-04-25T19:15:26.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.164+0000 7f8934ff9640 1 -- 172.21.15.76:0/2530462612 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+186749 (secure 0 0 0) 0x7f892807ff50 con 0x7f89380ca750 2024-04-25T19:15:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:26.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.167+0000 7f893e70d640 1 -- 172.21.15.76:0/2530462612 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f89240744b0 msgr2=0x7f89240769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:26.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.167+0000 7f893e70d640 1 --2- 172.21.15.76:0/2530462612 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f89240744b0 0x7f89240769c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f892c006fd0 tx=0x7f892c009060 comp rx=0 tx=0).stop 2024-04-25T19:15:26.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.167+0000 7f893e70d640 1 -- 172.21.15.76:0/2530462612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89380ca750 msgr2=0x7f893815e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:26.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.167+0000 7f893e70d640 1 --2- 172.21.15.76:0/2530462612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89380ca750 0x7f893815e530 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8928000c00 tx=0x7f89280046c0 comp rx=0 tx=0).stop 2024-04-25T19:15:26.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.167+0000 7f893e70d640 1 -- 172.21.15.76:0/2530462612 shutdown_connections 2024-04-25T19:15:26.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.167+0000 7f893e70d640 1 --2- 172.21.15.76:0/2530462612 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f89240744b0 0x7f89240769c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:26.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.167+0000 7f893e70d640 1 --2- 172.21.15.76:0/2530462612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8938158060 0x7f8938151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:26.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.167+0000 7f893e70d640 1 --2- 172.21.15.76:0/2530462612 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89380cab60 0x7f8938153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:26.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.167+0000 7f893e70d640 1 --2- 172.21.15.76:0/2530462612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89380ca750 0x7f893815e530 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:26.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.167+0000 7f893e70d640 1 -- 172.21.15.76:0/2530462612 >> 172.21.15.76:0/2530462612 conn(0x7f8938068b10 msgr2=0x7f8938154de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:26.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.167+0000 7f893e70d640 1 -- 172.21.15.76:0/2530462612 shutdown_connections 2024-04-25T19:15:26.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.167+0000 7f893e70d640 1 -- 172.21.15.76:0/2530462612 wait complete. 2024-04-25T19:15:26.179 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":5,"flags":0,"active_gid":4109,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":2585775531},{"type":"v1","addr":"172.21.15.105:6833","nonce":2585775531}]},"active_addr":"172.21.15.105:6833/2585775531","active_change":"2024-04-25T19:15:05.817568+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":1382159989}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":1903303414}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":164591954}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":2108424884}]}]} 2024-04-25T19:15:26.182 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T19:15:26.183 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T19:15:26.183 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:15:26.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.289+0000 7f955ac30640 1 Processor -- start 2024-04-25T19:15:26.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.289+0000 7f955ac30640 1 -- start start 2024-04-25T19:15:26.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.290+0000 7f955ac30640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9554157200 0x7f95540d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:26.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.290+0000 7f955ac30640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f95540cfd50 0x7f95540cee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:26.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.290+0000 7f955ac30640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f95540d0160 0x7f95540cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:26.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.290+0000 7f955ac30640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f955415dd00 con 0x7f95540cfd50 2024-04-25T19:15:26.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.290+0000 7f955ac30640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f955415cea0 con 0x7f95540d0160 2024-04-25T19:15:26.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.290+0000 7f955ac30640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f955415d180 con 0x7f9554157200 2024-04-25T19:15:26.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.290+0000 7f9553fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f95540cfd50 0x7f95540cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:26.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.290+0000 7f9553fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f95540cfd50 0x7f95540cee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59038/0 (socket says 172.21.15.76:59038) 2024-04-25T19:15:26.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.290+0000 7f9553fff640 1 -- 172.21.15.76:0/3411652530 learned_addr learned my addr 172.21.15.76:0/3411652530 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:26.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.290+0000 7f95591cd640 1 --2- 172.21.15.76:0/3411652530 >> v2:172.21.15.105:3300/0 conn(0x7f95540d0160 0x7f95540cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:26.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.290+0000 7f95589cc640 1 --2- 172.21.15.76:0/3411652530 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9554157200 0x7f95540d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:26.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.290+0000 7f9553fff640 1 -- 172.21.15.76:0/3411652530 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9554157200 msgr2=0x7f95540d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:26.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.290+0000 7f9553fff640 1 --2- 172.21.15.76:0/3411652530 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9554157200 0x7f95540d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:26.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.290+0000 7f9553fff640 1 -- 172.21.15.76:0/3411652530 >> v2:172.21.15.105:3300/0 conn(0x7f95540d0160 msgr2=0x7f95540cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:26.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.290+0000 7f9553fff640 1 --2- 172.21.15.76:0/3411652530 >> v2:172.21.15.105:3300/0 conn(0x7f95540d0160 0x7f95540cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:26.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.290+0000 7f9553fff640 1 -- 172.21.15.76:0/3411652530 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955415d460 con 0x7f95540cfd50 2024-04-25T19:15:26.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.290+0000 7f95591cd640 1 --2- 172.21.15.76:0/3411652530 >> v2:172.21.15.105:3300/0 conn(0x7f95540d0160 0x7f95540cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:26.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.291+0000 7f9553fff640 1 --2- 172.21.15.76:0/3411652530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95540cfd50 0x7f95540cee90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f95540cc530 tx=0x7f954802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4543434370de49a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:26.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.291+0000 7f95537fe640 1 -- 172.21.15.76:0/3411652530 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9548037070 con 0x7f95540cfd50 2024-04-25T19:15:26.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.291+0000 7f95537fe640 1 -- 172.21.15.76:0/3411652530 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9548032440 con 0x7f95540cfd50 2024-04-25T19:15:26.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.291+0000 7f95537fe640 1 -- 172.21.15.76:0/3411652530 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9548032810 con 0x7f95540cfd50 2024-04-25T19:15:26.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.292+0000 7f955ac30640 1 -- 172.21.15.76:0/3411652530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95540cfd50 msgr2=0x7f95540cee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:26.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.292+0000 7f955ac30640 1 --2- 172.21.15.76:0/3411652530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95540cfd50 0x7f95540cee90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f95540cc530 tx=0x7f954802a270 comp rx=0 tx=0).stop 2024-04-25T19:15:26.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.292+0000 7f955ac30640 1 -- 172.21.15.76:0/3411652530 shutdown_connections 2024-04-25T19:15:26.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.292+0000 7f955ac30640 1 --2- 172.21.15.76:0/3411652530 >> v2:172.21.15.105:3300/0 conn(0x7f95540d0160 0x7f95540cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:26.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.292+0000 7f955ac30640 1 --2- 172.21.15.76:0/3411652530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95540cfd50 0x7f95540cee90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:26.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.292+0000 7f955ac30640 1 --2- 172.21.15.76:0/3411652530 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9554157200 0x7f95540d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:26.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.292+0000 7f955ac30640 1 -- 172.21.15.76:0/3411652530 >> 172.21.15.76:0/3411652530 conn(0x7f95540d2370 msgr2=0x7f95540d2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:26.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.292+0000 7f955ac30640 1 -- 172.21.15.76:0/3411652530 shutdown_connections 2024-04-25T19:15:26.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.292+0000 7f955ac30640 1 -- 172.21.15.76:0/3411652530 wait complete. 2024-04-25T19:15:26.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.294+0000 7f955ac30640 1 Processor -- start 2024-04-25T19:15:26.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.294+0000 7f955ac30640 1 -- start start 2024-04-25T19:15:26.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.295+0000 7f955ac30640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f95540cfd50 0x7f95540b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:26.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.295+0000 7f955ac30640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95540d0160 0x7f95541f6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:26.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.295+0000 7f955ac30640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9554157200 0x7f95540d0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:26.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.295+0000 7f955ac30640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f95540ccb70 con 0x7f95540d0160 2024-04-25T19:15:26.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.295+0000 7f955ac30640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f955415da20 con 0x7f9554157200 2024-04-25T19:15:26.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.295+0000 7f955ac30640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f95540cccf0 con 0x7f95540cfd50 2024-04-25T19:15:26.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.295+0000 7f9553fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95540d0160 0x7f95541f6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:26.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.295+0000 7f9553fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95540d0160 0x7f95541f6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59046/0 (socket says 172.21.15.76:59046) 2024-04-25T19:15:26.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.295+0000 7f9553fff640 1 -- 172.21.15.76:0/1796352297 learned_addr learned my addr 172.21.15.76:0/1796352297 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:26.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.295+0000 7f95589cc640 1 --2- 172.21.15.76:0/1796352297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f95540cfd50 0x7f95540b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:26.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.295+0000 7f95591cd640 1 --2- 172.21.15.76:0/1796352297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9554157200 0x7f95540d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:26.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.295+0000 7f9553fff640 1 -- 172.21.15.76:0/1796352297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f95540cfd50 msgr2=0x7f95540b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:26.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.295+0000 7f9553fff640 1 --2- 172.21.15.76:0/1796352297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f95540cfd50 0x7f95540b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:26.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.295+0000 7f9553fff640 1 -- 172.21.15.76:0/1796352297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9554157200 msgr2=0x7f95540d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:26.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.295+0000 7f9553fff640 1 --2- 172.21.15.76:0/1796352297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9554157200 0x7f95540d0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:26.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.295+0000 7f9553fff640 1 -- 172.21.15.76:0/1796352297 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95540b4560 con 0x7f95540d0160 2024-04-25T19:15:26.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.296+0000 7f9553fff640 1 --2- 172.21.15.76:0/1796352297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95540d0160 0x7f95541f6e40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f954802a2b0 tx=0x7f95480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:26.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.296+0000 7f95517fa640 1 -- 172.21.15.76:0/1796352297 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9548041020 con 0x7f95540d0160 2024-04-25T19:15:26.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.296+0000 7f95517fa640 1 -- 172.21.15.76:0/1796352297 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95480306c0 con 0x7f95540d0160 2024-04-25T19:15:26.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.296+0000 7f955ac30640 1 -- 172.21.15.76:0/1796352297 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95540b2410 con 0x7f95540d0160 2024-04-25T19:15:26.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.296+0000 7f95517fa640 1 -- 172.21.15.76:0/1796352297 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9548037040 con 0x7f95540d0160 2024-04-25T19:15:26.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.296+0000 7f955ac30640 1 -- 172.21.15.76:0/1796352297 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95540b4800 con 0x7f95540d0160 2024-04-25T19:15:26.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.297+0000 7f955ac30640 1 -- 172.21.15.76:0/1796352297 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95540cf700 con 0x7f95540d0160 2024-04-25T19:15:26.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.297+0000 7f95517fa640 1 -- 172.21.15.76:0/1796352297 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9548044020 con 0x7f95540d0160 2024-04-25T19:15:26.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.298+0000 7f95517fa640 1 --2- 172.21.15.76:0/1796352297 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f95400744e0 0x7f95400769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:26.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.298+0000 7f95517fa640 1 -- 172.21.15.76:0/1796352297 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f95480b59c0 con 0x7f95540d0160 2024-04-25T19:15:26.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.298+0000 7f95589cc640 1 --2- 172.21.15.76:0/1796352297 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f95400744e0 0x7f95400769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:26.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.298+0000 7f95589cc640 1 --2- 172.21.15.76:0/1796352297 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f95400744e0 0x7f95400769f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f95440079a0 tx=0x7f9544008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:26.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.302+0000 7f95517fa640 1 -- 172.21.15.76:0/1796352297 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f95540cf700 con 0x7f95540d0160 2024-04-25T19:15:26.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.485+0000 7f955ac30640 1 -- 172.21.15.76:0/1796352297 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9554077e30 con 0x7f95540d0160 2024-04-25T19:15:26.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.486+0000 7f95517fa640 1 -- 172.21.15.76:0/1796352297 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7f9554077e30 con 0x7f95540d0160 2024-04-25T19:15:26.488 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:26.488 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":21,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","created":"2024-04-25T19:15:01.693137+0000","modified":"2024-04-25T19:15:20.401211+0000","last_up_change":"2024-04-25T19:15:11.891681+0000","last_in_change":"2024-04-25T19:15:03.898065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:15:13.846514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:15:16.606533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:15:17.719197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72a16c02-48f9-45aa-9447-fcf755ebf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409574998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409574998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409574998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409574998}]},"public_addr":"172.21.15.76:6817/1409574998","cluster_addr":"172.21.15.76:6819/1409574998","heartbeat_back_addr":"172.21.15.76:6823/1409574998","heartbeat_front_addr":"172.21.15.76:6821/1409574998","state":["exists","up"]},{"osd":1,"uuid":"818d50f0-76c6-45f8-bf79-5e304f14c5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6801","nonce":1302663821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6803","nonce":1302663821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6807","nonce":1302663821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6805","nonce":1302663821}]},"public_addr":"172.21.15.76:6801/1302663821","cluster_addr":"172.21.15.76:6803/1302663821","heartbeat_back_addr":"172.21.15.76:6807/1302663821","heartbeat_front_addr":"172.21.15.76:6805/1302663821","state":["exists","up"]},{"osd":2,"uuid":"5714c8f5-ae23-473c-8cf1-db73811acded","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6809","nonce":1992938181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6811","nonce":1992938181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6815","nonce":1992938181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6813","nonce":1992938181}]},"public_addr":"172.21.15.76:6809/1992938181","cluster_addr":"172.21.15.76:6811/1992938181","heartbeat_back_addr":"172.21.15.76:6815/1992938181","heartbeat_front_addr":"172.21.15.76:6813/1992938181","state":["exists","up"]},{"osd":3,"uuid":"13204cdb-d8e6-4f97-bc6e-9561f442decf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6825","nonce":3418257163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6827","nonce":3418257163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6831","nonce":3418257163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6829","nonce":3418257163}]},"public_addr":"172.21.15.76:6825/3418257163","cluster_addr":"172.21.15.76:6827/3418257163","heartbeat_back_addr":"172.21.15.76:6831/3418257163","heartbeat_front_addr":"172.21.15.76:6829/3418257163","state":["exists","up"]},{"osd":4,"uuid":"5bea5166-1986-4d56-b5f5-ecc8cbe7f3ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6801","nonce":2125002468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6803","nonce":2125002468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6807","nonce":2125002468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6805","nonce":2125002468}]},"public_addr":"172.21.15.105:6801/2125002468","cluster_addr":"172.21.15.105:6803/2125002468","heartbeat_back_addr":"172.21.15.105:6807/2125002468","heartbeat_front_addr":"172.21.15.105:6805/2125002468","state":["exists","up"]},{"osd":5,"uuid":"96f89dcc-270d-4421-b8e8-3b92ba5f9844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6819","nonce":3413103121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6821","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6823","nonce":3413103121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6831","nonce":3413103121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6828","nonce":3413103121}]},"public_addr":"172.21.15.105:6819/3413103121","cluster_addr":"172.21.15.105:6823/3413103121","heartbeat_back_addr":"172.21.15.105:6831/3413103121","heartbeat_front_addr":"172.21.15.105:6828/3413103121","state":["exists","up"]},{"osd":6,"uuid":"93eb087a-d9ba-4096-b85d-b74d96e819a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6809","nonce":2721832954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6811","nonce":2721832954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6815","nonce":2721832954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6813","nonce":2721832954}]},"public_addr":"172.21.15.105:6809/2721832954","cluster_addr":"172.21.15.105:6811/2721832954","heartbeat_back_addr":"172.21.15.105:6815/2721832954","heartbeat_front_addr":"172.21.15.105:6813/2721832954","state":["exists","up"]},{"osd":7,"uuid":"2f80f790-ad2d-460e-a37d-47bf1771ef43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6818","nonce":548350430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6822","nonce":548350430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6829","nonce":548350430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6825","nonce":548350430}]},"public_addr":"172.21.15.105:6818/548350430","cluster_addr":"172.21.15.105:6822/548350430","heartbeat_back_addr":"172.21.15.105:6829/548350430","heartbeat_front_addr":"172.21.15.105:6825/548350430","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.106901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.072819+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.126865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.053464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:08.982614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.104825+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.112714+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.107387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:15:26.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.488+0000 7f955ac30640 1 -- 172.21.15.76:0/1796352297 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f95400744e0 msgr2=0x7f95400769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:26.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.488+0000 7f955ac30640 1 --2- 172.21.15.76:0/1796352297 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f95400744e0 0x7f95400769f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f95440079a0 tx=0x7f9544008040 comp rx=0 tx=0).stop 2024-04-25T19:15:26.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.488+0000 7f955ac30640 1 -- 172.21.15.76:0/1796352297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95540d0160 msgr2=0x7f95541f6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:26.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.488+0000 7f955ac30640 1 --2- 172.21.15.76:0/1796352297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95540d0160 0x7f95541f6e40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f954802a2b0 tx=0x7f95480046c0 comp rx=0 tx=0).stop 2024-04-25T19:15:26.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.488+0000 7f955ac30640 1 -- 172.21.15.76:0/1796352297 shutdown_connections 2024-04-25T19:15:26.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.488+0000 7f955ac30640 1 --2- 172.21.15.76:0/1796352297 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f95400744e0 0x7f95400769f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:26.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.488+0000 7f955ac30640 1 --2- 172.21.15.76:0/1796352297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9554157200 0x7f95540d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:26.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.488+0000 7f955ac30640 1 --2- 172.21.15.76:0/1796352297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95540d0160 0x7f95541f6e40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:26.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.488+0000 7f955ac30640 1 --2- 172.21.15.76:0/1796352297 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f95540cfd50 0x7f95540b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:26.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.488+0000 7f955ac30640 1 -- 172.21.15.76:0/1796352297 >> 172.21.15.76:0/1796352297 conn(0x7f95540d2370 msgr2=0x7f955406f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:26.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.488+0000 7f955ac30640 1 -- 172.21.15.76:0/1796352297 shutdown_connections 2024-04-25T19:15:26.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.488+0000 7f955ac30640 1 -- 172.21.15.76:0/1796352297 wait complete. 2024-04-25T19:15:26.501 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T19:15:26.502 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:15:26.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.607+0000 7ff6087ce640 1 Processor -- start 2024-04-25T19:15:26.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.607+0000 7ff6087ce640 1 -- start start 2024-04-25T19:15:26.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.608+0000 7ff6087ce640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff60015cd10 0x7ff600156c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:26.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.608+0000 7ff6087ce640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6001571c0 con 0x7ff60006e0c0 2024-04-25T19:15:26.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.608+0000 7ff6087ce640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff60015d250 con 0x7ff60006e4d0 2024-04-25T19:15:26.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.608+0000 7ff6087ce640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff60006e8e0 con 0x7ff60015cd10 2024-04-25T19:15:26.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.608+0000 7ff605d69640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff60006e0c0 0x7ff6001609c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35202/0 (socket says 172.21.15.76:35202) 2024-04-25T19:15:26.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.609+0000 7ff605d69640 1 -- 172.21.15.76:0/854516463 learned_addr learned my addr 172.21.15.76:0/854516463 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:26.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.609+0000 7ff605568640 1 -- 172.21.15.76:0/854516463 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3835724461 0 0) 0x7ff6001571c0 con 0x7ff60006e0c0 2024-04-25T19:15:26.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.609+0000 7ff605568640 1 -- 172.21.15.76:0/854516463 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5ec003520 con 0x7ff60006e0c0 2024-04-25T19:15:26.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.609+0000 7ff60656a640 1 --2- 172.21.15.76:0/854516463 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff60015cd10 0x7ff600156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:26.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.609+0000 7ff605568640 1 -- 172.21.15.76:0/854516463 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2004212129 0 0) 0x7ff5ec003520 con 0x7ff60006e0c0 2024-04-25T19:15:26.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.609+0000 7ff605568640 1 -- 172.21.15.76:0/854516463 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff60015cd10 msgr2=0x7ff600156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:26.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.609+0000 7ff605568640 1 --2- 172.21.15.76:0/854516463 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff60015cd10 0x7ff600156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:26.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.609+0000 7ff605568640 1 -- 172.21.15.76:0/854516463 >> v1:172.21.15.105:6789/0 conn(0x7ff60006e4d0 legacy=0x7ff600072ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:26.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.609+0000 7ff605568640 1 -- 172.21.15.76:0/854516463 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60015d510 con 0x7ff60006e0c0 2024-04-25T19:15:26.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.609+0000 7ff605568640 1 -- 172.21.15.76:0/854516463 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7ff5fc001670 con 0x7ff60006e0c0 2024-04-25T19:15:26.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.609+0000 7ff605568640 1 -- 172.21.15.76:0/854516463 >> v1:172.21.15.76:6789/0 conn(0x7ff60006e0c0 legacy=0x7ff6001609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:26.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.609+0000 7ff605568640 1 --2- 172.21.15.76:0/854516463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5ec0039e0 0x7ff5ec003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:26.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.609+0000 7ff605568640 1 --2- 172.21.15.76:0/854516463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5ec004800 0x7ff5ec004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:26.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.609+0000 7ff605568640 1 --2- 172.21.15.76:0/854516463 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff60015cd10 0x7ff5ec0052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:26.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.609+0000 7ff605568640 1 -- 172.21.15.76:0/854516463 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff60006e8e0 con 0x7ff5ec0039e0 2024-04-25T19:15:26.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.609+0000 7ff605568640 1 -- 172.21.15.76:0/854516463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec00bba0 con 0x7ff5ec004800 2024-04-25T19:15:26.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.609+0000 7ff605568640 1 -- 172.21.15.76:0/854516463 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff5ec00bda0 con 0x7ff60015cd10 2024-04-25T19:15:26.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.609+0000 7ff60656a640 1 --2- 172.21.15.76:0/854516463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5ec0039e0 0x7ff5ec003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:26.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.609+0000 7ff605d69640 1 --2- 172.21.15.76:0/854516463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5ec004800 0x7ff5ec004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:26.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.609+0000 7ff606d6b640 1 --2- 172.21.15.76:0/854516463 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff60015cd10 0x7ff5ec0052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:26.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.609+0000 7ff60656a640 1 -- 172.21.15.76:0/854516463 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff60015cd10 msgr2=0x7ff5ec0052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:26.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.609+0000 7ff60656a640 1 --2- 172.21.15.76:0/854516463 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff60015cd10 0x7ff5ec0052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:26.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.609+0000 7ff60656a640 1 -- 172.21.15.76:0/854516463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5ec004800 msgr2=0x7ff5ec004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:26.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.609+0000 7ff60656a640 1 --2- 172.21.15.76:0/854516463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5ec004800 0x7ff5ec004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:26.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.609+0000 7ff60656a640 1 -- 172.21.15.76:0/854516463 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ec003520 con 0x7ff5ec0039e0 2024-04-25T19:15:26.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.609+0000 7ff606d6b640 1 --2- 172.21.15.76:0/854516463 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff60015cd10 0x7ff5ec0052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:26.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.610+0000 7ff60656a640 1 --2- 172.21.15.76:0/854516463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5ec0039e0 0x7ff5ec003e50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8007360 tx=0x7ff5ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248a5b53dcbac96a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:26.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.610+0000 7ff605568640 1 -- 172.21.15.76:0/854516463 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8009be0 con 0x7ff5ec0039e0 2024-04-25T19:15:26.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.610+0000 7ff605568640 1 -- 172.21.15.76:0/854516463 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5f8036600 con 0x7ff5ec0039e0 2024-04-25T19:15:26.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.610+0000 7ff605568640 1 -- 172.21.15.76:0/854516463 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f80369d0 con 0x7ff5ec0039e0 2024-04-25T19:15:26.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.611+0000 7ff6087ce640 1 -- 172.21.15.76:0/854516463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5ec0039e0 msgr2=0x7ff5ec003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:26.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.611+0000 7ff6087ce640 1 --2- 172.21.15.76:0/854516463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5ec0039e0 0x7ff5ec003e50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8007360 tx=0x7ff5ec005830 comp rx=0 tx=0).stop 2024-04-25T19:15:26.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.611+0000 7ff606d6b640 1 -- 172.21.15.76:0/854516463 reap_dead start 2024-04-25T19:15:26.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.611+0000 7ff6087ce640 1 -- 172.21.15.76:0/854516463 shutdown_connections 2024-04-25T19:15:26.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.611+0000 7ff6087ce640 1 -- 172.21.15.76:0/854516463 >> 172.21.15.76:0/854516463 conn(0x7ff600070b00 msgr2=0x7ff600070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:26.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.611+0000 7ff6087ce640 1 -- 172.21.15.76:0/854516463 shutdown_connections 2024-04-25T19:15:26.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.611+0000 7ff6087ce640 1 -- 172.21.15.76:0/854516463 wait complete. 2024-04-25T19:15:26.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.613+0000 7ff6087ce640 1 Processor -- start 2024-04-25T19:15:26.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.613+0000 7ff6087ce640 1 -- start start 2024-04-25T19:15:26.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.614+0000 7ff6087ce640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5ec0039e0 0x7ff6001f6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:26.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.614+0000 7ff6087ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6001e8820 0x7ff6001510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:26.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.614+0000 7ff6087ce640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6001e8c30 0x7ff6001515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:26.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.614+0000 7ff6087ce640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff6001604a0 con 0x7ff6001e8c30 2024-04-25T19:15:26.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.614+0000 7ff6087ce640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff60015dad0 con 0x7ff6001e8820 2024-04-25T19:15:26.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.614+0000 7ff6087ce640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff6001610d0 con 0x7ff5ec0039e0 2024-04-25T19:15:26.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.614+0000 7ff606d6b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6001e8c30 0x7ff6001515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:26.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.614+0000 7ff606d6b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6001e8c30 0x7ff6001515f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59056/0 (socket says 172.21.15.76:59056) 2024-04-25T19:15:26.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.614+0000 7ff606d6b640 1 -- 172.21.15.76:0/1696559270 learned_addr learned my addr 172.21.15.76:0/1696559270 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:26.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.614+0000 7ff60656a640 1 --2- 172.21.15.76:0/1696559270 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5ec0039e0 0x7ff6001f6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:26.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.614+0000 7ff605d69640 1 --2- 172.21.15.76:0/1696559270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6001e8820 0x7ff6001510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:26.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.614+0000 7ff606d6b640 1 -- 172.21.15.76:0/1696559270 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5ec0039e0 msgr2=0x7ff6001f6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:26.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.614+0000 7ff606d6b640 1 --2- 172.21.15.76:0/1696559270 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5ec0039e0 0x7ff6001f6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:26.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.614+0000 7ff606d6b640 1 -- 172.21.15.76:0/1696559270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6001e8820 msgr2=0x7ff6001510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:26.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.614+0000 7ff606d6b640 1 --2- 172.21.15.76:0/1696559270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6001e8820 0x7ff6001510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:26.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.614+0000 7ff606d6b640 1 -- 172.21.15.76:0/1696559270 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6001f6a90 con 0x7ff6001e8c30 2024-04-25T19:15:26.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.615+0000 7ff606d6b640 1 --2- 172.21.15.76:0/1696559270 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6001e8c30 0x7ff6001515f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff600072560 tx=0x7ff5f400e760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:26.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.615+0000 7ff5f2ffd640 1 -- 172.21.15.76:0/1696559270 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4019070 con 0x7ff6001e8c30 2024-04-25T19:15:26.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.615+0000 7ff5f2ffd640 1 -- 172.21.15.76:0/1696559270 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5f400ee40 con 0x7ff6001e8c30 2024-04-25T19:15:26.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.615+0000 7ff5f2ffd640 1 -- 172.21.15.76:0/1696559270 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4007030 con 0x7ff6001e8c30 2024-04-25T19:15:26.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.615+0000 7ff6087ce640 1 -- 172.21.15.76:0/1696559270 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff600155710 con 0x7ff6001e8c30 2024-04-25T19:15:26.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.615+0000 7ff6087ce640 1 -- 172.21.15.76:0/1696559270 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6000b3b20 con 0x7ff6001e8c30 2024-04-25T19:15:26.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.616+0000 7ff6087ce640 1 -- 172.21.15.76:0/1696559270 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff60006a390 con 0x7ff6001e8c30 2024-04-25T19:15:26.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.616+0000 7ff5f2ffd640 1 -- 172.21.15.76:0/1696559270 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff5f401c070 con 0x7ff6001e8c30 2024-04-25T19:15:26.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.617+0000 7ff5f2ffd640 1 --2- 172.21.15.76:0/1696559270 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff5e40744b0 0x7ff5e40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:26.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.617+0000 7ff5f2ffd640 1 -- 172.21.15.76:0/1696559270 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff5f40941b0 con 0x7ff6001e8c30 2024-04-25T19:15:26.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.617+0000 7ff60656a640 1 --2- 172.21.15.76:0/1696559270 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff5e40744b0 0x7ff5e40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:26.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.618+0000 7ff60656a640 1 --2- 172.21.15.76:0/1696559270 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff5e40744b0 0x7ff5e40769c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8035000 tx=0x7ff5f800b030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:26.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.622+0000 7ff5f2ffd640 1 -- 172.21.15.76:0/1696559270 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff5f405e410 con 0x7ff6001e8c30 2024-04-25T19:15:26.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.797+0000 7ff6087ce640 1 -- 172.21.15.76:0/1696559270 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff6000b5290 con 0x7ff6001e8c30 2024-04-25T19:15:26.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.798+0000 7ff5f2ffd640 1 -- 172.21.15.76:0/1696559270 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7ff5f4062710 con 0x7ff6001e8c30 2024-04-25T19:15:26.800 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:26.800 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":21,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","created":"2024-04-25T19:15:01.693137+0000","modified":"2024-04-25T19:15:20.401211+0000","last_up_change":"2024-04-25T19:15:11.891681+0000","last_in_change":"2024-04-25T19:15:03.898065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:15:13.846514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:15:16.606533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:15:17.719197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72a16c02-48f9-45aa-9447-fcf755ebf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409574998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409574998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409574998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409574998}]},"public_addr":"172.21.15.76:6817/1409574998","cluster_addr":"172.21.15.76:6819/1409574998","heartbeat_back_addr":"172.21.15.76:6823/1409574998","heartbeat_front_addr":"172.21.15.76:6821/1409574998","state":["exists","up"]},{"osd":1,"uuid":"818d50f0-76c6-45f8-bf79-5e304f14c5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6801","nonce":1302663821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6803","nonce":1302663821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6807","nonce":1302663821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6805","nonce":1302663821}]},"public_addr":"172.21.15.76:6801/1302663821","cluster_addr":"172.21.15.76:6803/1302663821","heartbeat_back_addr":"172.21.15.76:6807/1302663821","heartbeat_front_addr":"172.21.15.76:6805/1302663821","state":["exists","up"]},{"osd":2,"uuid":"5714c8f5-ae23-473c-8cf1-db73811acded","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6809","nonce":1992938181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6811","nonce":1992938181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6815","nonce":1992938181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6813","nonce":1992938181}]},"public_addr":"172.21.15.76:6809/1992938181","cluster_addr":"172.21.15.76:6811/1992938181","heartbeat_back_addr":"172.21.15.76:6815/1992938181","heartbeat_front_addr":"172.21.15.76:6813/1992938181","state":["exists","up"]},{"osd":3,"uuid":"13204cdb-d8e6-4f97-bc6e-9561f442decf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6825","nonce":3418257163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6827","nonce":3418257163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6831","nonce":3418257163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6829","nonce":3418257163}]},"public_addr":"172.21.15.76:6825/3418257163","cluster_addr":"172.21.15.76:6827/3418257163","heartbeat_back_addr":"172.21.15.76:6831/3418257163","heartbeat_front_addr":"172.21.15.76:6829/3418257163","state":["exists","up"]},{"osd":4,"uuid":"5bea5166-1986-4d56-b5f5-ecc8cbe7f3ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6801","nonce":2125002468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6803","nonce":2125002468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6807","nonce":2125002468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6805","nonce":2125002468}]},"public_addr":"172.21.15.105:6801/2125002468","cluster_addr":"172.21.15.105:6803/2125002468","heartbeat_back_addr":"172.21.15.105:6807/2125002468","heartbeat_front_addr":"172.21.15.105:6805/2125002468","state":["exists","up"]},{"osd":5,"uuid":"96f89dcc-270d-4421-b8e8-3b92ba5f9844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6819","nonce":3413103121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6821","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6823","nonce":3413103121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6831","nonce":3413103121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6828","nonce":3413103121}]},"public_addr":"172.21.15.105:6819/3413103121","cluster_addr":"172.21.15.105:6823/3413103121","heartbeat_back_addr":"172.21.15.105:6831/3413103121","heartbeat_front_addr":"172.21.15.105:6828/3413103121","state":["exists","up"]},{"osd":6,"uuid":"93eb087a-d9ba-4096-b85d-b74d96e819a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6809","nonce":2721832954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6811","nonce":2721832954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6815","nonce":2721832954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6813","nonce":2721832954}]},"public_addr":"172.21.15.105:6809/2721832954","cluster_addr":"172.21.15.105:6811/2721832954","heartbeat_back_addr":"172.21.15.105:6815/2721832954","heartbeat_front_addr":"172.21.15.105:6813/2721832954","state":["exists","up"]},{"osd":7,"uuid":"2f80f790-ad2d-460e-a37d-47bf1771ef43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6818","nonce":548350430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6822","nonce":548350430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6829","nonce":548350430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6825","nonce":548350430}]},"public_addr":"172.21.15.105:6818/548350430","cluster_addr":"172.21.15.105:6822/548350430","heartbeat_back_addr":"172.21.15.105:6829/548350430","heartbeat_front_addr":"172.21.15.105:6825/548350430","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.106901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.072819+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.126865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.053464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:08.982614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.104825+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.112714+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.107387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:15:26.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.800+0000 7ff6087ce640 1 -- 172.21.15.76:0/1696559270 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff5e40744b0 msgr2=0x7ff5e40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:26.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.800+0000 7ff6087ce640 1 --2- 172.21.15.76:0/1696559270 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff5e40744b0 0x7ff5e40769c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8035000 tx=0x7ff5f800b030 comp rx=0 tx=0).stop 2024-04-25T19:15:26.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.800+0000 7ff6087ce640 1 -- 172.21.15.76:0/1696559270 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6001e8c30 msgr2=0x7ff6001515f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:26.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.800+0000 7ff6087ce640 1 --2- 172.21.15.76:0/1696559270 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6001e8c30 0x7ff6001515f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff600072560 tx=0x7ff5f400e760 comp rx=0 tx=0).stop 2024-04-25T19:15:26.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.800+0000 7ff6087ce640 1 -- 172.21.15.76:0/1696559270 shutdown_connections 2024-04-25T19:15:26.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.800+0000 7ff6087ce640 1 --2- 172.21.15.76:0/1696559270 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff5e40744b0 0x7ff5e40769c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:26.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.800+0000 7ff6087ce640 1 --2- 172.21.15.76:0/1696559270 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6001e8c30 0x7ff6001515f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:26.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.800+0000 7ff6087ce640 1 --2- 172.21.15.76:0/1696559270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6001e8820 0x7ff6001510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:26.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.800+0000 7ff6087ce640 1 --2- 172.21.15.76:0/1696559270 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5ec0039e0 0x7ff6001f6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:26.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.800+0000 7ff6087ce640 1 -- 172.21.15.76:0/1696559270 >> 172.21.15.76:0/1696559270 conn(0x7ff600070b00 msgr2=0x7ff600075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:26.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.800+0000 7ff6087ce640 1 -- 172.21.15.76:0/1696559270 shutdown_connections 2024-04-25T19:15:26.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:26.800+0000 7ff6087ce640 1 -- 172.21.15.76:0/1696559270 wait complete. 2024-04-25T19:15:26.814 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T19:15:26.814 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T19:15:26.814 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T19:15:26.815 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T19:15:26.815 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T19:15:26.816 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T19:15:26.816 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T19:15:26.816 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T19:15:27.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.024+0000 7f784e586640 1 Processor -- start 2024-04-25T19:15:27.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.024+0000 7f6f428d1640 1 Processor -- start 2024-04-25T19:15:27.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.024+0000 7f6f428d1640 1 -- start start 2024-04-25T19:15:27.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.024+0000 7fbdab1c5640 1 Processor -- start 2024-04-25T19:15:27.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.024+0000 7fbdab1c5640 1 -- start start 2024-04-25T19:15:27.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.025+0000 7f784e586640 1 -- start start 2024-04-25T19:15:27.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.026+0000 7fbdab1c5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbd9c00e480 0x7fbd9c006090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.026+0000 7f310b96c640 1 Processor -- start 2024-04-25T19:15:27.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.026+0000 7f1b40b2e640 1 Processor -- start 2024-04-25T19:15:27.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.026+0000 7fbdab1c5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbd9c04c940 0x7fbd9c0d52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.026+0000 7fbdab1c5640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd9c0df6e0 con 0x7fbd9c0471b0 2024-04-25T19:15:27.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.026+0000 7fbdab1c5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fbd9c046a40 con 0x7fbd9c00e480 2024-04-25T19:15:27.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.026+0000 7fbdab1c5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fbd9c033570 con 0x7fbd9c04c940 2024-04-25T19:15:27.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.026+0000 7f4f5759e640 1 Processor -- start 2024-04-25T19:15:27.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.026+0000 7f4fe96b1640 1 Processor -- start 2024-04-25T19:15:27.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.026+0000 7f4fe96b1640 1 -- start start 2024-04-25T19:15:27.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.026+0000 7f6f428d1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6f3c0ceca0 0x7f6f3c15b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.026+0000 7f6f428d1640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f3c0cda30 con 0x7f6f3c0ca190 2024-04-25T19:15:27.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.026+0000 7f6f428d1640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f3c0cdd10 con 0x7f6f3c0c9d80 2024-04-25T19:15:27.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.027+0000 7f6f428d1640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f6f3c0cc880 con 0x7f6f3c0ceca0 2024-04-25T19:15:27.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.027+0000 7f6f3bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6f3c0ceca0 0x7f6f3c15b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.027+0000 7f6f3bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6f3c0ceca0 0x7f6f3c15b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59064/0 (socket says 172.21.15.76:59064) 2024-04-25T19:15:27.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.027+0000 7f784e586640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78480cb800 0x7f78480cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.027+0000 7f784e586640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78480c6850 con 0x7f78480d0510 2024-04-25T19:15:27.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.027+0000 7f784e586640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78480c6b30 con 0x7f78480c62c0 2024-04-25T19:15:27.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.027+0000 7f784e586640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f784815af90 con 0x7f78480cb800 2024-04-25T19:15:27.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.026+0000 7f1b40b2e640 1 -- start start 2024-04-25T19:15:27.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.026+0000 7f4f5759e640 1 -- start start 2024-04-25T19:15:27.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.027+0000 7fbdaa1c3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbd9c00e480 0x7fbd9c006090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.027+0000 7fbdaa9c4640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fbd9c0471b0 0x7fbd9c0de910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.76:59510/0 (socket says 172.21.15.76:59510) 2024-04-25T19:15:27.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.027+0000 7fbdaa9c4640 1 -- 172.21.15.76:0/2762203170 learned_addr learned my addr 172.21.15.76:0/2762203170 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:27.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.027+0000 7f2089842640 1 Processor -- start 2024-04-25T19:15:27.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.027+0000 7f784cb23640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f78480d0510 0x7f78481653b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35204/0 (socket says 172.21.15.76:35204) 2024-04-25T19:15:27.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.027+0000 7f784cb23640 1 -- 172.21.15.76:0/3675998944 learned_addr learned my addr 172.21.15.76:0/3675998944 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:27.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.027+0000 7f2089842640 1 -- start start 2024-04-25T19:15:27.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.027+0000 7f6f40e6e640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f6f3c0ca190 0x7f6f3c16b810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35218/0 (socket says 172.21.15.76:35218) 2024-04-25T19:15:27.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.027+0000 7f6f40e6e640 1 -- 172.21.15.76:0/2955240250 learned_addr learned my addr 172.21.15.76:0/2955240250 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:27.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.028+0000 7fbda99c2640 1 --2- 172.21.15.76:0/2762203170 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbd9c04c940 0x7fbd9c0d52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.026+0000 7f310b96c640 1 -- start start 2024-04-25T19:15:27.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.028+0000 7f78477fe640 1 --2- 172.21.15.76:0/3675998944 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78480cb800 0x7f78480cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.028+0000 7f4fe96b1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4fe4152fa0 0x7f4fe40684e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.028+0000 7f4fe96b1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4fe40c4200 0x7f4fe40c4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.028+0000 7f4fe96b1640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fe415d380 con 0x7f4fe40731f0 2024-04-25T19:15:27.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.028+0000 7f4fe96b1640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4fe40711e0 con 0x7f4fe4152fa0 2024-04-25T19:15:27.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.028+0000 7f4fe96b1640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f4fe40736e0 con 0x7f4fe40c4200 2024-04-25T19:15:27.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.028+0000 7f4fe2ffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4fe40731f0 0x7f4fe4159860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.76:59538/0 (socket says 172.21.15.76:59538) 2024-04-25T19:15:27.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.028+0000 7f4fe2ffd640 1 -- 172.21.15.76:0/4012125902 learned_addr learned my addr 172.21.15.76:0/4012125902 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:27.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.028+0000 7fbda91c1640 1 -- 172.21.15.76:0/2762203170 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3954809724 0 0) 0x7fbd9c0df6e0 con 0x7fbd9c0471b0 2024-04-25T19:15:27.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.028+0000 7fbda91c1640 1 -- 172.21.15.76:0/2762203170 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd90003520 con 0x7fbd9c0471b0 2024-04-25T19:15:27.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.028+0000 7f7846ffd640 1 -- 172.21.15.76:0/3675998944 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1531199104 0 0) 0x7f78480c6b30 con 0x7f78480c62c0 2024-04-25T19:15:27.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.028+0000 7f7846ffd640 1 -- 172.21.15.76:0/3675998944 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7830003520 con 0x7f78480c62c0 2024-04-25T19:15:27.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.028+0000 7f6f3bfff640 1 -- 172.21.15.76:0/2955240250 >> v1:172.21.15.105:6789/0 conn(0x7f6f3c0c9d80 legacy=0x7f6f3c164480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:27.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.028+0000 7f6f3bfff640 1 -- 172.21.15.76:0/2955240250 >> v1:172.21.15.76:6789/0 conn(0x7f6f3c0ca190 legacy=0x7f6f3c16b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.028+0000 7f6f3bfff640 1 -- 172.21.15.76:0/2955240250 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3c0ccad0 con 0x7f6f3c0ceca0 2024-04-25T19:15:27.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.028+0000 7f310b96c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30fc045bb0 0x7f30fc047890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.028+0000 7f4fe27fc640 1 --2- 172.21.15.76:0/4012125902 >> v2:172.21.15.76:3300/0 conn(0x7f4fe4152fa0 0x7f4fe40684e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.028+0000 7f7846ffd640 1 -- 172.21.15.76:0/3675998944 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3804482493 0 0) 0x7f78480c6850 con 0x7f78480d0510 2024-04-25T19:15:27.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f7846ffd640 1 -- 172.21.15.76:0/3675998944 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78480c6b30 con 0x7f78480d0510 2024-04-25T19:15:27.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f1b40b2e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1b3c0c6640 0x7f1b3c1618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f4f5759e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4f580ca5d0 0x7f4f580c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f3109708640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30fc045bb0 0x7f30fc047890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f3109708640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30fc045bb0 0x7f30fc047890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:45682/0 (socket says 172.21.15.76:45682) 2024-04-25T19:15:27.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f3109708640 1 -- 172.21.15.76:0/1711982448 learned_addr learned my addr 172.21.15.76:0/1711982448 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:27.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f4f5759e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4f58074110 0x7f4f580c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f4f5759e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f580737b0 con 0x7f4f5815b220 2024-04-25T19:15:27.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f4f5759e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4f580b3dd0 con 0x7f4f58074110 2024-04-25T19:15:27.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f4f5759e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f4f580b4020 con 0x7f4f580ca5d0 2024-04-25T19:15:27.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f1b40b2e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b3c0cdd60 0x7f1b3c0ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f310b96c640 1 -- 172.21.15.76:0/1711982448 --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30fc0dc270 con 0x7f30fc04d190 2024-04-25T19:15:27.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f310b96c640 1 -- 172.21.15.76:0/1711982448 --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30fc0dc550 con 0x7f30fc046510 2024-04-25T19:15:27.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f310b96c640 1 -- 172.21.15.76:0/1711982448 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f30fc033980 con 0x7f30fc045bb0 2024-04-25T19:15:27.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f1b3a59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1b3c0c6640 0x7f1b3c1618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f1b3a59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1b3c0c6640 0x7f1b3c1618e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59080/0 (socket says 172.21.15.76:59080) 2024-04-25T19:15:27.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f1b3a59c640 1 -- 172.21.15.76:0/1997543209 learned_addr learned my addr 172.21.15.76:0/1997543209 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:27.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f7846ffd640 1 -- 172.21.15.76:0/3675998944 <== mon.1 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 409303881 0 0) 0x7f7830003520 con 0x7f78480c62c0 2024-04-25T19:15:27.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f7846ffd640 1 -- 172.21.15.76:0/3675998944 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78480cb800 msgr2=0x7f78480cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:27.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f4f56d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4f58074110 0x7f4f580c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f7846ffd640 1 --2- 172.21.15.76:0/3675998944 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78480cb800 0x7f78480cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f7846ffd640 1 -- 172.21.15.76:0/3675998944 >> v1:172.21.15.76:6789/0 conn(0x7f78480d0510 legacy=0x7f78481653b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f7846ffd640 1 -- 172.21.15.76:0/3675998944 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784806f6f0 con 0x7f78480c62c0 2024-04-25T19:15:27.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7fbda99c2640 1 -- 172.21.15.76:0/2762203170 >> v2:172.21.15.105:3300/0 conn(0x7fbd9c00e480 msgr2=0x7fbd9c006090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:27.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f4f55d9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4f580ca5d0 0x7f4f580c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f4f55d9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4f580ca5d0 0x7f4f580c8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:42476/0 (socket says 172.21.15.76:42476) 2024-04-25T19:15:27.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f4f55d9b640 1 -- 172.21.15.76:0/2355763152 learned_addr learned my addr 172.21.15.76:0/2355763152 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:27.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7fbda91c1640 1 -- 172.21.15.76:0/2762203170 <== mon.1 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2261320349 0 0) 0x7fbd90003520 con 0x7fbd9c0471b0 2024-04-25T19:15:27.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7fbda99c2640 1 --2- 172.21.15.76:0/2762203170 >> v2:172.21.15.105:3300/0 conn(0x7fbd9c00e480 0x7fbd9c006090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7fbda99c2640 1 -- 172.21.15.76:0/2762203170 >> v1:172.21.15.105:6789/0 conn(0x7fbd9c0471b0 legacy=0x7fbd9c0de910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f1b3ad9d640 1 --2- 172.21.15.76:0/1997543209 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b3c0cdd60 0x7f1b3c0ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7fbda99c2640 1 -- 172.21.15.76:0/2762203170 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9c0df440 con 0x7fbd9c04c940 2024-04-25T19:15:27.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f1b40b2e640 1 -- 172.21.15.76:0/1997543209 --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b3c160e60 con 0x7f1b3c074300 2024-04-25T19:15:27.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.030+0000 7f1b40b2e640 1 -- 172.21.15.76:0/1997543209 --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f1b3c161e20 con 0x7f1b3c0c6640 2024-04-25T19:15:27.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.030+0000 7fbdaa1c3640 1 --2- 172.21.15.76:0/2762203170 >> v2:172.21.15.105:3300/0 conn(0x7fbd9c00e480 0x7fbd9c006090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:27.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.030+0000 7f1b40b2e640 1 -- 172.21.15.76:0/1997543209 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f1b3c0c8be0 con 0x7f1b3c0cdd60 2024-04-25T19:15:27.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.030+0000 7f7846ffd640 1 -- 172.21.15.76:0/3675998944 <== mon.1 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f7840001920 con 0x7f78480c62c0 2024-04-25T19:15:27.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.030+0000 7f7846ffd640 1 -- 172.21.15.76:0/3675998944 >> v1:172.21.15.105:6789/0 conn(0x7f78480c62c0 legacy=0x7f784815b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.030+0000 7f6f3bfff640 1 --2- 172.21.15.76:0/2955240250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f3c0ceca0 0x7f6f3c15b0f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009d90 tx=0x7f6f3402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7f38fba0f1d9f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f4fe27fc640 1 -- 172.21.15.76:0/4012125902 >> v2:172.21.15.105:3300/0 conn(0x7f4fe40c4200 msgr2=0x7f4fe40c4610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T19:15:27.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f4fe27fc640 1 --2- 172.21.15.76:0/4012125902 >> v2:172.21.15.105:3300/0 conn(0x7f4fe40c4200 0x7f4fe40c4610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f4fe27fc640 1 -- 172.21.15.76:0/4012125902 >> v1:172.21.15.105:6789/0 conn(0x7f4fe40731f0 legacy=0x7f4fe4159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:27.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f4fe27fc640 1 -- 172.21.15.76:0/4012125902 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe415c5c0 con 0x7f4fe4152fa0 2024-04-25T19:15:27.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.030+0000 7f7846ffd640 1 --2- 172.21.15.76:0/3675998944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7830003b40 0x7f7830003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.030+0000 7f7846ffd640 1 --2- 172.21.15.76:0/3675998944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7830004800 0x7f7830004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.030+0000 7f7846ffd640 1 --2- 172.21.15.76:0/3675998944 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78480cb800 0x7f7830005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f3109708640 1 -- 172.21.15.76:0/1711982448 >> v1:172.21.15.105:6789/0 conn(0x7f30fc046510 legacy=0x7f30fc006490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f3109708640 1 -- 172.21.15.76:0/1711982448 >> v1:172.21.15.76:6789/0 conn(0x7f30fc04d190 legacy=0x7f30fc04c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.029+0000 7f3109708640 1 -- 172.21.15.76:0/1711982448 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30fc046db0 con 0x7f30fc045bb0 2024-04-25T19:15:27.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.030+0000 7f2089842640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2084074300 0x7f208415b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.030+0000 7f2089842640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f20840cdd60 0x7f20840ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.030+0000 7f2089842640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2084160e60 con 0x7f20840c6640 2024-04-25T19:15:27.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.030+0000 7f2089842640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f208415b590 con 0x7f20840cdd60 2024-04-25T19:15:27.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.030+0000 7f2089842640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f20840c8be0 con 0x7f2084074300 2024-04-25T19:15:27.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.030+0000 7f7846ffd640 1 -- 172.21.15.76:0/3675998944 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f784815af90 con 0x7f7830003b40 2024-04-25T19:15:27.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.030+0000 7f7846ffd640 1 -- 172.21.15.76:0/3675998944 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f783000bb80 con 0x7f7830004800 2024-04-25T19:15:27.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.030+0000 7f7846ffd640 1 -- 172.21.15.76:0/3675998944 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f783000bd80 con 0x7f78480cb800 2024-04-25T19:15:27.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.030+0000 7f1b3ad9d640 1 -- 172.21.15.76:0/1997543209 >> v2:172.21.15.76:3300/0 conn(0x7f1b3c0c6640 msgr2=0x7f1b3c1618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:27.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.030+0000 7f1b3ad9d640 1 --2- 172.21.15.76:0/1997543209 >> v2:172.21.15.76:3300/0 conn(0x7f1b3c0c6640 0x7f1b3c1618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.030+0000 7f1b3ad9d640 1 -- 172.21.15.76:0/1997543209 >> v1:172.21.15.76:6789/0 conn(0x7f1b3c074300 legacy=0x7f1b3c15b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:27.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.030+0000 7f1b3ad9d640 1 -- 172.21.15.76:0/1997543209 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3c0c8e80 con 0x7f1b3c0cdd60 2024-04-25T19:15:27.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f78477fe640 1 --2- 172.21.15.76:0/3675998944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7830004800 0x7f7830004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f2082ffd640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f20840c6640 0x7f20841618e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.76:59572/0 (socket says 172.21.15.76:59572) 2024-04-25T19:15:27.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f2082ffd640 1 -- 172.21.15.76:0/1118355094 learned_addr learned my addr 172.21.15.76:0/1118355094 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:27.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f7847fff640 1 --2- 172.21.15.76:0/3675998944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7830003b40 0x7f7830003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f784cb23640 1 --2- 172.21.15.76:0/3675998944 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78480cb800 0x7f7830005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f3109708640 1 --2- 172.21.15.76:0/1711982448 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30fc045bb0 0x7f30fc047890 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f31000098a0 tx=0x7f310002a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee1ead8e87b3ea5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f20827fc640 1 --2- 172.21.15.76:0/1118355094 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2084074300 0x7f208415b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.030+0000 7f4f55d9b640 1 -- 172.21.15.76:0/2355763152 >> v2:172.21.15.76:3300/0 conn(0x7f4f58074110 msgr2=0x7f4f580c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:27.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.030+0000 7f4f55d9b640 1 --2- 172.21.15.76:0/2355763152 >> v2:172.21.15.76:3300/0 conn(0x7f4f58074110 0x7f4f580c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.030+0000 7f4f55d9b640 1 -- 172.21.15.76:0/2355763152 >> v1:172.21.15.105:6789/0 conn(0x7f4f5815b220 legacy=0x7f4f581613f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T19:15:27.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.030+0000 7f4f55d9b640 1 -- 172.21.15.76:0/2355763152 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f580b4300 con 0x7f4f580ca5d0 2024-04-25T19:15:27.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f4fe27fc640 1 --2- 172.21.15.76:0/4012125902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fe4152fa0 0x7f4fe40684e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40c4120 tx=0x7f4fd802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51543eb79e736190 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7fbda99c2640 1 --2- 172.21.15.76:0/2762203170 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbd9c04c940 0x7fbd9c0d52f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbd940097a0 tx=0x7fbd94029fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=450827ae20aacfb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f6f3affd640 1 -- 172.21.15.76:0/2955240250 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34007740 con 0x7f6f3c0ceca0 2024-04-25T19:15:27.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f20837fe640 1 --2- 172.21.15.76:0/1118355094 >> v2:172.21.15.76:3300/0 conn(0x7f20840cdd60 0x7f20840ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f30fbfff640 1 -- 172.21.15.76:0/1711982448 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100037070 con 0x7f30fc045bb0 2024-04-25T19:15:27.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f30fbfff640 1 -- 172.21.15.76:0/1711982448 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f310002ae50 con 0x7f30fc045bb0 2024-04-25T19:15:27.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f30fbfff640 1 -- 172.21.15.76:0/1711982448 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100032840 con 0x7f30fc045bb0 2024-04-25T19:15:27.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f4fe1ffb640 1 -- 172.21.15.76:0/4012125902 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8037070 con 0x7f4fe4152fa0 2024-04-25T19:15:27.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f2081ffb640 1 -- 172.21.15.76:0/1118355094 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1386434030 0 0) 0x7f207c001490 con 0x7f20840c6640 2024-04-25T19:15:27.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f2081ffb640 1 -- 172.21.15.76:0/1118355094 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f206c003520 con 0x7f20840c6640 2024-04-25T19:15:27.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f6f3affd640 1 -- 172.21.15.76:0/2955240250 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f340078e0 con 0x7f6f3c0ceca0 2024-04-25T19:15:27.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f4f55d9b640 1 --2- 172.21.15.76:0/2355763152 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f580ca5d0 0x7f4f580c8d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4f58068500 tx=0x7f4f4802a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffea263ecb63a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f4fe1ffb640 1 -- 172.21.15.76:0/4012125902 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd802ae10 con 0x7f4fe4152fa0 2024-04-25T19:15:27.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f6f3affd640 1 -- 172.21.15.76:0/2955240250 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34007cb0 con 0x7f6f3c0ceca0 2024-04-25T19:15:27.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f4fe1ffb640 1 -- 172.21.15.76:0/4012125902 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8032660 con 0x7f4fe4152fa0 2024-04-25T19:15:27.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f4f5559a640 1 -- 172.21.15.76:0/2355763152 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f48037070 con 0x7f4f580ca5d0 2024-04-25T19:15:27.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f4f5559a640 1 -- 172.21.15.76:0/2355763152 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f48032440 con 0x7f4f580ca5d0 2024-04-25T19:15:27.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f4f5559a640 1 -- 172.21.15.76:0/2355763152 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f48032810 con 0x7f4f580ca5d0 2024-04-25T19:15:27.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f1b3ad9d640 1 --2- 172.21.15.76:0/1997543209 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b3c0cdd60 0x7f1b3c0ce170 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009d90 tx=0x7f1b3402a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5966bc8b6a66eae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.032+0000 7f1b39d9b640 1 -- 172.21.15.76:0/1997543209 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34007740 con 0x7f1b3c0cdd60 2024-04-25T19:15:27.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.032+0000 7f1b39d9b640 1 -- 172.21.15.76:0/1997543209 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b340078e0 con 0x7f1b3c0cdd60 2024-04-25T19:15:27.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f78477fe640 1 -- 172.21.15.76:0/3675998944 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78480cb800 msgr2=0x7f7830005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:27.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f78477fe640 1 --2- 172.21.15.76:0/3675998944 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f78480cb800 0x7f7830005280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f78477fe640 1 -- 172.21.15.76:0/3675998944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7830003b40 msgr2=0x7f7830003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:27.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f78477fe640 1 --2- 172.21.15.76:0/3675998944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7830003b40 0x7f7830003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f78477fe640 1 -- 172.21.15.76:0/3675998944 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7830003520 con 0x7f7830004800 2024-04-25T19:15:27.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.032+0000 7f1b39d9b640 1 -- 172.21.15.76:0/1997543209 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34007cb0 con 0x7f1b3c0cdd60 2024-04-25T19:15:27.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.032+0000 7f4fe96b1640 1 -- 172.21.15.76:0/4012125902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fe4152fa0 msgr2=0x7f4fe40684e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f20827fc640 1 -- 172.21.15.76:0/1118355094 >> v2:172.21.15.76:3300/0 conn(0x7f20840cdd60 msgr2=0x7f20840ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:27.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f20827fc640 1 --2- 172.21.15.76:0/1118355094 >> v2:172.21.15.76:3300/0 conn(0x7f20840cdd60 0x7f20840ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.032+0000 7f6f428d1640 1 -- 172.21.15.76:0/2955240250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f3c0ceca0 msgr2=0x7f6f3c15b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.032+0000 7f6f428d1640 1 --2- 172.21.15.76:0/2955240250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f3c0ceca0 0x7f6f3c15b0f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009d90 tx=0x7f6f3402a730 comp rx=0 tx=0).stop 2024-04-25T19:15:27.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.032+0000 7f310b96c640 1 -- 172.21.15.76:0/1711982448 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30fc045bb0 msgr2=0x7f30fc047890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.032+0000 7f310b96c640 1 --2- 172.21.15.76:0/1711982448 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30fc045bb0 0x7f30fc047890 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f31000098a0 tx=0x7f310002a300 comp rx=0 tx=0).stop 2024-04-25T19:15:27.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f20827fc640 1 -- 172.21.15.76:0/1118355094 >> v1:172.21.15.105:6789/0 conn(0x7f20840c6640 legacy=0x7f20841618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.031+0000 7f20827fc640 1 -- 172.21.15.76:0/1118355094 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20840c8e80 con 0x7f2084074300 2024-04-25T19:15:27.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.032+0000 7f20837fe640 1 --2- 172.21.15.76:0/1118355094 >> v2:172.21.15.76:3300/0 conn(0x7f20840cdd60 0x7f20840ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:27.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.032+0000 7fbda91c1640 1 -- 172.21.15.76:0/2762203170 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94037070 con 0x7fbd9c04c940 2024-04-25T19:15:27.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.032+0000 7fbda91c1640 1 -- 172.21.15.76:0/2762203170 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd9402aa30 con 0x7fbd9c04c940 2024-04-25T19:15:27.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.032+0000 7fbda91c1640 1 -- 172.21.15.76:0/2762203170 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda4050c90 con 0x7fbd9c04c940 2024-04-25T19:15:27.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.032+0000 7f4fe96b1640 1 --2- 172.21.15.76:0/4012125902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fe4152fa0 0x7f4fe40684e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40c4120 tx=0x7f4fd802a100 comp rx=0 tx=0).stop 2024-04-25T19:15:27.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.032+0000 7f4fe96b1640 1 -- 172.21.15.76:0/4012125902 shutdown_connections 2024-04-25T19:15:27.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.032+0000 7f4fe96b1640 1 --2- 172.21.15.76:0/4012125902 >> v2:172.21.15.105:3300/0 conn(0x7f4fe40c4200 0x7f4fe40c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.032+0000 7f4fe96b1640 1 --2- 172.21.15.76:0/4012125902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fe4152fa0 0x7f4fe40684e0 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40c4120 tx=0x7f4fd802a100 comp rx=0 tx=0).stop 2024-04-25T19:15:27.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.032+0000 7f4fe96b1640 1 -- 172.21.15.76:0/4012125902 >> 172.21.15.76:0/4012125902 conn(0x7f4fe40caff0 msgr2=0x7f4fe40cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.032+0000 7f6f428d1640 1 -- 172.21.15.76:0/2955240250 shutdown_connections 2024-04-25T19:15:27.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.032+0000 7f20827fc640 1 --2- 172.21.15.76:0/1118355094 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2084074300 0x7f208415b050 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f20780097a0 tx=0x7f207802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0dd286f4ba18157 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.032+0000 7f6f428d1640 1 --2- 172.21.15.76:0/2955240250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f3c0ceca0 0x7f6f3c15b0f0 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009d90 tx=0x7f6f3402a730 comp rx=0 tx=0).stop 2024-04-25T19:15:27.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.032+0000 7f6f428d1640 1 -- 172.21.15.76:0/2955240250 >> 172.21.15.76:0/2955240250 conn(0x7f6f3c070e70 msgr2=0x7f6f3c0ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f4fe96b1640 1 -- 172.21.15.76:0/4012125902 shutdown_connections 2024-04-25T19:15:27.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f6f428d1640 1 -- 172.21.15.76:0/2955240250 shutdown_connections 2024-04-25T19:15:27.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f78477fe640 1 --2- 172.21.15.76:0/3675998944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7830004800 0x7f7830004c10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f78480c7360 tx=0x7f7830005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60e0be4b82594ff1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f310b96c640 1 -- 172.21.15.76:0/1711982448 shutdown_connections 2024-04-25T19:15:27.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f4fe96b1640 1 -- 172.21.15.76:0/4012125902 wait complete. 2024-04-25T19:15:27.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f310b96c640 1 --2- 172.21.15.76:0/1711982448 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30fc045bb0 0x7f30fc047890 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f310b96c640 1 -- 172.21.15.76:0/1711982448 >> 172.21.15.76:0/1711982448 conn(0x7f30fc050c10 msgr2=0x7f30fc051020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f2081ffb640 1 -- 172.21.15.76:0/1118355094 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2078037070 con 0x7f2084074300 2024-04-25T19:15:27.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f2081ffb640 1 -- 172.21.15.76:0/1118355094 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2078032440 con 0x7f2084074300 2024-04-25T19:15:27.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f2081ffb640 1 -- 172.21.15.76:0/1118355094 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2078032850 con 0x7f2084074300 2024-04-25T19:15:27.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f7846ffd640 1 -- 172.21.15.76:0/3675998944 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c030070 con 0x7f7830004800 2024-04-25T19:15:27.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f4f5759e640 1 -- 172.21.15.76:0/2355763152 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f580ca5d0 msgr2=0x7f4f580c8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f4f5759e640 1 --2- 172.21.15.76:0/2355763152 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f580ca5d0 0x7f4f580c8d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4f58068500 tx=0x7f4f4802a2c0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f7846ffd640 1 -- 172.21.15.76:0/3675998944 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f783c02ada0 con 0x7f7830004800 2024-04-25T19:15:27.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f7846ffd640 1 -- 172.21.15.76:0/3675998944 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c00b610 con 0x7f7830004800 2024-04-25T19:15:27.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f4f5759e640 1 -- 172.21.15.76:0/2355763152 shutdown_connections 2024-04-25T19:15:27.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f4f5759e640 1 --2- 172.21.15.76:0/2355763152 >> v2:172.21.15.76:3300/0 conn(0x7f4f58074110 0x7f4f580c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f4f5759e640 1 --2- 172.21.15.76:0/2355763152 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f580ca5d0 0x7f4f580c8d80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f4f5759e640 1 -- 172.21.15.76:0/2355763152 >> 172.21.15.76:0/2355763152 conn(0x7f4f580cae80 msgr2=0x7f4f580cb290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f4f5759e640 1 -- 172.21.15.76:0/2355763152 shutdown_connections 2024-04-25T19:15:27.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7fbdab1c5640 1 -- 172.21.15.76:0/2762203170 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbd9c04c940 msgr2=0x7fbd9c0d52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7fbdab1c5640 1 --2- 172.21.15.76:0/2762203170 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbd9c04c940 0x7fbd9c0d52f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbd940097a0 tx=0x7fbd94029fb0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f4f5759e640 1 -- 172.21.15.76:0/2355763152 wait complete. 2024-04-25T19:15:27.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f1b40b2e640 1 -- 172.21.15.76:0/1997543209 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b3c0cdd60 msgr2=0x7f1b3c0ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f1b40b2e640 1 --2- 172.21.15.76:0/1997543209 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b3c0cdd60 0x7f1b3c0ce170 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009d90 tx=0x7f1b3402a6a0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f1b40b2e640 1 -- 172.21.15.76:0/1997543209 shutdown_connections 2024-04-25T19:15:27.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f6f428d1640 1 -- 172.21.15.76:0/2955240250 wait complete. 2024-04-25T19:15:27.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f1b40b2e640 1 --2- 172.21.15.76:0/1997543209 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b3c0cdd60 0x7f1b3c0ce170 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f1b40b2e640 1 --2- 172.21.15.76:0/1997543209 >> v2:172.21.15.76:3300/0 conn(0x7f1b3c0c6640 0x7f1b3c1618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f310b96c640 1 -- 172.21.15.76:0/1711982448 shutdown_connections 2024-04-25T19:15:27.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.033+0000 7f1b40b2e640 1 -- 172.21.15.76:0/1997543209 >> 172.21.15.76:0/1997543209 conn(0x7f1b3c070800 msgr2=0x7f1b3c070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.034+0000 7fbdab1c5640 1 -- 172.21.15.76:0/2762203170 shutdown_connections 2024-04-25T19:15:27.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.034+0000 7fbdab1c5640 1 --2- 172.21.15.76:0/2762203170 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbd9c04c940 0x7fbd9c0d52f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.034+0000 7fbdab1c5640 1 --2- 172.21.15.76:0/2762203170 >> v2:172.21.15.105:3300/0 conn(0x7fbd9c00e480 0x7fbd9c006090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.034+0000 7fbdab1c5640 1 -- 172.21.15.76:0/2762203170 >> 172.21.15.76:0/2762203170 conn(0x7fbd9c0503b0 msgr2=0x7fbd9c0507c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.034+0000 7f310b96c640 1 -- 172.21.15.76:0/1711982448 wait complete. 2024-04-25T19:15:27.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.034+0000 7fbdab1c5640 1 -- 172.21.15.76:0/2762203170 shutdown_connections 2024-04-25T19:15:27.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.034+0000 7f784e586640 1 -- 172.21.15.76:0/3675998944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7830004800 msgr2=0x7f7830004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.034+0000 7f784e586640 1 --2- 172.21.15.76:0/3675998944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7830004800 0x7f7830004c10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f78480c7360 tx=0x7f7830005830 comp rx=0 tx=0).stop 2024-04-25T19:15:27.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.034+0000 7f2089842640 1 -- 172.21.15.76:0/1118355094 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2084074300 msgr2=0x7f208415b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.034+0000 7f784cb23640 1 -- 172.21.15.76:0/3675998944 reap_dead start 2024-04-25T19:15:27.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.034+0000 7f784e586640 1 -- 172.21.15.76:0/3675998944 shutdown_connections 2024-04-25T19:15:27.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.034+0000 7f784e586640 1 -- 172.21.15.76:0/3675998944 >> 172.21.15.76:0/3675998944 conn(0x7f78480684e0 msgr2=0x7f78480688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.034+0000 7f2089842640 1 --2- 172.21.15.76:0/1118355094 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2084074300 0x7f208415b050 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f20780097a0 tx=0x7f207802a250 comp rx=0 tx=0).stop 2024-04-25T19:15:27.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.035+0000 7f2089842640 1 -- 172.21.15.76:0/1118355094 shutdown_connections 2024-04-25T19:15:27.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.035+0000 7f2089842640 1 --2- 172.21.15.76:0/1118355094 >> v2:172.21.15.76:3300/0 conn(0x7f20840cdd60 0x7f20840ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.035+0000 7f2089842640 1 --2- 172.21.15.76:0/1118355094 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2084074300 0x7f208415b050 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.035+0000 7f2089842640 1 -- 172.21.15.76:0/1118355094 >> 172.21.15.76:0/1118355094 conn(0x7f2084070800 msgr2=0x7f2084070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.034+0000 7fbdab1c5640 1 -- 172.21.15.76:0/2762203170 wait complete. 2024-04-25T19:15:27.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.035+0000 7f2089842640 1 -- 172.21.15.76:0/1118355094 shutdown_connections 2024-04-25T19:15:27.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.035+0000 7f784e586640 1 -- 172.21.15.76:0/3675998944 shutdown_connections 2024-04-25T19:15:27.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.035+0000 7f784e586640 1 -- 172.21.15.76:0/3675998944 wait complete. 2024-04-25T19:15:27.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.035+0000 7f4fe96b1640 1 Processor -- start 2024-04-25T19:15:27.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.035+0000 7f4fe96b1640 1 -- start start 2024-04-25T19:15:27.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.035+0000 7f2089842640 1 -- 172.21.15.76:0/1118355094 wait complete. 2024-04-25T19:15:27.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.035+0000 7f1b40b2e640 1 -- 172.21.15.76:0/1997543209 shutdown_connections 2024-04-25T19:15:27.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.036+0000 7f6f428d1640 1 Processor -- start 2024-04-25T19:15:27.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.037+0000 7f310b96c640 1 Processor -- start 2024-04-25T19:15:27.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.037+0000 7f310b96c640 1 -- start start 2024-04-25T19:15:27.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.037+0000 7f4fe96b1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4fe40731f0 0x7f4fe40c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.037+0000 7f4fe96b1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fe40c4200 0x7f4fe4155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.038+0000 7f4fe2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4fe40731f0 0x7f4fe40c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.038+0000 7f4fe27fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fe40c4200 0x7f4fe4155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.038+0000 7f4fe2ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4fe40731f0 0x7f4fe40c6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:42490/0 (socket says 172.21.15.76:42490) 2024-04-25T19:15:27.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.038+0000 7f4fe96b1640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4fe40c56d0 0x7f4fe40c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.038+0000 7f4fe27fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fe40c4200 0x7f4fe4155990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59104/0 (socket says 172.21.15.76:59104) 2024-04-25T19:15:27.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.038+0000 7f784e586640 1 Processor -- start 2024-04-25T19:15:27.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.037+0000 7fbdab1c5640 1 Processor -- start 2024-04-25T19:15:27.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.037+0000 7fbdab1c5640 1 -- start start 2024-04-25T19:15:27.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.037+0000 7f2089842640 1 Processor -- start 2024-04-25T19:15:27.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.036+0000 7f6f428d1640 1 -- start start 2024-04-25T19:15:27.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.038+0000 7f6f428d1640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6f3c0c9d80 0x7f6f3c1ec7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.036+0000 7f1b40b2e640 1 -- 172.21.15.76:0/1997543209 wait complete. 2024-04-25T19:15:27.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.036+0000 7f4f5759e640 1 Processor -- start 2024-04-25T19:15:27.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.036+0000 7f4f5759e640 1 -- start start 2024-04-25T19:15:27.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.038+0000 7f6f428d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f3c0ca190 0x7f6f3c1fb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.038+0000 7f6f428d1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f3c1f77e0 0x7f6f3c14ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.038+0000 7f6f428d1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c0cd2d0 con 0x7f6f3c1f77e0 2024-04-25T19:15:27.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.038+0000 7f6f428d1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c0cce40 con 0x7f6f3c0ca190 2024-04-25T19:15:27.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.038+0000 7f784e586640 1 -- start start 2024-04-25T19:15:27.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.038+0000 7f6f428d1640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f6f3c0cd450 con 0x7f6f3c0c9d80 2024-04-25T19:15:27.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f4f5759e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f58074110 0x7f4f580c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7fbdab1c5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd9c00e480 0x7fbd9c0dce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f6f40e6e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f3c1f77e0 0x7f6f3c14ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7fbdab1c5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbd9c0471b0 0x7fbd9c17cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f310b96c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30fc045bb0 0x7f30fc0cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7fbdab1c5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd9c04c940 0x7fbd9c17d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f4f5759e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4f580ca5d0 0x7f4f581ec330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f310b96c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30fc046510 0x7f30fc0c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7fbdaa1c3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd9c00e480 0x7fbd9c0dce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7fbdaa1c3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd9c00e480 0x7fbd9c0dce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:42506/0 (socket says 172.21.15.76:42506) 2024-04-25T19:15:27.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f310b96c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30fc04d190 0x7f30fc0c2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f1b40b2e640 1 Processor -- start 2024-04-25T19:15:27.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f310b96c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc047400 con 0x7f30fc045bb0 2024-04-25T19:15:27.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f1b40b2e640 1 -- start start 2024-04-25T19:15:27.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f4f5759e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f5815b220 0x7f4f581509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f4f5659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f58074110 0x7f4f580c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f4f55d9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4f580ca5d0 0x7f4f581ec330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f4f55d9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4f580ca5d0 0x7f4f581ec330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:45734/0 (socket says 172.21.15.76:45734) 2024-04-25T19:15:27.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f6f40e6e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f3c1f77e0 0x7f6f3c14ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59114/0 (socket says 172.21.15.76:59114) 2024-04-25T19:15:27.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f4f5659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f58074110 0x7f4f580c6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:42516/0 (socket says 172.21.15.76:42516) 2024-04-25T19:15:27.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f6f40e6e640 1 -- 172.21.15.76:0/4146577706 learned_addr learned my addr 172.21.15.76:0/4146577706 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:27.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f3109f09640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30fc04d190 0x7f30fc0c2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f6f3b7fe640 1 --2- 172.21.15.76:0/4146577706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f3c0ca190 0x7f6f3c1fb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f3109f09640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30fc04d190 0x7f30fc0c2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:42524/0 (socket says 172.21.15.76:42524) 2024-04-25T19:15:27.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.038+0000 7f4fe2ffd640 1 -- 172.21.15.76:0/3818361545 learned_addr learned my addr 172.21.15.76:0/3818361545 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:27.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.038+0000 7f4fe96b1640 1 -- 172.21.15.76:0/3818361545 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe415ce60 con 0x7f4fe40c4200 2024-04-25T19:15:27.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.038+0000 7f4fe96b1640 1 -- 172.21.15.76:0/3818361545 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe415cb80 con 0x7f4fe40731f0 2024-04-25T19:15:27.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.038+0000 7f4fe96b1640 1 -- 172.21.15.76:0/3818361545 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f4fe4074ab0 con 0x7f4fe40c56d0 2024-04-25T19:15:27.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.038+0000 7f4fe37fe640 1 --2- 172.21.15.76:0/3818361545 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4fe40c56d0 0x7f4fe40c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f4fe37fe640 1 -- 172.21.15.76:0/3818361545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4fe40731f0 msgr2=0x7f4fe40c6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f4fe37fe640 1 --2- 172.21.15.76:0/3818361545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4fe40731f0 0x7f4fe40c6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f4fe37fe640 1 -- 172.21.15.76:0/3818361545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fe40c4200 msgr2=0x7f4fe4155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f4fe37fe640 1 --2- 172.21.15.76:0/3818361545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fe40c4200 0x7f4fe4155990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f4fe37fe640 1 -- 172.21.15.76:0/3818361545 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe40c73d0 con 0x7f4fe40c56d0 2024-04-25T19:15:27.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f4fe37fe640 1 --2- 172.21.15.76:0/3818361545 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4fe40c56d0 0x7f4fe40c5ae0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400be20 tx=0x7f4fd400bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f3109f09640 1 -- 172.21.15.76:0/4131854023 learned_addr learned my addr 172.21.15.76:0/4131854023 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:27.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7fbdaa9c4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd9c04c940 0x7fbd9c17d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7fbdaa9c4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd9c04c940 0x7fbd9c17d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59126/0 (socket says 172.21.15.76:59126) 2024-04-25T19:15:27.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7f4fe27fc640 1 --2- 172.21.15.76:0/3818361545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4fe40c4200 0x7f4fe4155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-25T19:15:27.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7f4fcb7fe640 1 -- 172.21.15.76:0/3818361545 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4019070 con 0x7f4fe40c56d0 2024-04-25T19:15:27.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7fbdaa1c3640 1 -- 172.21.15.76:0/411451022 learned_addr learned my addr 172.21.15.76:0/411451022 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:27.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7f4fcb7fe640 1 -- 172.21.15.76:0/3818361545 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd4014440 con 0x7f4fe40c56d0 2024-04-25T19:15:27.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.039+0000 7f3108f07640 1 --2- 172.21.15.76:0/4131854023 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30fc046510 0x7f30fc0c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7fbda99c2640 1 --2- 172.21.15.76:0/411451022 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbd9c0471b0 0x7fbd9c17cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7f310b96c640 1 -- 172.21.15.76:0/4131854023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc046b60 con 0x7f30fc04d190 2024-04-25T19:15:27.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7f310b96c640 1 -- 172.21.15.76:0/4131854023 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f30fc047dd0 con 0x7f30fc046510 2024-04-25T19:15:27.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7f6f3bfff640 1 --2- 172.21.15.76:0/4146577706 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6f3c0c9d80 0x7f6f3c1ec7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7f6f40e6e640 1 -- 172.21.15.76:0/4146577706 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6f3c0c9d80 msgr2=0x7f6f3c1ec7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7f6f40e6e640 1 --2- 172.21.15.76:0/4146577706 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6f3c0c9d80 0x7f6f3c1ec7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7f6f40e6e640 1 -- 172.21.15.76:0/4146577706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f3c0ca190 msgr2=0x7f6f3c1fb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7f6f40e6e640 1 --2- 172.21.15.76:0/4146577706 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6f3c0ca190 0x7f6f3c1fb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.038+0000 7f2089842640 1 -- start start 2024-04-25T19:15:27.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7f6f40e6e640 1 -- 172.21.15.76:0/4146577706 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3c15dff0 con 0x7f6f3c1f77e0 2024-04-25T19:15:27.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7fbdab1c5640 1 -- 172.21.15.76:0/411451022 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c046c90 con 0x7fbd9c04c940 2024-04-25T19:15:27.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7f2089842640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2084074300 0x7f20841fa520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7f2089842640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20840c6640 0x7f20840c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7f2089842640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20840cdd60 0x7f208414b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7f2082ffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2084074300 0x7f20841fa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7f2082ffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2084074300 0x7f20841fa520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:45774/0 (socket says 172.21.15.76:45774) 2024-04-25T19:15:27.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7f2082ffd640 1 -- 172.21.15.76:0/737530913 learned_addr learned my addr 172.21.15.76:0/737530913 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:27.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7f2089842640 1 -- 172.21.15.76:0/737530913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f20841610e0 con 0x7f20840cdd60 2024-04-25T19:15:27.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7f20827fc640 1 --2- 172.21.15.76:0/737530913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20840c6640 0x7f20840c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7f2089842640 1 -- 172.21.15.76:0/737530913 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f20840c9440 con 0x7f20840c6640 2024-04-25T19:15:27.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7f2089842640 1 -- 172.21.15.76:0/737530913 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f20841613c0 con 0x7f2084074300 2024-04-25T19:15:27.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7f20837fe640 1 --2- 172.21.15.76:0/737530913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20840cdd60 0x7f208414b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7f4fcb7fe640 1 -- 172.21.15.76:0/3818361545 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4014830 con 0x7f4fe40c56d0 2024-04-25T19:15:27.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7f4fe96b1640 1 -- 172.21.15.76:0/3818361545 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe40c62e0 con 0x7f4fe40c56d0 2024-04-25T19:15:27.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7fbdab1c5640 1 -- 172.21.15.76:0/411451022 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c04bf80 con 0x7fbd9c00e480 2024-04-25T19:15:27.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7fbdab1c5640 1 -- 172.21.15.76:0/411451022 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fbd9c034480 con 0x7fbd9c0471b0 2024-04-25T19:15:27.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f4fe96b1640 1 -- 172.21.15.76:0/3818361545 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe4154a30 con 0x7f4fe40c56d0 2024-04-25T19:15:27.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f784e586640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7848155320 0x7f78481f6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f3109f09640 1 -- 172.21.15.76:0/4131854023 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30fc046510 msgr2=0x7f30fc0c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f3109f09640 1 --2- 172.21.15.76:0/4131854023 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f30fc046510 0x7f30fc0c70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f3109f09640 1 -- 172.21.15.76:0/4131854023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30fc045bb0 msgr2=0x7f30fc0cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f784e586640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78480c8390 0x7f78481f3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f3109f09640 1 --2- 172.21.15.76:0/4131854023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30fc045bb0 0x7f30fc0cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f3109f09640 1 -- 172.21.15.76:0/4131854023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30fc174dc0 con 0x7f30fc04d190 2024-04-25T19:15:27.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f7847fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7848155320 0x7f78481f6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f2082ffd640 1 -- 172.21.15.76:0/737530913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20840c6640 msgr2=0x7f20840c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f2082ffd640 1 --2- 172.21.15.76:0/737530913 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20840c6640 0x7f20840c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f2082ffd640 1 -- 172.21.15.76:0/737530913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20840cdd60 msgr2=0x7f208414b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f7847fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7848155320 0x7f78481f6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:45788/0 (socket says 172.21.15.76:45788) 2024-04-25T19:15:27.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f2082ffd640 1 --2- 172.21.15.76:0/737530913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20840cdd60 0x7f208414b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f2082ffd640 1 -- 172.21.15.76:0/737530913 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208415fe40 con 0x7f2084074300 2024-04-25T19:15:27.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f1b40b2e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b3c074300 0x7f1b3c1ebab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f78477fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78480c8390 0x7f78481f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f1b40b2e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b3c0c6640 0x7f1b3c156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f1b40b2e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b3c0cdd60 0x7f1b3c155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f6f40e6e640 1 --2- 172.21.15.76:0/4146577706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f3c1f77e0 0x7f6f3c14ddc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00c3c0 tx=0x7f6f2c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f78477fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78480c8390 0x7f78481f3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:42548/0 (socket says 172.21.15.76:42548) 2024-04-25T19:15:27.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f784e586640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78480c7740 0x7f784806eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f1b40b2e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c1610e0 con 0x7f1b3c0cdd60 2024-04-25T19:15:27.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f1b40b2e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c0c9440 con 0x7f1b3c0c6640 2024-04-25T19:15:27.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f1b40b2e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f1b3c1613c0 con 0x7f1b3c074300 2024-04-25T19:15:27.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f2082ffd640 1 --2- 172.21.15.76:0/737530913 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2084074300 0x7f20841fa520 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f207c01dbc0 tx=0x7f207c01b440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f7847fff640 1 -- 172.21.15.76:0/4205870920 learned_addr learned my addr 172.21.15.76:0/4205870920 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:27.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f1b3a59c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b3c074300 0x7f1b3c1ebab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f1b33fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b3c0c6640 0x7f1b3c156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f784e586640 1 -- 172.21.15.76:0/4205870920 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f784815ac20 con 0x7f78480c7740 2024-04-25T19:15:27.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f1b3a59c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b3c074300 0x7f1b3c1ebab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:45800/0 (socket says 172.21.15.76:45800) 2024-04-25T19:15:27.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f1b33fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b3c0c6640 0x7f1b3c156960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:42564/0 (socket says 172.21.15.76:42564) 2024-04-25T19:15:27.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f1b3a59c640 1 -- 172.21.15.76:0/169352287 learned_addr learned my addr 172.21.15.76:0/169352287 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:27.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f784cb23640 1 --2- 172.21.15.76:0/4205870920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78480c7740 0x7f784806eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.040+0000 7f4f5759e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4f58073d70 con 0x7f4f5815b220 2024-04-25T19:15:27.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f4f55d9b640 1 -- 172.21.15.76:0/366121253 learned_addr learned my addr 172.21.15.76:0/366121253 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:27.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f4f56d9d640 1 --2- 172.21.15.76:0/366121253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f5815b220 0x7f4f581509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f4f5759e640 1 -- 172.21.15.76:0/366121253 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4f58073bf0 con 0x7f4f58074110 2024-04-25T19:15:27.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f4f5759e640 1 -- 172.21.15.76:0/366121253 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f4f58161b00 con 0x7f4f580ca5d0 2024-04-25T19:15:27.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f4f55d9b640 1 -- 172.21.15.76:0/366121253 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f58074110 msgr2=0x7f4f580c6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f4f55d9b640 1 --2- 172.21.15.76:0/366121253 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f58074110 0x7f4f580c6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f4f55d9b640 1 -- 172.21.15.76:0/366121253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f5815b220 msgr2=0x7f4f581509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7fbda99c2640 1 -- 172.21.15.76:0/411451022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd9c00e480 msgr2=0x7fbd9c0dce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7fbda99c2640 1 --2- 172.21.15.76:0/411451022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd9c00e480 0x7fbd9c0dce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7fbda99c2640 1 -- 172.21.15.76:0/411451022 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd9c04c940 msgr2=0x7fbd9c17d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7fbda99c2640 1 --2- 172.21.15.76:0/411451022 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd9c04c940 0x7fbd9c17d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f4f55d9b640 1 --2- 172.21.15.76:0/366121253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f5815b220 0x7f4f581509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.041+0000 7f4f55d9b640 1 -- 172.21.15.76:0/366121253 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f581ecd80 con 0x7f4f580ca5d0 2024-04-25T19:15:27.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f784e586640 1 -- 172.21.15.76:0/4205870920 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f78480c70f0 con 0x7f78480c8390 2024-04-25T19:15:27.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7fbda99c2640 1 -- 172.21.15.76:0/411451022 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9c0dd3c0 con 0x7fbd9c0471b0 2024-04-25T19:15:27.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7fbdaa1c3640 1 --2- 172.21.15.76:0/411451022 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd9c00e480 0x7fbd9c0dce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:27.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f784e586640 1 -- 172.21.15.76:0/4205870920 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f78480cbe10 con 0x7f7848155320 2024-04-25T19:15:27.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f1b3ad9d640 1 --2- 172.21.15.76:0/169352287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b3c0cdd60 0x7f1b3c155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7fbdaa9c4640 1 --2- 172.21.15.76:0/411451022 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd9c04c940 0x7fbd9c17d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:27.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f4fe96b1640 1 -- 172.21.15.76:0/3818361545 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4fe40c6a70 con 0x7f4fe40c56d0 2024-04-25T19:15:27.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f4f5659c640 1 --2- 172.21.15.76:0/366121253 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f58074110 0x7f4f580c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-25T19:15:27.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f3109f09640 1 --2- 172.21.15.76:0/4131854023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30fc04d190 0x7f30fc0c2450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f310406c350 tx=0x7f3104078200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f206bfff640 1 -- 172.21.15.76:0/737530913 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c01bb50 con 0x7f2084074300 2024-04-25T19:15:27.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f206bfff640 1 -- 172.21.15.76:0/737530913 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f207c01bcf0 con 0x7f2084074300 2024-04-25T19:15:27.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f206bfff640 1 -- 172.21.15.76:0/737530913 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c023610 con 0x7f2084074300 2024-04-25T19:15:27.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f2089842640 1 -- 172.21.15.76:0/737530913 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20841faa60 con 0x7f2084074300 2024-04-25T19:15:27.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f2089842640 1 -- 172.21.15.76:0/737530913 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f208415f400 con 0x7f2084074300 2024-04-25T19:15:27.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f4f55d9b640 1 --2- 172.21.15.76:0/366121253 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4f580ca5d0 0x7f4f581ec330 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4f48000c00 tx=0x7f4f480046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7fbda99c2640 1 --2- 172.21.15.76:0/411451022 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbd9c0471b0 0x7fbd9c17cfd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c046bc0 tx=0x7fbd940046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f1b33fff640 1 -- 172.21.15.76:0/169352287 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b3c074300 msgr2=0x7f1b3c1ebab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f1b33fff640 1 --2- 172.21.15.76:0/169352287 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1b3c074300 0x7f1b3c1ebab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f1b33fff640 1 -- 172.21.15.76:0/169352287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b3c0cdd60 msgr2=0x7f1b3c155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f1b33fff640 1 --2- 172.21.15.76:0/169352287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b3c0cdd60 0x7f1b3c155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f1b33fff640 1 -- 172.21.15.76:0/169352287 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3c1ebff0 con 0x7f1b3c0c6640 2024-04-25T19:15:27.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f7847fff640 1 -- 172.21.15.76:0/4205870920 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78480c8390 msgr2=0x7f78481f3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f7847fff640 1 --2- 172.21.15.76:0/4205870920 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f78480c8390 0x7f78481f3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f7847fff640 1 -- 172.21.15.76:0/4205870920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78480c7740 msgr2=0x7f784806eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f7847fff640 1 --2- 172.21.15.76:0/4205870920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78480c7740 0x7f784806eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f7847fff640 1 -- 172.21.15.76:0/4205870920 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78481f2c90 con 0x7f7848155320 2024-04-25T19:15:27.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f1b33fff640 1 --2- 172.21.15.76:0/169352287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b3c0c6640 0x7f1b3c156960 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c06e150 tx=0x7f1b2400b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f30f9ffb640 1 -- 172.21.15.76:0/4131854023 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310406db50 con 0x7f30fc04d190 2024-04-25T19:15:27.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f310b96c640 1 -- 172.21.15.76:0/4131854023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30fc0c8260 con 0x7f30fc04d190 2024-04-25T19:15:27.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f310b96c640 1 -- 172.21.15.76:0/4131854023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30fc0ca5c0 con 0x7f30fc04d190 2024-04-25T19:15:27.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.043+0000 7f30f9ffb640 1 -- 172.21.15.76:0/4131854023 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3104092030 con 0x7f30fc04d190 2024-04-25T19:15:27.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.043+0000 7f30f9ffb640 1 -- 172.21.15.76:0/4131854023 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310408e450 con 0x7f30fc04d190 2024-04-25T19:15:27.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.043+0000 7f2089842640 1 -- 172.21.15.76:0/737530913 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f208406a390 con 0x7f2084074300 2024-04-25T19:15:27.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.043+0000 7f4fcb7fe640 1 -- 172.21.15.76:0/3818361545 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4fd401c070 con 0x7f4fe40c56d0 2024-04-25T19:15:27.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.043+0000 7f310b96c640 1 -- 172.21.15.76:0/4131854023 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f30e4001030 con 0x7f30fc04d190 2024-04-25T19:15:27.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.043+0000 7f4f46ffd640 1 -- 172.21.15.76:0/366121253 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f48041020 con 0x7f4f580ca5d0 2024-04-25T19:15:27.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.044+0000 7f4fcb7fe640 1 --2- 172.21.15.76:0/3818361545 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4fd0074580 0x7f4fd0076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.044+0000 7f4f46ffd640 1 -- 172.21.15.76:0/366121253 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f48030970 con 0x7f4f580ca5d0 2024-04-25T19:15:27.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.044+0000 7f4f5759e640 1 -- 172.21.15.76:0/366121253 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f581faed0 con 0x7f4f580ca5d0 2024-04-25T19:15:27.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.044+0000 7f4f5759e640 1 -- 172.21.15.76:0/366121253 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f581f7170 con 0x7f4f580ca5d0 2024-04-25T19:15:27.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.044+0000 7f4f46ffd640 1 -- 172.21.15.76:0/366121253 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f48037040 con 0x7f4f580ca5d0 2024-04-25T19:15:27.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.044+0000 7fbd9affd640 1 -- 172.21.15.76:0/411451022 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94037070 con 0x7fbd9c0471b0 2024-04-25T19:15:27.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.044+0000 7fbd9affd640 1 -- 172.21.15.76:0/411451022 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd94038040 con 0x7fbd9c0471b0 2024-04-25T19:15:27.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.044+0000 7fbd9affd640 1 -- 172.21.15.76:0/411451022 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94004030 con 0x7fbd9c0471b0 2024-04-25T19:15:27.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.044+0000 7f4fe2ffd640 1 --2- 172.21.15.76:0/3818361545 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4fd0074580 0x7f4fd0076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f6f38ff9640 1 -- 172.21.15.76:0/4146577706 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c017070 con 0x7f6f3c1f77e0 2024-04-25T19:15:27.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f6f38ff9640 1 -- 172.21.15.76:0/4146577706 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f2c005ce0 con 0x7f6f3c1f77e0 2024-04-25T19:15:27.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f6f38ff9640 1 -- 172.21.15.76:0/4146577706 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0072f0 con 0x7f6f3c1f77e0 2024-04-25T19:15:27.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f6f428d1640 1 -- 172.21.15.76:0/4146577706 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f3c1eac70 con 0x7f6f3c1f77e0 2024-04-25T19:15:27.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.042+0000 7f6f428d1640 1 -- 172.21.15.76:0/4146577706 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f3c15e290 con 0x7f6f3c1f77e0 2024-04-25T19:15:27.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.044+0000 7f6f428d1640 1 -- 172.21.15.76:0/4146577706 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6f3c06a390 con 0x7f6f3c1f77e0 2024-04-25T19:15:27.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.044+0000 7f4fcb7fe640 1 -- 172.21.15.76:0/3818361545 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4fd40951e0 con 0x7f4fe40c56d0 2024-04-25T19:15:27.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.044+0000 7f1b32ffd640 1 -- 172.21.15.76:0/169352287 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24015020 con 0x7f1b3c0c6640 2024-04-25T19:15:27.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.044+0000 7f1b32ffd640 1 -- 172.21.15.76:0/169352287 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b24011830 con 0x7f1b3c0c6640 2024-04-25T19:15:27.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.044+0000 7f1b32ffd640 1 -- 172.21.15.76:0/169352287 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24011c00 con 0x7f1b3c0c6640 2024-04-25T19:15:27.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.044+0000 7f1b40b2e640 1 -- 172.21.15.76:0/169352287 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b3c0cd750 con 0x7f1b3c0c6640 2024-04-25T19:15:27.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.044+0000 7f7847fff640 1 --2- 172.21.15.76:0/4205870920 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7848155320 0x7f78481f6c10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f78400052e0 tx=0x7f78400057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.045+0000 7f4fcb7fe640 1 --2- 172.21.15.76:0/3818361545 >> [v2:172.21.15.76:6800/1302663821,v1:172.21.15.76:6801/1302663821] conn(0x7f4fd007c850 0x7f4fd007eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.045+0000 7f4fe2ffd640 1 --2- 172.21.15.76:0/3818361545 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4fd0074580 0x7f4fd0076a90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009340 tx=0x7f4fdc0092d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.045+0000 7f4fe27fc640 1 --2- 172.21.15.76:0/3818361545 >> [v2:172.21.15.76:6800/1302663821,v1:172.21.15.76:6801/1302663821] conn(0x7f4fd007c850 0x7f4fd007eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.045+0000 7f7844ff9640 1 -- 172.21.15.76:0/4205870920 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7840017020 con 0x7f7848155320 2024-04-25T19:15:27.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.045+0000 7f7844ff9640 1 -- 172.21.15.76:0/4205870920 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f784000ab50 con 0x7f7848155320 2024-04-25T19:15:27.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.045+0000 7f7844ff9640 1 -- 172.21.15.76:0/4205870920 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78400133f0 con 0x7f7848155320 2024-04-25T19:15:27.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.045+0000 7f784e586640 1 -- 172.21.15.76:0/4205870920 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7848154460 con 0x7f7848155320 2024-04-25T19:15:27.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.045+0000 7f784e586640 1 -- 172.21.15.76:0/4205870920 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f78481f2f50 con 0x7f7848155320 2024-04-25T19:15:27.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.045+0000 7f1b40b2e640 1 -- 172.21.15.76:0/169352287 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b3c0cbb70 con 0x7f1b3c0c6640 2024-04-25T19:15:27.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.045+0000 7f4fcb7fe640 1 -- 172.21.15.76:0/3818361545 --> [v2:172.21.15.76:6800/1302663821,v1:172.21.15.76:6801/1302663821] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4fd007f370 con 0x7f4fd007c850 2024-04-25T19:15:27.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.045+0000 7f6f38ff9640 1 -- 172.21.15.76:0/4146577706 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6f2c01a070 con 0x7f6f3c1f77e0 2024-04-25T19:15:27.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.046+0000 7f4fcb7fe640 1 -- 172.21.15.76:0/3818361545 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4fd40955d0 con 0x7f4fe40c56d0 2024-04-25T19:15:27.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.045+0000 7f4f5759e640 1 -- 172.21.15.76:0/366121253 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4f58050740 con 0x7f4f580ca5d0 2024-04-25T19:15:27.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.046+0000 7f4fe27fc640 1 --2- 172.21.15.76:0/3818361545 >> [v2:172.21.15.76:6800/1302663821,v1:172.21.15.76:6801/1302663821] conn(0x7f4fd007c850 0x7f4fd007eca0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.046+0000 7f6f38ff9640 1 --2- 172.21.15.76:0/4146577706 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f6f1c074500 0x7f6f1c076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.046+0000 7fbdab1c5640 1 -- 172.21.15.76:0/411451022 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd9c04a130 con 0x7fbd9c0471b0 2024-04-25T19:15:27.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.046+0000 7fbdab1c5640 1 -- 172.21.15.76:0/411451022 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd9c0dc680 con 0x7fbd9c0471b0 2024-04-25T19:15:27.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.046+0000 7f1b30ff9640 1 -- 172.21.15.76:0/169352287 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1b14001030 con 0x7f1b3c0c6640 2024-04-25T19:15:27.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.046+0000 7f6f38ff9640 1 -- 172.21.15.76:0/4146577706 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6f2c094ab0 con 0x7f6f3c1f77e0 2024-04-25T19:15:27.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.047+0000 7f7844ff9640 1 -- 172.21.15.76:0/4205870920 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f784001a020 con 0x7f7848155320 2024-04-25T19:15:27.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.044+0000 7f206bfff640 1 -- 172.21.15.76:0/737530913 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f207c0237b0 con 0x7f2084074300 2024-04-25T19:15:27.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.045+0000 7f206bfff640 1 --2- 172.21.15.76:0/737530913 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f2070074580 0x7f2070076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.045+0000 7f20827fc640 1 --2- 172.21.15.76:0/737530913 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f2070074580 0x7f2070076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.045+0000 7f206bfff640 1 -- 172.21.15.76:0/737530913 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f207c0a6a50 con 0x7f2084074300 2024-04-25T19:15:27.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.047+0000 7f206bfff640 1 --2- 172.21.15.76:0/737530913 >> [v2:172.21.15.105:6800/2125002468,v1:172.21.15.105:6801/2125002468] conn(0x7f207007c850 0x7f207007eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.047+0000 7f206bfff640 1 -- 172.21.15.76:0/737530913 --> [v2:172.21.15.105:6800/2125002468,v1:172.21.15.105:6801/2125002468] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f207c02be50 con 0x7f207007c850 2024-04-25T19:15:27.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.047+0000 7f20827fc640 1 --2- 172.21.15.76:0/737530913 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f2070074580 0x7f2070076a90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2078034000 tx=0x7f207802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.047+0000 7f206bfff640 1 -- 172.21.15.76:0/737530913 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f207c0a6e40 con 0x7f2084074300 2024-04-25T19:15:27.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.047+0000 7f4f46ffd640 1 -- 172.21.15.76:0/366121253 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4f48044020 con 0x7f4f580ca5d0 2024-04-25T19:15:27.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.047+0000 7fbdab1c5640 1 -- 172.21.15.76:0/411451022 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbd9c17da50 con 0x7fbd9c0471b0 2024-04-25T19:15:27.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.047+0000 7f6f38ff9640 1 --2- 172.21.15.76:0/4146577706 >> [v2:172.21.15.105:6817/3413103121,v1:172.21.15.105:6819/3413103121] conn(0x7f6f1c07c7d0 0x7f6f1c07ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.047+0000 7f6f38ff9640 1 -- 172.21.15.76:0/4146577706 --> [v2:172.21.15.105:6817/3413103121,v1:172.21.15.105:6819/3413103121] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6f2c01dec0 con 0x7f6f1c07c7d0 2024-04-25T19:15:27.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.047+0000 7f6f38ff9640 1 -- 172.21.15.76:0/4146577706 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6f2c094f90 con 0x7f6f3c1f77e0 2024-04-25T19:15:27.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.047+0000 7f784e586640 1 -- 172.21.15.76:0/4205870920 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7848154210 con 0x7f7848155320 2024-04-25T19:15:27.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.047+0000 7f20837fe640 1 --2- 172.21.15.76:0/737530913 >> [v2:172.21.15.105:6800/2125002468,v1:172.21.15.105:6801/2125002468] conn(0x7f207007c850 0x7f207007eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.047+0000 7f6f3bfff640 1 --2- 172.21.15.76:0/4146577706 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f6f1c074500 0x7f6f1c076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.047+0000 7f6f3b7fe640 1 --2- 172.21.15.76:0/4146577706 >> [v2:172.21.15.105:6817/3413103121,v1:172.21.15.105:6819/3413103121] conn(0x7f6f1c07c7d0 0x7f6f1c07ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.047+0000 7f6f3b7fe640 1 --2- 172.21.15.76:0/4146577706 >> [v2:172.21.15.105:6817/3413103121,v1:172.21.15.105:6819/3413103121] conn(0x7f6f1c07c7d0 0x7f6f1c07ec20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.047+0000 7f7844ff9640 1 --2- 172.21.15.76:0/4205870920 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f7834074530 0x7f7834076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.048+0000 7f30f9ffb640 1 -- 172.21.15.76:0/4131854023 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f310408e5f0 con 0x7f30fc04d190 2024-04-25T19:15:27.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.048+0000 7f7844ff9640 1 -- 172.21.15.76:0/4205870920 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7840095880 con 0x7f7848155320 2024-04-25T19:15:27.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.048+0000 7f78477fe640 1 --2- 172.21.15.76:0/4205870920 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f7834074530 0x7f7834076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.048+0000 7f1b32ffd640 1 -- 172.21.15.76:0/169352287 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1b24018020 con 0x7f1b3c0c6640 2024-04-25T19:15:27.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.048+0000 7f4f46ffd640 1 --2- 172.21.15.76:0/366121253 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4f38074580 0x7f4f38076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.048+0000 7f7844ff9640 1 --2- 172.21.15.76:0/4205870920 >> [v2:172.21.15.76:6808/1992938181,v1:172.21.15.76:6809/1992938181] conn(0x7f783407c780 0x7f783407ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.048+0000 7f20837fe640 1 --2- 172.21.15.76:0/737530913 >> [v2:172.21.15.105:6800/2125002468,v1:172.21.15.105:6801/2125002468] conn(0x7f207007c850 0x7f207007eca0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.048+0000 7f4f46ffd640 1 -- 172.21.15.76:0/366121253 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4f480b5cd0 con 0x7f4f580ca5d0 2024-04-25T19:15:27.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.048+0000 7f7844ff9640 1 -- 172.21.15.76:0/4205870920 --> [v2:172.21.15.76:6808/1992938181,v1:172.21.15.76:6809/1992938181] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f783407f2a0 con 0x7f783407c780 2024-04-25T19:15:27.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.048+0000 7f4f5659c640 1 --2- 172.21.15.76:0/366121253 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4f38074580 0x7f4f38076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.048+0000 7f4f46ffd640 1 --2- 172.21.15.76:0/366121253 >> [v2:172.21.15.105:6816/548350430,v1:172.21.15.105:6818/548350430] conn(0x7f4f3807c7f0 0x7f4f3807ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.048+0000 7f784cb23640 1 --2- 172.21.15.76:0/4205870920 >> [v2:172.21.15.76:6808/1992938181,v1:172.21.15.76:6809/1992938181] conn(0x7f783407c780 0x7f783407ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.048+0000 7f4f46ffd640 1 -- 172.21.15.76:0/366121253 --> [v2:172.21.15.105:6816/548350430,v1:172.21.15.105:6818/548350430] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4f48006810 con 0x7f4f3807c7f0 2024-04-25T19:15:27.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.048+0000 7f7844ff9640 1 -- 172.21.15.76:0/4205870920 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f784005bff0 con 0x7f7848155320 2024-04-25T19:15:27.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.048+0000 7f4f46ffd640 1 -- 172.21.15.76:0/366121253 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4f480b8330 con 0x7f4f580ca5d0 2024-04-25T19:15:27.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.048+0000 7f6f3bfff640 1 --2- 172.21.15.76:0/4146577706 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f6f1c074500 0x7f6f1c076a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6f34034000 tx=0x7f6f3402f000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.048+0000 7f30f9ffb640 1 --2- 172.21.15.76:0/4131854023 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f30e80744e0 0x7f30e80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.048+0000 7f1b32ffd640 1 --2- 172.21.15.76:0/169352287 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1b1c074580 0x7f1b1c076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.048+0000 7f4f56d9d640 1 --2- 172.21.15.76:0/366121253 >> [v2:172.21.15.105:6816/548350430,v1:172.21.15.105:6818/548350430] conn(0x7f4f3807c7f0 0x7f4f3807ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.048+0000 7f6f38ff9640 1 -- 172.21.15.76:0/4146577706 <== osd.5 v2:172.21.15.105:6817/3413103121 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f2c01dec0 con 0x7f6f1c07c7d0 2024-04-25T19:15:27.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.049+0000 7f30f9ffb640 1 -- 172.21.15.76:0/4131854023 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3104111fc0 con 0x7f30fc04d190 2024-04-25T19:15:27.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.049+0000 7f3109708640 1 --2- 172.21.15.76:0/4131854023 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f30e80744e0 0x7f30e80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.049+0000 7f78477fe640 1 --2- 172.21.15.76:0/4205870920 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f7834074530 0x7f7834076a40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f78300057e0 tx=0x7f783c030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.049+0000 7f1b32ffd640 1 -- 172.21.15.76:0/169352287 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1b24095020 con 0x7f1b3c0c6640 2024-04-25T19:15:27.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.049+0000 7f1b3a59c640 1 --2- 172.21.15.76:0/169352287 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1b1c074580 0x7f1b1c076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.049+0000 7f30f9ffb640 1 --2- 172.21.15.76:0/4131854023 >> [v2:172.21.15.76:6816/1409574998,v1:172.21.15.76:6817/1409574998] conn(0x7f30e807c730 0x7f30e807eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.049+0000 7fbd9affd640 1 -- 172.21.15.76:0/411451022 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbd940041d0 con 0x7fbd9c0471b0 2024-04-25T19:15:27.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.049+0000 7f1b32ffd640 1 --2- 172.21.15.76:0/169352287 >> [v2:172.21.15.105:6808/2721832954,v1:172.21.15.105:6809/2721832954] conn(0x7f1b1c07c850 0x7f1b1c07eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.049+0000 7f4f56d9d640 1 --2- 172.21.15.76:0/366121253 >> [v2:172.21.15.105:6816/548350430,v1:172.21.15.105:6818/548350430] conn(0x7f4f3807c7f0 0x7f4f3807ec40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.049+0000 7f1b32ffd640 1 -- 172.21.15.76:0/169352287 --> [v2:172.21.15.105:6808/2721832954,v1:172.21.15.105:6809/2721832954] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1b2400de50 con 0x7f1b1c07c850 2024-04-25T19:15:27.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.049+0000 7f4f5659c640 1 --2- 172.21.15.76:0/366121253 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4f38074580 0x7f4f38076a90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4f58070440 tx=0x7f4f4c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.049+0000 7f1b32ffd640 1 -- 172.21.15.76:0/169352287 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1b2409c050 con 0x7f1b3c0c6640 2024-04-25T19:15:27.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.049+0000 7f1b3ad9d640 1 --2- 172.21.15.76:0/169352287 >> [v2:172.21.15.105:6808/2721832954,v1:172.21.15.105:6809/2721832954] conn(0x7f1b1c07c850 0x7f1b1c07eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.049+0000 7f3108f07640 1 --2- 172.21.15.76:0/4131854023 >> [v2:172.21.15.76:6816/1409574998,v1:172.21.15.76:6817/1409574998] conn(0x7f30e807c730 0x7f30e807eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.049+0000 7f3109708640 1 --2- 172.21.15.76:0/4131854023 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f30e80744e0 0x7f30e80769f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3100034000 tx=0x7f310002c000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.049+0000 7f784cb23640 1 --2- 172.21.15.76:0/4205870920 >> [v2:172.21.15.76:6808/1992938181,v1:172.21.15.76:6809/1992938181] conn(0x7f783407c780 0x7f783407ebd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.049+0000 7f206bfff640 1 -- 172.21.15.76:0/737530913 <== osd.4 v2:172.21.15.105:6800/2125002468 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f207c02be50 con 0x7f207007c850 2024-04-25T19:15:27.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.049+0000 7fbd9affd640 1 --2- 172.21.15.76:0/411451022 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbd880745b0 0x7fbd88076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.050+0000 7f1b3a59c640 1 --2- 172.21.15.76:0/169352287 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1b1c074580 0x7f1b1c076a90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c006f40 tx=0x7f1b2c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.050+0000 7fbd9affd640 1 -- 172.21.15.76:0/411451022 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbd940b5070 con 0x7fbd9c0471b0 2024-04-25T19:15:27.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.050+0000 7fbdaa1c3640 1 --2- 172.21.15.76:0/411451022 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbd880745b0 0x7fbd88076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.050+0000 7fbd9affd640 1 --2- 172.21.15.76:0/411451022 >> [v2:172.21.15.76:6824/3418257163,v1:172.21.15.76:6825/3418257163] conn(0x7fbd8807c880 0x7fbd8807ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.050+0000 7f1b3ad9d640 1 --2- 172.21.15.76:0/169352287 >> [v2:172.21.15.105:6808/2721832954,v1:172.21.15.105:6809/2721832954] conn(0x7f1b1c07c850 0x7f1b1c07eca0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.050+0000 7f4f46ffd640 1 -- 172.21.15.76:0/366121253 <== osd.7 v2:172.21.15.105:6816/548350430 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4f48006810 con 0x7f4f3807c7f0 2024-04-25T19:15:27.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.050+0000 7fbd9affd640 1 -- 172.21.15.76:0/411451022 --> [v2:172.21.15.76:6824/3418257163,v1:172.21.15.76:6825/3418257163] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbd94033e50 con 0x7fbd8807c880 2024-04-25T19:15:27.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.050+0000 7fbd9affd640 1 -- 172.21.15.76:0/411451022 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbd940b5460 con 0x7fbd9c0471b0 2024-04-25T19:15:27.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.050+0000 7fbdaa9c4640 1 --2- 172.21.15.76:0/411451022 >> [v2:172.21.15.76:6824/3418257163,v1:172.21.15.76:6825/3418257163] conn(0x7fbd8807c880 0x7fbd8807ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.050+0000 7f30f9ffb640 1 -- 172.21.15.76:0/4131854023 --> [v2:172.21.15.76:6816/1409574998,v1:172.21.15.76:6817/1409574998] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f30e807f250 con 0x7f30e807c730 2024-04-25T19:15:27.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.050+0000 7f3108f07640 1 --2- 172.21.15.76:0/4131854023 >> [v2:172.21.15.76:6816/1409574998,v1:172.21.15.76:6817/1409574998] conn(0x7f30e807c730 0x7f30e807eb80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.050+0000 7f30f9ffb640 1 -- 172.21.15.76:0/4131854023 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f3104112370 con 0x7f30fc04d190 2024-04-25T19:15:27.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.050+0000 7f7844ff9640 1 -- 172.21.15.76:0/4205870920 <== osd.2 v2:172.21.15.76:6808/1992938181 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f783407f2a0 con 0x7f783407c780 2024-04-25T19:15:27.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.050+0000 7fbdaa1c3640 1 --2- 172.21.15.76:0/411451022 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbd880745b0 0x7fbd88076ac0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbda00095e0 tx=0x7fbda0009190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.051+0000 7fbdaa9c4640 1 --2- 172.21.15.76:0/411451022 >> [v2:172.21.15.76:6824/3418257163,v1:172.21.15.76:6825/3418257163] conn(0x7fbd8807c880 0x7fbd8807ecd0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.051+0000 7f1b32ffd640 1 -- 172.21.15.76:0/169352287 <== osd.6 v2:172.21.15.105:6808/2721832954 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1b2400de50 con 0x7f1b1c07c850 2024-04-25T19:15:27.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.051+0000 7f30f9ffb640 1 -- 172.21.15.76:0/4131854023 <== osd.0 v2:172.21.15.76:6816/1409574998 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f30e807f250 con 0x7f30e807c730 2024-04-25T19:15:27.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.052+0000 7fbd9affd640 1 -- 172.21.15.76:0/411451022 <== osd.3 v2:172.21.15.76:6824/3418257163 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbd94033e50 con 0x7fbd8807c880 2024-04-25T19:15:27.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.066+0000 7f310b96c640 1 -- 172.21.15.76:0/4131854023 --> [v2:172.21.15.76:6816/1409574998,v1:172.21.15.76:6817/1409574998] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f30e4002dd0 con 0x7f30e807c730 2024-04-25T19:15:27.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.066+0000 7f30f9ffb640 1 -- 172.21.15.76:0/4131854023 <== osd.0 v2:172.21.15.76:6816/1409574998 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f30e4002dd0 con 0x7f30e807c730 2024-04-25T19:15:27.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.066+0000 7f310b96c640 1 -- 172.21.15.76:0/4131854023 >> [v2:172.21.15.76:6816/1409574998,v1:172.21.15.76:6817/1409574998] conn(0x7f30e807c730 msgr2=0x7f30e807eb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.066+0000 7f310b96c640 1 --2- 172.21.15.76:0/4131854023 >> [v2:172.21.15.76:6816/1409574998,v1:172.21.15.76:6817/1409574998] conn(0x7f30e807c730 0x7f30e807eb80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.067+0000 7f310b96c640 1 -- 172.21.15.76:0/4131854023 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f30e80744e0 msgr2=0x7f30e80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.067+0000 7f310b96c640 1 --2- 172.21.15.76:0/4131854023 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f30e80744e0 0x7f30e80769f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3100034000 tx=0x7f310002c000 comp rx=0 tx=0).stop 2024-04-25T19:15:27.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.067+0000 7f310b96c640 1 -- 172.21.15.76:0/4131854023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30fc04d190 msgr2=0x7f30fc0c2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.067+0000 7f310b96c640 1 --2- 172.21.15.76:0/4131854023 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30fc04d190 0x7f30fc0c2450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f310406c350 tx=0x7f3104078200 comp rx=0 tx=0).stop 2024-04-25T19:15:27.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.067+0000 7f3109f09640 1 -- 172.21.15.76:0/4131854023 reap_dead start 2024-04-25T19:15:27.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.067+0000 7f310b96c640 1 -- 172.21.15.76:0/4131854023 shutdown_connections 2024-04-25T19:15:27.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.067+0000 7f310b96c640 1 -- 172.21.15.76:0/4131854023 >> 172.21.15.76:0/4131854023 conn(0x7f30fc050c10 msgr2=0x7f30fc0db000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.067+0000 7f310b96c640 1 -- 172.21.15.76:0/4131854023 shutdown_connections 2024-04-25T19:15:27.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.067+0000 7f310b96c640 1 -- 172.21.15.76:0/4131854023 wait complete. 2024-04-25T19:15:27.151 INFO:teuthology.orchestra.run.smithi076.stdout:55834574853 2024-04-25T19:15:27.151 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T19:15:27.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.081+0000 7f6f428d1640 1 -- 172.21.15.76:0/4146577706 --> [v2:172.21.15.105:6817/3413103121,v1:172.21.15.105:6819/3413103121] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6f3c1eaa20 con 0x7f6f1c07c7d0 2024-04-25T19:15:27.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.082+0000 7f2089842640 1 -- 172.21.15.76:0/737530913 --> [v2:172.21.15.105:6800/2125002468,v1:172.21.15.105:6801/2125002468] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f20840740e0 con 0x7f207007c850 2024-04-25T19:15:27.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.082+0000 7f6f38ff9640 1 -- 172.21.15.76:0/4146577706 <== osd.5 v2:172.21.15.105:6817/3413103121 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6f3c1eaa20 con 0x7f6f1c07c7d0 2024-04-25T19:15:27.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.082+0000 7f4f5759e640 1 -- 172.21.15.76:0/366121253 --> [v2:172.21.15.105:6816/548350430,v1:172.21.15.105:6818/548350430] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4f581fb220 con 0x7f4f3807c7f0 2024-04-25T19:15:27.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.082+0000 7f206bfff640 1 -- 172.21.15.76:0/737530913 <== osd.4 v2:172.21.15.105:6800/2125002468 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f20840740e0 con 0x7f207007c850 2024-04-25T19:15:27.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.082+0000 7f6f428d1640 1 -- 172.21.15.76:0/4146577706 >> [v2:172.21.15.105:6817/3413103121,v1:172.21.15.105:6819/3413103121] conn(0x7f6f1c07c7d0 msgr2=0x7f6f1c07ec20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.082+0000 7f6f428d1640 1 --2- 172.21.15.76:0/4146577706 >> [v2:172.21.15.105:6817/3413103121,v1:172.21.15.105:6819/3413103121] conn(0x7f6f1c07c7d0 0x7f6f1c07ec20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.082+0000 7f6f428d1640 1 -- 172.21.15.76:0/4146577706 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f6f1c074500 msgr2=0x7f6f1c076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.082+0000 7f6f428d1640 1 --2- 172.21.15.76:0/4146577706 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f6f1c074500 0x7f6f1c076a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6f34034000 tx=0x7f6f3402f000 comp rx=0 tx=0).stop 2024-04-25T19:15:27.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.082+0000 7f6f428d1640 1 -- 172.21.15.76:0/4146577706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f3c1f77e0 msgr2=0x7f6f3c14ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.082+0000 7f6f428d1640 1 --2- 172.21.15.76:0/4146577706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f3c1f77e0 0x7f6f3c14ddc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00c3c0 tx=0x7f6f2c00c880 comp rx=0 tx=0).stop 2024-04-25T19:15:27.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.082+0000 7f6f40e6e640 1 -- 172.21.15.76:0/4146577706 reap_dead start 2024-04-25T19:15:27.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.082+0000 7f6f428d1640 1 -- 172.21.15.76:0/4146577706 shutdown_connections 2024-04-25T19:15:27.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.082+0000 7f6f428d1640 1 -- 172.21.15.76:0/4146577706 >> 172.21.15.76:0/4146577706 conn(0x7f6f3c070e70 msgr2=0x7f6f3c0ca5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.082+0000 7f4f46ffd640 1 -- 172.21.15.76:0/366121253 <== osd.7 v2:172.21.15.105:6816/548350430 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4f581fb220 con 0x7f4f3807c7f0 2024-04-25T19:15:27.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.082+0000 7f6f428d1640 1 -- 172.21.15.76:0/4146577706 shutdown_connections 2024-04-25T19:15:27.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.082+0000 7f6f428d1640 1 -- 172.21.15.76:0/4146577706 wait complete. 2024-04-25T19:15:27.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.083+0000 7f2089842640 1 -- 172.21.15.76:0/737530913 >> [v2:172.21.15.105:6800/2125002468,v1:172.21.15.105:6801/2125002468] conn(0x7f207007c850 msgr2=0x7f207007eca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.083+0000 7f2089842640 1 --2- 172.21.15.76:0/737530913 >> [v2:172.21.15.105:6800/2125002468,v1:172.21.15.105:6801/2125002468] conn(0x7f207007c850 0x7f207007eca0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.083+0000 7f2089842640 1 -- 172.21.15.76:0/737530913 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f2070074580 msgr2=0x7f2070076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.083+0000 7f2089842640 1 --2- 172.21.15.76:0/737530913 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f2070074580 0x7f2070076a90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2078034000 tx=0x7f207802e000 comp rx=0 tx=0).stop 2024-04-25T19:15:27.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.083+0000 7f2089842640 1 -- 172.21.15.76:0/737530913 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2084074300 msgr2=0x7f20841fa520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.083+0000 7f2089842640 1 --2- 172.21.15.76:0/737530913 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2084074300 0x7f20841fa520 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f207c01dbc0 tx=0x7f207c01b440 comp rx=0 tx=0).stop 2024-04-25T19:15:27.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.083+0000 7f20837fe640 1 -- 172.21.15.76:0/737530913 reap_dead start 2024-04-25T19:15:27.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.083+0000 7f2089842640 1 -- 172.21.15.76:0/737530913 shutdown_connections 2024-04-25T19:15:27.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.083+0000 7f2089842640 1 -- 172.21.15.76:0/737530913 >> 172.21.15.76:0/737530913 conn(0x7f2084070800 msgr2=0x7f20840ce170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.083+0000 7f4f5759e640 1 -- 172.21.15.76:0/366121253 >> [v2:172.21.15.105:6816/548350430,v1:172.21.15.105:6818/548350430] conn(0x7f4f3807c7f0 msgr2=0x7f4f3807ec40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.083+0000 7f2089842640 1 -- 172.21.15.76:0/737530913 shutdown_connections 2024-04-25T19:15:27.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.083+0000 7f4f5759e640 1 --2- 172.21.15.76:0/366121253 >> [v2:172.21.15.105:6816/548350430,v1:172.21.15.105:6818/548350430] conn(0x7f4f3807c7f0 0x7f4f3807ec40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.083+0000 7f2089842640 1 -- 172.21.15.76:0/737530913 wait complete. 2024-04-25T19:15:27.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.083+0000 7f4f5759e640 1 -- 172.21.15.76:0/366121253 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4f38074580 msgr2=0x7f4f38076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.083+0000 7f4f5759e640 1 --2- 172.21.15.76:0/366121253 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4f38074580 0x7f4f38076a90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4f58070440 tx=0x7f4f4c008040 comp rx=0 tx=0).stop 2024-04-25T19:15:27.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.083+0000 7f4f5759e640 1 -- 172.21.15.76:0/366121253 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4f580ca5d0 msgr2=0x7f4f581ec330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.083+0000 7f4f5759e640 1 --2- 172.21.15.76:0/366121253 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4f580ca5d0 0x7f4f581ec330 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4f48000c00 tx=0x7f4f480046c0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.083+0000 7f4f56d9d640 1 -- 172.21.15.76:0/366121253 reap_dead start 2024-04-25T19:15:27.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.083+0000 7f4f5759e640 1 -- 172.21.15.76:0/366121253 shutdown_connections 2024-04-25T19:15:27.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.083+0000 7f4f5759e640 1 -- 172.21.15.76:0/366121253 >> 172.21.15.76:0/366121253 conn(0x7f4f580cae80 msgr2=0x7f4f58070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.083+0000 7f784e586640 1 -- 172.21.15.76:0/4205870920 --> [v2:172.21.15.76:6808/1992938181,v1:172.21.15.76:6809/1992938181] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f784806a390 con 0x7f783407c780 2024-04-25T19:15:27.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.083+0000 7f7844ff9640 1 -- 172.21.15.76:0/4205870920 <== osd.2 v2:172.21.15.76:6808/1992938181 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f784806a390 con 0x7f783407c780 2024-04-25T19:15:27.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.084+0000 7f4f5759e640 1 -- 172.21.15.76:0/366121253 shutdown_connections 2024-04-25T19:15:27.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.084+0000 7f4f5759e640 1 -- 172.21.15.76:0/366121253 wait complete. 2024-04-25T19:15:27.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.084+0000 7f782e7fc640 1 -- 172.21.15.76:0/4205870920 >> [v2:172.21.15.76:6808/1992938181,v1:172.21.15.76:6809/1992938181] conn(0x7f783407c780 msgr2=0x7f783407ebd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.084+0000 7f782e7fc640 1 --2- 172.21.15.76:0/4205870920 >> [v2:172.21.15.76:6808/1992938181,v1:172.21.15.76:6809/1992938181] conn(0x7f783407c780 0x7f783407ebd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.084+0000 7f1b30ff9640 1 -- 172.21.15.76:0/169352287 --> [v2:172.21.15.105:6808/2721832954,v1:172.21.15.105:6809/2721832954] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1b14002dd0 con 0x7f1b1c07c850 2024-04-25T19:15:27.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.084+0000 7f782e7fc640 1 -- 172.21.15.76:0/4205870920 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f7834074530 msgr2=0x7f7834076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.084+0000 7f782e7fc640 1 --2- 172.21.15.76:0/4205870920 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f7834074530 0x7f7834076a40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f78300057e0 tx=0x7f783c030000 comp rx=0 tx=0).stop 2024-04-25T19:15:27.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.084+0000 7f782e7fc640 1 -- 172.21.15.76:0/4205870920 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7848155320 msgr2=0x7f78481f6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.084+0000 7f782e7fc640 1 --2- 172.21.15.76:0/4205870920 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f7848155320 0x7f78481f6c10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f78400052e0 tx=0x7f78400057a0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.084+0000 7fbdab1c5640 1 -- 172.21.15.76:0/411451022 --> [v2:172.21.15.76:6824/3418257163,v1:172.21.15.76:6825/3418257163] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbd9c005570 con 0x7fbd8807c880 2024-04-25T19:15:27.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.084+0000 7f784cb23640 1 -- 172.21.15.76:0/4205870920 reap_dead start 2024-04-25T19:15:27.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.084+0000 7f782e7fc640 1 -- 172.21.15.76:0/4205870920 shutdown_connections 2024-04-25T19:15:27.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.085+0000 7f782e7fc640 1 -- 172.21.15.76:0/4205870920 >> 172.21.15.76:0/4205870920 conn(0x7f78480684e0 msgr2=0x7f78480b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.085+0000 7f1b32ffd640 1 -- 172.21.15.76:0/169352287 <== osd.6 v2:172.21.15.105:6808/2721832954 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1b14002dd0 con 0x7f1b1c07c850 2024-04-25T19:15:27.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.085+0000 7f782e7fc640 1 -- 172.21.15.76:0/4205870920 shutdown_connections 2024-04-25T19:15:27.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.085+0000 7f782e7fc640 1 -- 172.21.15.76:0/4205870920 wait complete. 2024-04-25T19:15:27.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.085+0000 7fbd9affd640 1 -- 172.21.15.76:0/411451022 <== osd.3 v2:172.21.15.76:6824/3418257163 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbd9c005570 con 0x7fbd8807c880 2024-04-25T19:15:27.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.085+0000 7f1b30ff9640 1 -- 172.21.15.76:0/169352287 >> [v2:172.21.15.105:6808/2721832954,v1:172.21.15.105:6809/2721832954] conn(0x7f1b1c07c850 msgr2=0x7f1b1c07eca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.085+0000 7f1b30ff9640 1 --2- 172.21.15.76:0/169352287 >> [v2:172.21.15.105:6808/2721832954,v1:172.21.15.105:6809/2721832954] conn(0x7f1b1c07c850 0x7f1b1c07eca0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.085+0000 7f1b30ff9640 1 -- 172.21.15.76:0/169352287 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1b1c074580 msgr2=0x7f1b1c076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.085+0000 7f1b30ff9640 1 --2- 172.21.15.76:0/169352287 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1b1c074580 0x7f1b1c076a90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c006f40 tx=0x7f1b2c008040 comp rx=0 tx=0).stop 2024-04-25T19:15:27.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.085+0000 7f1b30ff9640 1 -- 172.21.15.76:0/169352287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b3c0c6640 msgr2=0x7f1b3c156960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.085+0000 7f1b30ff9640 1 --2- 172.21.15.76:0/169352287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1b3c0c6640 0x7f1b3c156960 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c06e150 tx=0x7f1b2400b870 comp rx=0 tx=0).stop 2024-04-25T19:15:27.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.085+0000 7f1b3ad9d640 1 -- 172.21.15.76:0/169352287 reap_dead start 2024-04-25T19:15:27.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.085+0000 7f1b30ff9640 1 -- 172.21.15.76:0/169352287 shutdown_connections 2024-04-25T19:15:27.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.085+0000 7f1b30ff9640 1 -- 172.21.15.76:0/169352287 >> 172.21.15.76:0/169352287 conn(0x7f1b3c070800 msgr2=0x7f1b3c070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.085+0000 7f1b30ff9640 1 -- 172.21.15.76:0/169352287 shutdown_connections 2024-04-25T19:15:27.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.085+0000 7f1b30ff9640 1 -- 172.21.15.76:0/169352287 wait complete. 2024-04-25T19:15:27.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.085+0000 7fbdab1c5640 1 -- 172.21.15.76:0/411451022 >> [v2:172.21.15.76:6824/3418257163,v1:172.21.15.76:6825/3418257163] conn(0x7fbd8807c880 msgr2=0x7fbd8807ecd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.085+0000 7fbdab1c5640 1 --2- 172.21.15.76:0/411451022 >> [v2:172.21.15.76:6824/3418257163,v1:172.21.15.76:6825/3418257163] conn(0x7fbd8807c880 0x7fbd8807ecd0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.086+0000 7fbdab1c5640 1 -- 172.21.15.76:0/411451022 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbd880745b0 msgr2=0x7fbd88076ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.086+0000 7fbdab1c5640 1 --2- 172.21.15.76:0/411451022 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbd880745b0 0x7fbd88076ac0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbda00095e0 tx=0x7fbda0009190 comp rx=0 tx=0).stop 2024-04-25T19:15:27.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.086+0000 7fbdab1c5640 1 -- 172.21.15.76:0/411451022 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbd9c0471b0 msgr2=0x7fbd9c17cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.086+0000 7fbdab1c5640 1 --2- 172.21.15.76:0/411451022 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbd9c0471b0 0x7fbd9c17cfd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c046bc0 tx=0x7fbd940046c0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.086+0000 7fbdaa9c4640 1 -- 172.21.15.76:0/411451022 reap_dead start 2024-04-25T19:15:27.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.086+0000 7fbdab1c5640 1 -- 172.21.15.76:0/411451022 shutdown_connections 2024-04-25T19:15:27.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.086+0000 7fbdab1c5640 1 -- 172.21.15.76:0/411451022 >> 172.21.15.76:0/411451022 conn(0x7fbd9c0503b0 msgr2=0x7fbd9c005d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.086+0000 7fbdab1c5640 1 -- 172.21.15.76:0/411451022 shutdown_connections 2024-04-25T19:15:27.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.086+0000 7fbdab1c5640 1 -- 172.21.15.76:0/411451022 wait complete. 2024-04-25T19:15:27.168 INFO:teuthology.orchestra.run.smithi076.stdout:55834574853 2024-04-25T19:15:27.168 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T19:15:27.169 INFO:teuthology.orchestra.run.smithi076.stdout:55834574853 2024-04-25T19:15:27.169 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T19:15:27.170 INFO:teuthology.orchestra.run.smithi076.stdout:55834574853 2024-04-25T19:15:27.170 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T19:15:27.171 INFO:teuthology.orchestra.run.smithi076.stdout:55834574853 2024-04-25T19:15:27.172 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T19:15:27.172 INFO:teuthology.orchestra.run.smithi076.stdout:55834574853 2024-04-25T19:15:27.173 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T19:15:27.173 INFO:teuthology.orchestra.run.smithi076.stdout:55834574853 2024-04-25T19:15:27.174 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T19:15:27.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.363+0000 7f4fcb7fe640 1 -- 172.21.15.76:0/3818361545 <== osd.1 v2:172.21.15.76:6800/1302663821 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4fd007f370 con 0x7f4fd007c850 2024-04-25T19:15:27.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.380+0000 7f4fe96b1640 1 -- 172.21.15.76:0/3818361545 --> [v2:172.21.15.76:6800/1302663821,v1:172.21.15.76:6801/1302663821] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4fe4154820 con 0x7f4fd007c850 2024-04-25T19:15:27.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.380+0000 7f4fcb7fe640 1 -- 172.21.15.76:0/3818361545 <== osd.1 v2:172.21.15.76:6800/1302663821 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4fe4154820 con 0x7f4fd007c850 2024-04-25T19:15:27.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.381+0000 7f4fe96b1640 1 -- 172.21.15.76:0/3818361545 >> [v2:172.21.15.76:6800/1302663821,v1:172.21.15.76:6801/1302663821] conn(0x7f4fd007c850 msgr2=0x7f4fd007eca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.381+0000 7f4fe96b1640 1 --2- 172.21.15.76:0/3818361545 >> [v2:172.21.15.76:6800/1302663821,v1:172.21.15.76:6801/1302663821] conn(0x7f4fd007c850 0x7f4fd007eca0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.381+0000 7f4fe96b1640 1 -- 172.21.15.76:0/3818361545 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4fd0074580 msgr2=0x7f4fd0076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.381+0000 7f4fe96b1640 1 --2- 172.21.15.76:0/3818361545 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4fd0074580 0x7f4fd0076a90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009340 tx=0x7f4fdc0092d0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.381+0000 7f4fe96b1640 1 -- 172.21.15.76:0/3818361545 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4fe40c56d0 msgr2=0x7f4fe40c5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.381+0000 7f4fe96b1640 1 --2- 172.21.15.76:0/3818361545 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4fe40c56d0 0x7f4fe40c5ae0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400be20 tx=0x7f4fd400bf10 comp rx=0 tx=0).stop 2024-04-25T19:15:27.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.381+0000 7f4fe37fe640 1 -- 172.21.15.76:0/3818361545 reap_dead start 2024-04-25T19:15:27.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.381+0000 7f4fe96b1640 1 -- 172.21.15.76:0/3818361545 shutdown_connections 2024-04-25T19:15:27.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.381+0000 7f4fe96b1640 1 -- 172.21.15.76:0/3818361545 >> 172.21.15.76:0/3818361545 conn(0x7f4fe40caff0 msgr2=0x7f4fe40ccca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.381+0000 7f4fe96b1640 1 -- 172.21.15.76:0/3818361545 shutdown_connections 2024-04-25T19:15:27.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.381+0000 7f4fe96b1640 1 -- 172.21.15.76:0/3818361545 wait complete. 2024-04-25T19:15:27.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.394+0000 7fa280535640 1 Processor -- start 2024-04-25T19:15:27.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.394+0000 7fa280535640 1 -- start start 2024-04-25T19:15:27.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.395+0000 7fa280535640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa2780c8520 0x7fa278163a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.395+0000 7fa280535640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2780cdd60 0x7fa2780ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.395+0000 7fa280535640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa27816a320 con 0x7fa27808d990 2024-04-25T19:15:27.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.395+0000 7fa280535640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fa2780cd390 con 0x7fa2780c8520 2024-04-25T19:15:27.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.395+0000 7fa280535640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa2780cd670 con 0x7fa2780cdd60 2024-04-25T19:15:27.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.395+0000 7fa27dad0640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa27808d990 0x7fa2780c65d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35248/0 (socket says 172.21.15.76:35248) 2024-04-25T19:15:27.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.395+0000 7fa27dad0640 1 -- 172.21.15.76:0/3606499472 learned_addr learned my addr 172.21.15.76:0/3606499472 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:27.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.396+0000 7fa27e2d1640 1 --2- 172.21.15.76:0/3606499472 >> v2:172.21.15.105:3300/0 conn(0x7fa2780c8520 0x7fa278163a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.396+0000 7fa27ead2640 1 --2- 172.21.15.76:0/3606499472 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2780cdd60 0x7fa2780ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.346+0000 7f921cbc0640 1 Processor -- start 2024-04-25T19:15:27.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.346+0000 7f921cbc0640 1 -- start start 2024-04-25T19:15:27.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.347+0000 7f921cbc0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f92180c8520 0x7f9218163a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.347+0000 7f921cbc0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92180cdd60 0x7f92180ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.348+0000 7f921cbc0640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f921816a320 con 0x7f921808d990 2024-04-25T19:15:27.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.348+0000 7f921cbc0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f92180cd390 con 0x7f92180c8520 2024-04-25T19:15:27.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.348+0000 7f921cbc0640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f92180cd670 con 0x7f92180cdd60 2024-04-25T19:15:27.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.348+0000 7f921659c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f92180c8520 0x7f9218163a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.348+0000 7f921659c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f92180c8520 0x7f9218163a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59184/0 (socket says 172.21.15.76:59184) 2024-04-25T19:15:27.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.348+0000 7f921659c640 1 -- 172.21.15.76:0/3184427577 learned_addr learned my addr 172.21.15.76:0/3184427577 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:27.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.348+0000 7f921559a640 1 -- 172.21.15.76:0/3184427577 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1577849677 0 0) 0x7f921816a320 con 0x7f921808d990 2024-04-25T19:15:27.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.395+0000 7f8e31492640 1 Processor -- start 2024-04-25T19:15:27.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.395+0000 7f8e31492640 1 -- start start 2024-04-25T19:15:27.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.397+0000 7f8e31492640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e2c163a60 0x7f8e2c175090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.348+0000 7f9216d9d640 1 --2- 172.21.15.76:0/3184427577 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92180cdd60 0x7f92180ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.397+0000 7f8e31492640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e2c163e70 con 0x7f8e2c08d990 2024-04-25T19:15:27.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.397+0000 7f8e31492640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e2c16a320 con 0x7f8e2c0c8520 2024-04-25T19:15:27.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.397+0000 7f8e31492640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8e2c0cd390 con 0x7f8e2c163a60 2024-04-25T19:15:27.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.397+0000 7f8e23fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8e2c08d990 0x7f8e2c0cdd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35262/0 (socket says 172.21.15.76:35262) 2024-04-25T19:15:27.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.397+0000 7f8e23fff640 1 -- 172.21.15.76:0/3819414658 learned_addr learned my addr 172.21.15.76:0/3819414658 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:27.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.397+0000 7f8e2b7fe640 1 --2- 172.21.15.76:0/3819414658 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e2c163a60 0x7f8e2c175090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.437+0000 7f921559a640 1 -- 172.21.15.76:0/3184427577 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9204005730 con 0x7f921808d990 2024-04-25T19:15:27.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.397+0000 7f8e2a7fc640 1 -- 172.21.15.76:0/3819414658 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3678880268 0 0) 0x7f8e2c163e70 con 0x7f8e2c08d990 2024-04-25T19:15:27.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.437+0000 7f8e2a7fc640 1 -- 172.21.15.76:0/3819414658 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e18003970 con 0x7f8e2c08d990 2024-04-25T19:15:27.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.395+0000 7fdb27ba8640 1 Processor -- start 2024-04-25T19:15:27.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.395+0000 7fdb27ba8640 1 -- start start 2024-04-25T19:15:27.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.396+0000 7fdb27ba8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb20163a60 0x7fdb20175090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.396+0000 7fdb27ba8640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb20163e70 con 0x7fdb200c8520 2024-04-25T19:15:27.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.396+0000 7fdb27ba8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb2016a320 con 0x7fdb2008d990 2024-04-25T19:15:27.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.396+0000 7fdb27ba8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fdb200cd390 con 0x7fdb20163a60 2024-04-25T19:15:27.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.397+0000 7fdb25944640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fdb200c8520 0x7fdb200c65d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35260/0 (socket says 172.21.15.76:35260) 2024-04-25T19:15:27.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.397+0000 7fdb25944640 1 -- 172.21.15.76:0/894505848 learned_addr learned my addr 172.21.15.76:0/894505848 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:27.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.397+0000 7fdb26145640 1 --2- 172.21.15.76:0/894505848 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb20163a60 0x7fdb20175090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.437+0000 7f8e2a7fc640 1 -- 172.21.15.76:0/3819414658 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1514524075 0 0) 0x7f8e2c16a320 con 0x7f8e2c0c8520 2024-04-25T19:15:27.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.437+0000 7f8e2a7fc640 1 -- 172.21.15.76:0/3819414658 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e2c163e70 con 0x7f8e2c0c8520 2024-04-25T19:15:27.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.437+0000 7f921659c640 1 -- 172.21.15.76:0/3184427577 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92180cdd60 msgr2=0x7f92180ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:27.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.437+0000 7f921659c640 1 --2- 172.21.15.76:0/3184427577 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92180cdd60 0x7f92180ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.397+0000 7fdb24942640 1 -- 172.21.15.76:0/894505848 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2842542629 0 0) 0x7fdb2016a320 con 0x7fdb2008d990 2024-04-25T19:15:27.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.437+0000 7fdb24942640 1 -- 172.21.15.76:0/894505848 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb0c003500 con 0x7fdb2008d990 2024-04-25T19:15:27.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.437+0000 7fdb24942640 1 -- 172.21.15.76:0/894505848 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1665906296 0 0) 0x7fdb20163e70 con 0x7fdb200c8520 2024-04-25T19:15:27.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.437+0000 7fdb24942640 1 -- 172.21.15.76:0/894505848 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb2016a320 con 0x7fdb200c8520 2024-04-25T19:15:27.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.437+0000 7f921659c640 1 -- 172.21.15.76:0/3184427577 >> v1:172.21.15.105:6789/0 conn(0x7f921808d990 legacy=0x7f92180c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.437+0000 7fa27e2d1640 1 -- 172.21.15.76:0/3606499472 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2780cdd60 msgr2=0x7fa2780ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:27.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.437+0000 7fa27e2d1640 1 --2- 172.21.15.76:0/3606499472 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2780cdd60 0x7fa2780ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.437+0000 7fa27e2d1640 1 -- 172.21.15.76:0/3606499472 >> v1:172.21.15.76:6789/0 conn(0x7fa27808d990 legacy=0x7fa2780c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.437+0000 7fa27e2d1640 1 -- 172.21.15.76:0/3606499472 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27816a600 con 0x7fa2780c8520 2024-04-25T19:15:27.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.437+0000 7f8e2a7fc640 1 -- 172.21.15.76:0/3819414658 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3681331469 0 0) 0x7f8e18003970 con 0x7f8e2c08d990 2024-04-25T19:15:27.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.437+0000 7f8e2a7fc640 1 -- 172.21.15.76:0/3819414658 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e2c163a60 msgr2=0x7f8e2c175090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:27.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.437+0000 7f8e2a7fc640 1 --2- 172.21.15.76:0/3819414658 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e2c163a60 0x7f8e2c175090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.437+0000 7f8e2a7fc640 1 -- 172.21.15.76:0/3819414658 >> v1:172.21.15.105:6789/0 conn(0x7f8e2c0c8520 legacy=0x7f8e2c0c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.437+0000 7f8e2a7fc640 1 -- 172.21.15.76:0/3819414658 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e2c0cd670 con 0x7f8e2c08d990 2024-04-25T19:15:27.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.437+0000 7f8e2a7fc640 1 -- 172.21.15.76:0/3819414658 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f8e14001670 con 0x7f8e2c08d990 2024-04-25T19:15:27.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.437+0000 7f8e2a7fc640 1 -- 172.21.15.76:0/3819414658 >> v1:172.21.15.76:6789/0 conn(0x7f8e2c08d990 legacy=0x7f8e2c0cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7f8e2a7fc640 1 --2- 172.21.15.76:0/3819414658 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e18003420 0x7f8e18005020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7f8e2a7fc640 1 --2- 172.21.15.76:0/3819414658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e180055f0 0x7f8e18003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7f8e2a7fc640 1 --2- 172.21.15.76:0/3819414658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e18004180 0x7f8e18004650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7f8e2a7fc640 1 -- 172.21.15.76:0/3819414658 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c0cd390 con 0x7f8e18004180 2024-04-25T19:15:27.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7f8e2a7fc640 1 -- 172.21.15.76:0/3819414658 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8e18005e40 con 0x7f8e180055f0 2024-04-25T19:15:27.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7f8e2a7fc640 1 -- 172.21.15.76:0/3819414658 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8e18006040 con 0x7f8e18003420 2024-04-25T19:15:27.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.396+0000 7fa27d2cf640 1 -- 172.21.15.76:0/3606499472 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1918674216 0 0) 0x7fa27816a320 con 0x7fa27808d990 2024-04-25T19:15:27.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7f8e23fff640 1 --2- 172.21.15.76:0/3819414658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e180055f0 0x7f8e18003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7f8e2affd640 1 --2- 172.21.15.76:0/3819414658 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e18003420 0x7f8e18005020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7fdb24942640 1 -- 172.21.15.76:0/894505848 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2516197960 0 0) 0x7fdb2016a320 con 0x7fdb200c8520 2024-04-25T19:15:27.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7fdb24942640 1 -- 172.21.15.76:0/894505848 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb20163a60 msgr2=0x7fdb20175090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:27.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7fdb24942640 1 --2- 172.21.15.76:0/894505848 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb20163a60 0x7fdb20175090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7fdb24942640 1 -- 172.21.15.76:0/894505848 >> v1:172.21.15.105:6789/0 conn(0x7fdb2008d990 legacy=0x7fdb200cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7fdb24942640 1 -- 172.21.15.76:0/894505848 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb200cd670 con 0x7fdb200c8520 2024-04-25T19:15:27.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7fdb24942640 1 -- 172.21.15.76:0/894505848 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7fdb1c001670 con 0x7fdb200c8520 2024-04-25T19:15:27.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7fdb24942640 1 -- 172.21.15.76:0/894505848 >> v1:172.21.15.76:6789/0 conn(0x7fdb200c8520 legacy=0x7fdb200c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7fdb24942640 1 --2- 172.21.15.76:0/894505848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb0c003890 0x7fdb0c003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7fdb24942640 1 --2- 172.21.15.76:0/894505848 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb0c004240 0x7fdb0c004da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7fdb24942640 1 --2- 172.21.15.76:0/894505848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb0c0052e0 0x7fdb0c005710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7fdb24942640 1 -- 172.21.15.76:0/894505848 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdb200cd390 con 0x7fdb0c0052e0 2024-04-25T19:15:27.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7fdb24942640 1 -- 172.21.15.76:0/894505848 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c005fe0 con 0x7fdb0c003890 2024-04-25T19:15:27.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7fdb24942640 1 -- 172.21.15.76:0/894505848 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fdb0c0061e0 con 0x7fdb0c004240 2024-04-25T19:15:27.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.437+0000 7f921559a640 1 -- 172.21.15.76:0/3184427577 <== mon.1 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1876547548 0 0) 0x7f9204005730 con 0x7f921808d990 2024-04-25T19:15:27.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.437+0000 7f921659c640 1 -- 172.21.15.76:0/3184427577 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f921816a600 con 0x7f92180c8520 2024-04-25T19:15:27.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7fdb25944640 1 --2- 172.21.15.76:0/894505848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb0c003890 0x7fdb0c003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.439+0000 7fdb26145640 1 --2- 172.21.15.76:0/894505848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb0c0052e0 0x7fdb0c005710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.439+0000 7fdb25143640 1 --2- 172.21.15.76:0/894505848 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb0c004240 0x7fdb0c004da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.439+0000 7fa27e2d1640 1 --2- 172.21.15.76:0/3606499472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2780c8520 0x7fa278163a60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa2740087a0 tx=0x7fa27402a320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cd55ea6251cff45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.396+0000 7fa3988e0640 1 Processor -- start 2024-04-25T19:15:27.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.397+0000 7fa3988e0640 1 -- start start 2024-04-25T19:15:27.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.398+0000 7fa3988e0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa3940c8520 0x7fa394163a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.399+0000 7fa3988e0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa39416a320 con 0x7fa39408d990 2024-04-25T19:15:27.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.399+0000 7fa3988e0640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3940cd390 con 0x7fa3940cdd60 2024-04-25T19:15:27.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.399+0000 7fa3988e0640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa3940cd670 con 0x7fa3940c8520 2024-04-25T19:15:27.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.399+0000 7fa391d9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa39408d990 0x7fa3940c65d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35270/0 (socket says 172.21.15.76:35270) 2024-04-25T19:15:27.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.399+0000 7fa391d9b640 1 -- 172.21.15.76:0/1573770556 learned_addr learned my addr 172.21.15.76:0/1573770556 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:27.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.399+0000 7fa39259c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa3940c8520 0x7fa394163a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.399+0000 7fa39159a640 1 -- 172.21.15.76:0/1573770556 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 624048568 0 0) 0x7fa39416a320 con 0x7fa39408d990 2024-04-25T19:15:27.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.439+0000 7fa39159a640 1 -- 172.21.15.76:0/1573770556 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa37c003500 con 0x7fa39408d990 2024-04-25T19:15:27.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.439+0000 7fa39159a640 1 -- 172.21.15.76:0/1573770556 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 537997410 0 0) 0x7fa3940cd390 con 0x7fa3940cdd60 2024-04-25T19:15:27.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.439+0000 7fa39159a640 1 -- 172.21.15.76:0/1573770556 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa39416a320 con 0x7fa3940cdd60 2024-04-25T19:15:27.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.398+0000 7f88d5bff640 1 Processor -- start 2024-04-25T19:15:27.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.398+0000 7f88d5bff640 1 -- start start 2024-04-25T19:15:27.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.439+0000 7f921659c640 1 --2- 172.21.15.76:0/3184427577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92180c8520 0x7f9218163a60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f92000088f0 tx=0x7f9200008230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae469bca825b8a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.399+0000 7f88d5bff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f88d0163a60 0x7f88d0175090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.399+0000 7f88d5bff640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88d0163e70 con 0x7f88d008d990 2024-04-25T19:15:27.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.399+0000 7f88d5bff640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88d016a320 con 0x7f88d00c8520 2024-04-25T19:15:27.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.399+0000 7f88d5bff640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f88d00cd390 con 0x7f88d0163a60 2024-04-25T19:15:27.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.399+0000 7f88ceffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f88d008d990 0x7f88d00cdd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35280/0 (socket says 172.21.15.76:35280) 2024-04-25T19:15:27.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.399+0000 7f88ceffd640 1 -- 172.21.15.76:0/1915103044 learned_addr learned my addr 172.21.15.76:0/1915103044 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:27.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.399+0000 7f88cffff640 1 --2- 172.21.15.76:0/1915103044 >> v2:172.21.15.105:3300/0 conn(0x7f88d0163a60 0x7f88d0175090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.439+0000 7fa39259c640 1 -- 172.21.15.76:0/1573770556 >> v1:172.21.15.105:6789/0 conn(0x7fa3940cdd60 legacy=0x7fa394174080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.439+0000 7fa39259c640 1 -- 172.21.15.76:0/1573770556 >> v1:172.21.15.76:6789/0 conn(0x7fa39408d990 legacy=0x7fa3940c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.439+0000 7fa27d2cf640 1 -- 172.21.15.76:0/3606499472 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa274030070 con 0x7fa2780c8520 2024-04-25T19:15:27.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.399+0000 7f88ce7fc640 1 -- 172.21.15.76:0/1915103044 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1957498638 0 0) 0x7f88d0163e70 con 0x7f88d008d990 2024-04-25T19:15:27.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.439+0000 7fa39259c640 1 -- 172.21.15.76:0/1573770556 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39416a600 con 0x7fa3940c8520 2024-04-25T19:15:27.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7f8e23fff640 1 -- 172.21.15.76:0/3819414658 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e18003420 msgr2=0x7f8e18005020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:27.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7f8e23fff640 1 --2- 172.21.15.76:0/3819414658 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e18003420 0x7f8e18005020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7f8e23fff640 1 -- 172.21.15.76:0/3819414658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e18004180 msgr2=0x7f8e18004650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:15:27.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7f8e23fff640 1 --2- 172.21.15.76:0/3819414658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e18004180 0x7f8e18004650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.438+0000 7f8e23fff640 1 -- 172.21.15.76:0/3819414658 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e2c0cd670 con 0x7f8e180055f0 2024-04-25T19:15:27.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.439+0000 7fa27d2cf640 1 -- 172.21.15.76:0/3606499472 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa274034600 con 0x7fa2780c8520 2024-04-25T19:15:27.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.439+0000 7fdb26145640 1 -- 172.21.15.76:0/894505848 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb0c004240 msgr2=0x7fdb0c004da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:27.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.439+0000 7fdb26145640 1 --2- 172.21.15.76:0/894505848 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb0c004240 0x7fdb0c004da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.439+0000 7fdb26145640 1 -- 172.21.15.76:0/894505848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb0c003890 msgr2=0x7fdb0c003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:27.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.439+0000 7fdb26145640 1 --2- 172.21.15.76:0/894505848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb0c003890 0x7fdb0c003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.439+0000 7fdb26145640 1 -- 172.21.15.76:0/894505848 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb200cd670 con 0x7fdb0c0052e0 2024-04-25T19:15:27.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.439+0000 7fdb25944640 1 --2- 172.21.15.76:0/894505848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb0c003890 0x7fdb0c003cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:27.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.439+0000 7f88ce7fc640 1 -- 172.21.15.76:0/1915103044 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88b8003970 con 0x7f88d008d990 2024-04-25T19:15:27.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.439+0000 7f88ce7fc640 1 -- 172.21.15.76:0/1915103044 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3420134675 0 0) 0x7f88d016a320 con 0x7f88d00c8520 2024-04-25T19:15:27.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.439+0000 7f88ce7fc640 1 -- 172.21.15.76:0/1915103044 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88d0163e70 con 0x7f88d00c8520 2024-04-25T19:15:27.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.398+0000 7f198ac6e640 1 Processor -- start 2024-04-25T19:15:27.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.398+0000 7f198ac6e640 1 -- start start 2024-04-25T19:15:27.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.400+0000 7f198ac6e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f198408d990 0x7f1984163a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.400+0000 7f198ac6e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19840cdd60 0x7f19840ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.440+0000 7fa27d2cf640 1 -- 172.21.15.76:0/3606499472 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa274030070 con 0x7fa2780c8520 2024-04-25T19:15:27.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.440+0000 7f88cffff640 1 -- 172.21.15.76:0/1915103044 >> v1:172.21.15.105:6789/0 conn(0x7f88d00c8520 legacy=0x7f88d00c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.440+0000 7f88cffff640 1 -- 172.21.15.76:0/1915103044 >> v1:172.21.15.76:6789/0 conn(0x7f88d008d990 legacy=0x7f88d00cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.440+0000 7f88cffff640 1 -- 172.21.15.76:0/1915103044 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d00cd670 con 0x7f88d0163a60 2024-04-25T19:15:27.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.440+0000 7f921559a640 1 -- 172.21.15.76:0/3184427577 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208001670 con 0x7f92180c8520 2024-04-25T19:15:27.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.400+0000 7f198ac6e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f198416a320 con 0x7f19840c8520 2024-04-25T19:15:27.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.440+0000 7fa39259c640 1 --2- 172.21.15.76:0/1573770556 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa3940c8520 0x7fa394163a60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa384009830 tx=0x7fa38402a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2e944b435995073 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.440+0000 7f8e23fff640 1 --2- 172.21.15.76:0/3819414658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e180055f0 0x7f8e18003bf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400c210 tx=0x7f8e18005c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f3990419b75eb2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.400+0000 7f198ac6e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f19840cd390 con 0x7f198408d990 2024-04-25T19:15:27.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.400+0000 7f198ac6e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f19840cd670 con 0x7f19840cdd60 2024-04-25T19:15:27.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.400+0000 7f1983fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f198408d990 0x7f1984163a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.440+0000 7f8e2a7fc640 1 -- 172.21.15.76:0/3819414658 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e14007d30 con 0x7f8e180055f0 2024-04-25T19:15:27.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.440+0000 7f8e2a7fc640 1 -- 172.21.15.76:0/3819414658 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e14005ad0 con 0x7f8e180055f0 2024-04-25T19:15:27.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.400+0000 7f1983fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f198408d990 0x7f1984163a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59192/0 (socket says 172.21.15.76:59192) 2024-04-25T19:15:27.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.440+0000 7f8e2a7fc640 1 -- 172.21.15.76:0/3819414658 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e14007070 con 0x7f8e180055f0 2024-04-25T19:15:27.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.400+0000 7f1983fff640 1 -- 172.21.15.76:0/1826037682 learned_addr learned my addr 172.21.15.76:0/1826037682 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:27.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.400+0000 7f198920b640 1 --2- 172.21.15.76:0/1826037682 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19840cdd60 0x7f19840ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.440+0000 7fa39159a640 1 -- 172.21.15.76:0/1573770556 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38402ac10 con 0x7fa3940c8520 2024-04-25T19:15:27.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.440+0000 7fa39159a640 1 -- 172.21.15.76:0/1573770556 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa38402adb0 con 0x7fa3940c8520 2024-04-25T19:15:27.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.440+0000 7fa39159a640 1 -- 172.21.15.76:0/1573770556 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3840076b0 con 0x7fa3940c8520 2024-04-25T19:15:27.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.440+0000 7fa280535640 1 -- 172.21.15.76:0/3606499472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2780c8520 msgr2=0x7fa278163a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.440+0000 7fa280535640 1 --2- 172.21.15.76:0/3606499472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2780c8520 0x7fa278163a60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa2740087a0 tx=0x7fa27402a320 comp rx=0 tx=0).stop 2024-04-25T19:15:27.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.440+0000 7fdb26145640 1 --2- 172.21.15.76:0/894505848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb0c0052e0 0x7fdb0c005710 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdb200c47f0 tx=0x7fdb0c00afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98249cd2b70fdea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.441+0000 7f921559a640 1 -- 172.21.15.76:0/3184427577 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f920000ae50 con 0x7f92180c8520 2024-04-25T19:15:27.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.441+0000 7f921559a640 1 -- 172.21.15.76:0/3184427577 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92000323f0 con 0x7f92180c8520 2024-04-25T19:15:27.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.441+0000 7f88cffff640 1 --2- 172.21.15.76:0/1915103044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88d0163a60 0x7f88d0175090 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009b10 tx=0x7f88c802bc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29a29cc73f3bd487 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.441+0000 7fa280535640 1 -- 172.21.15.76:0/3606499472 shutdown_connections 2024-04-25T19:15:27.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.400+0000 7f19837fe640 1 -- 172.21.15.76:0/1826037682 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3280437757 0 0) 0x7f198416a320 con 0x7f19840c8520 2024-04-25T19:15:27.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.441+0000 7fa280535640 1 --2- 172.21.15.76:0/3606499472 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2780cdd60 0x7fa2780ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.441+0000 7fa280535640 1 --2- 172.21.15.76:0/3606499472 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2780c8520 0x7fa278163a60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.441+0000 7fa280535640 1 -- 172.21.15.76:0/3606499472 >> 172.21.15.76:0/3606499472 conn(0x7fa278070800 msgr2=0x7fa278070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.440+0000 7f19837fe640 1 -- 172.21.15.76:0/1826037682 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1974003500 con 0x7f19840c8520 2024-04-25T19:15:27.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.441+0000 7f1983fff640 1 -- 172.21.15.76:0/1826037682 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19840cdd60 msgr2=0x7f19840ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:27.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.441+0000 7fa280535640 1 -- 172.21.15.76:0/3606499472 shutdown_connections 2024-04-25T19:15:27.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.441+0000 7f1983fff640 1 --2- 172.21.15.76:0/1826037682 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19840cdd60 0x7f19840ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.441+0000 7f1983fff640 1 -- 172.21.15.76:0/1826037682 >> v1:172.21.15.76:6789/0 conn(0x7f19840c8520 legacy=0x7f19840c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.441+0000 7f1983fff640 1 -- 172.21.15.76:0/1826037682 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198416a600 con 0x7f198408d990 2024-04-25T19:15:27.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.441+0000 7fa280535640 1 -- 172.21.15.76:0/3606499472 wait complete. 2024-04-25T19:15:27.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.441+0000 7f8e31492640 1 -- 172.21.15.76:0/3819414658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e180055f0 msgr2=0x7f8e18003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.441+0000 7f8e31492640 1 --2- 172.21.15.76:0/3819414658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e180055f0 0x7f8e18003bf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400c210 tx=0x7f8e18005c80 comp rx=0 tx=0).stop 2024-04-25T19:15:27.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.441+0000 7f8e2b7fe640 1 -- 172.21.15.76:0/3819414658 reap_dead start 2024-04-25T19:15:27.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.441+0000 7fa3988e0640 1 -- 172.21.15.76:0/1573770556 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa3940c8520 msgr2=0x7fa394163a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.441+0000 7fa3988e0640 1 --2- 172.21.15.76:0/1573770556 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa3940c8520 0x7fa394163a60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa384009830 tx=0x7fa38402a110 comp rx=0 tx=0).stop 2024-04-25T19:15:27.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.442+0000 7f8e31492640 1 -- 172.21.15.76:0/3819414658 shutdown_connections 2024-04-25T19:15:27.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.442+0000 7f8e31492640 1 -- 172.21.15.76:0/3819414658 >> 172.21.15.76:0/3819414658 conn(0x7f8e2c070800 msgr2=0x7f8e2c070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.442+0000 7fa3988e0640 1 -- 172.21.15.76:0/1573770556 shutdown_connections 2024-04-25T19:15:27.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.442+0000 7fa3988e0640 1 --2- 172.21.15.76:0/1573770556 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa3940c8520 0x7fa394163a60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.442+0000 7fa3988e0640 1 -- 172.21.15.76:0/1573770556 >> 172.21.15.76:0/1573770556 conn(0x7fa394070800 msgr2=0x7fa394070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.442+0000 7f1983fff640 1 --2- 172.21.15.76:0/1826037682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f198408d990 0x7f1984163a60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f197000b970 tx=0x7f1970033190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=541ab2d3793b4881 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.442+0000 7f8e31492640 1 -- 172.21.15.76:0/3819414658 shutdown_connections 2024-04-25T19:15:27.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.442+0000 7f8e31492640 1 -- 172.21.15.76:0/3819414658 wait complete. 2024-04-25T19:15:27.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.442+0000 7f921cbc0640 1 -- 172.21.15.76:0/3184427577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92180c8520 msgr2=0x7f9218163a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.442+0000 7f921cbc0640 1 --2- 172.21.15.76:0/3184427577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92180c8520 0x7f9218163a60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f92000088f0 tx=0x7f9200008230 comp rx=0 tx=0).stop 2024-04-25T19:15:27.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.442+0000 7f921cbc0640 1 -- 172.21.15.76:0/3184427577 shutdown_connections 2024-04-25T19:15:27.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.442+0000 7f921cbc0640 1 --2- 172.21.15.76:0/3184427577 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92180cdd60 0x7f92180ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.442+0000 7f921cbc0640 1 --2- 172.21.15.76:0/3184427577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92180c8520 0x7f9218163a60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.442+0000 7f921cbc0640 1 -- 172.21.15.76:0/3184427577 >> 172.21.15.76:0/3184427577 conn(0x7f9218070800 msgr2=0x7f9218070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.442+0000 7f921cbc0640 1 -- 172.21.15.76:0/3184427577 shutdown_connections 2024-04-25T19:15:27.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.442+0000 7fa3988e0640 1 -- 172.21.15.76:0/1573770556 shutdown_connections 2024-04-25T19:15:27.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.442+0000 7fa3988e0640 1 -- 172.21.15.76:0/1573770556 wait complete. 2024-04-25T19:15:27.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.442+0000 7f921cbc0640 1 -- 172.21.15.76:0/3184427577 wait complete. 2024-04-25T19:15:27.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.443+0000 7fa280535640 1 Processor -- start 2024-04-25T19:15:27.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.443+0000 7fa280535640 1 -- start start 2024-04-25T19:15:27.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.444+0000 7f88ce7fc640 1 -- 172.21.15.76:0/1915103044 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c802d920 con 0x7f88d0163a60 2024-04-25T19:15:27.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.444+0000 7f88ce7fc640 1 -- 172.21.15.76:0/1915103044 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88c802dac0 con 0x7f88d0163a60 2024-04-25T19:15:27.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.444+0000 7f88ce7fc640 1 -- 172.21.15.76:0/1915103044 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c802c440 con 0x7f88d0163a60 2024-04-25T19:15:27.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.444+0000 7fa3988e0640 1 Processor -- start 2024-04-25T19:15:27.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.444+0000 7fa3988e0640 1 -- start start 2024-04-25T19:15:27.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.445+0000 7f8e31492640 1 Processor -- start 2024-04-25T19:15:27.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.445+0000 7fa280535640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa27808d990 0x7fa278203c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.445+0000 7f8e31492640 1 -- start start 2024-04-25T19:15:27.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.445+0000 7f88d5bff640 1 -- 172.21.15.76:0/1915103044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88d0163a60 msgr2=0x7f88d0175090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.445+0000 7f88d5bff640 1 --2- 172.21.15.76:0/1915103044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88d0163a60 0x7f88d0175090 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009b10 tx=0x7f88c802bc00 comp rx=0 tx=0).stop 2024-04-25T19:15:27.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.445+0000 7fa280535640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2780c8520 0x7fa278163e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.445+0000 7fa280535640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2780cdd60 0x7fa278162690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.445+0000 7fa280535640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa27808e060 con 0x7fa2780c8520 2024-04-25T19:15:27.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.445+0000 7fa280535640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa27808dee0 con 0x7fa27808d990 2024-04-25T19:15:27.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.445+0000 7fa280535640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa2780cc950 con 0x7fa2780cdd60 2024-04-25T19:15:27.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.445+0000 7f88d5bff640 1 -- 172.21.15.76:0/1915103044 shutdown_connections 2024-04-25T19:15:27.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.445+0000 7f88d5bff640 1 --2- 172.21.15.76:0/1915103044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88d0163a60 0x7f88d0175090 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.445+0000 7f88d5bff640 1 -- 172.21.15.76:0/1915103044 >> 172.21.15.76:0/1915103044 conn(0x7f88d0070800 msgr2=0x7f88d0070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.445+0000 7fa27dad0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2780c8520 0x7fa278163e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.445+0000 7fa27dad0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2780c8520 0x7fa278163e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59206/0 (socket says 172.21.15.76:59206) 2024-04-25T19:15:27.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.445+0000 7fa27dad0640 1 -- 172.21.15.76:0/2630922321 learned_addr learned my addr 172.21.15.76:0/2630922321 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:27.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.445+0000 7fa27e2d1640 1 --2- 172.21.15.76:0/2630922321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa27808d990 0x7fa278203c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.445+0000 7f88d5bff640 1 -- 172.21.15.76:0/1915103044 shutdown_connections 2024-04-25T19:15:27.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.445+0000 7fa27ead2640 1 --2- 172.21.15.76:0/2630922321 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2780cdd60 0x7fa278162690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.446+0000 7f921cbc0640 1 Processor -- start 2024-04-25T19:15:27.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.446+0000 7fa27dad0640 1 -- 172.21.15.76:0/2630922321 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2780cdd60 msgr2=0x7fa278162690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.446+0000 7fa27dad0640 1 --2- 172.21.15.76:0/2630922321 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2780cdd60 0x7fa278162690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.446+0000 7f921cbc0640 1 -- start start 2024-04-25T19:15:27.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.446+0000 7fa27dad0640 1 -- 172.21.15.76:0/2630922321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa27808d990 msgr2=0x7fa278203c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.446+0000 7fa27dad0640 1 --2- 172.21.15.76:0/2630922321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa27808d990 0x7fa278203c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.446+0000 7fa27dad0640 1 -- 172.21.15.76:0/2630922321 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2781649e0 con 0x7fa2780c8520 2024-04-25T19:15:27.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.446+0000 7fa27dad0640 1 --2- 172.21.15.76:0/2630922321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2780c8520 0x7fa278163e70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa26800c460 tx=0x7fa26800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.445+0000 7f88d5bff640 1 -- 172.21.15.76:0/1915103044 wait complete. 2024-04-25T19:15:27.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.446+0000 7fa3988e0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa39408d990 0x7fa394165260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.446+0000 7fa3988e0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3940c8520 0x7fa3940c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.446+0000 7fa39259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa39408d990 0x7fa394165260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.446+0000 7fdb24942640 1 -- 172.21.15.76:0/894505848 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1800e2a0 con 0x7fdb0c0052e0 2024-04-25T19:15:27.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.446+0000 7fa39259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa39408d990 0x7fa394165260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:42620/0 (socket says 172.21.15.76:42620) 2024-04-25T19:15:27.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.447+0000 7fdb24942640 1 -- 172.21.15.76:0/894505848 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb18007dc0 con 0x7fdb0c0052e0 2024-04-25T19:15:27.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.447+0000 7fa26effd640 1 -- 172.21.15.76:0/2630922321 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa268018070 con 0x7fa2780c8520 2024-04-25T19:15:27.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.447+0000 7fa280535640 1 -- 172.21.15.76:0/2630922321 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2781660f0 con 0x7fa2780c8520 2024-04-25T19:15:27.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.447+0000 7fa280535640 1 -- 172.21.15.76:0/2630922321 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2781666b0 con 0x7fa2780c8520 2024-04-25T19:15:27.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.447+0000 7f19837fe640 1 -- 172.21.15.76:0/1826037682 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1970039070 con 0x7f198408d990 2024-04-25T19:15:27.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.447+0000 7f19837fe640 1 -- 172.21.15.76:0/1826037682 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f197003d400 con 0x7f198408d990 2024-04-25T19:15:27.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.447+0000 7f19837fe640 1 -- 172.21.15.76:0/1826037682 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197003d7d0 con 0x7f198408d990 2024-04-25T19:15:27.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.447+0000 7fdb24942640 1 -- 172.21.15.76:0/894505848 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1802bcb0 con 0x7fdb0c0052e0 2024-04-25T19:15:27.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.447+0000 7fa3988e0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa3940cdd60 0x7fa394202030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.447+0000 7fa3988e0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa39408e060 con 0x7fa3940c8520 2024-04-25T19:15:27.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.447+0000 7fa3988e0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa39408dee0 con 0x7fa39408d990 2024-04-25T19:15:27.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.447+0000 7fa3988e0640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa3940cc950 con 0x7fa3940cdd60 2024-04-25T19:15:27.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7fa391d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3940c8520 0x7fa3940c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7fa39259c640 1 -- 172.21.15.76:0/964138709 learned_addr learned my addr 172.21.15.76:0/964138709 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:27.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7f198ac6e640 1 -- 172.21.15.76:0/1826037682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f198408d990 msgr2=0x7f1984163a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7f198ac6e640 1 --2- 172.21.15.76:0/1826037682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f198408d990 0x7f1984163a60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f197000b970 tx=0x7f1970033190 comp rx=0 tx=0).stop 2024-04-25T19:15:27.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.447+0000 7f8e31492640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e180055f0 0x7f8e2c164070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.447+0000 7f8e31492640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e2c165050 0x7f8e2c165db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7f8e2affd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e180055f0 0x7f8e2c164070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7f8e2affd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e180055f0 0x7f8e2c164070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:45882/0 (socket says 172.21.15.76:45882) 2024-04-25T19:15:27.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7f8e2affd640 1 -- 172.21.15.76:0/4172268895 learned_addr learned my addr 172.21.15.76:0/4172268895 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:27.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7f8e31492640 1 --2- 172.21.15.76:0/4172268895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e2c15d470 0x7f8e2c15d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7f8e31492640 1 -- 172.21.15.76:0/4172268895 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c08dee0 con 0x7f8e2c165050 2024-04-25T19:15:27.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7f8e31492640 1 -- 172.21.15.76:0/4172268895 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c0ccdc0 con 0x7f8e2c15d470 2024-04-25T19:15:27.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7f8e2b7fe640 1 --2- 172.21.15.76:0/4172268895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e2c15d470 0x7f8e2c15d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7f8e31492640 1 -- 172.21.15.76:0/4172268895 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8e2c08e060 con 0x7f8e180055f0 2024-04-25T19:15:27.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7f88d5bff640 1 Processor -- start 2024-04-25T19:15:27.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7f88d5bff640 1 -- start start 2024-04-25T19:15:27.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7fdb27ba8640 1 -- 172.21.15.76:0/894505848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb0c0052e0 msgr2=0x7fdb0c005710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7fdb27ba8640 1 --2- 172.21.15.76:0/894505848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb0c0052e0 0x7fdb0c005710 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdb200c47f0 tx=0x7fdb0c00afc0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7fdb26145640 1 -- 172.21.15.76:0/894505848 reap_dead start 2024-04-25T19:15:27.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7fdb27ba8640 1 -- 172.21.15.76:0/894505848 shutdown_connections 2024-04-25T19:15:27.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7fdb27ba8640 1 -- 172.21.15.76:0/894505848 >> 172.21.15.76:0/894505848 conn(0x7fdb20070800 msgr2=0x7fdb20070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7fdb27ba8640 1 -- 172.21.15.76:0/894505848 shutdown_connections 2024-04-25T19:15:27.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7fdb27ba8640 1 -- 172.21.15.76:0/894505848 wait complete. 2024-04-25T19:15:27.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7f921cbc0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f921808d990 0x7f9218162a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7f921cbc0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92180c8520 0x7f92180c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7f921cbc0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92180cdd60 0x7f9218073a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7f921659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f921808d990 0x7f9218162a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7f921659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f921808d990 0x7f9218162a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:42644/0 (socket says 172.21.15.76:42644) 2024-04-25T19:15:27.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7f921659c640 1 -- 172.21.15.76:0/555275107 learned_addr learned my addr 172.21.15.76:0/555275107 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:27.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7f921cbc0640 1 -- 172.21.15.76:0/555275107 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f921808e060 con 0x7f92180c8520 2024-04-25T19:15:27.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7f9216d9d640 1 --2- 172.21.15.76:0/555275107 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92180cdd60 0x7f9218073a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7f921cbc0640 1 -- 172.21.15.76:0/555275107 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f921808dee0 con 0x7f921808d990 2024-04-25T19:15:27.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7f921cbc0640 1 -- 172.21.15.76:0/555275107 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f92180cc950 con 0x7f92180cdd60 2024-04-25T19:15:27.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7f8e2affd640 1 -- 172.21.15.76:0/4172268895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e2c15d470 msgr2=0x7f8e2c15d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7f8e2affd640 1 --2- 172.21.15.76:0/4172268895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e2c15d470 0x7f8e2c15d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7f8e2affd640 1 -- 172.21.15.76:0/4172268895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e2c165050 msgr2=0x7f8e2c165db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7f8e2affd640 1 --2- 172.21.15.76:0/4172268895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e2c165050 0x7f8e2c165db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7fa392d9d640 1 --2- 172.21.15.76:0/964138709 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa3940cdd60 0x7fa394202030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.448+0000 7fa280535640 1 -- 172.21.15.76:0/2630922321 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa27806a390 con 0x7fa2780c8520 2024-04-25T19:15:27.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7f8e2affd640 1 -- 172.21.15.76:0/4172268895 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e2c203720 con 0x7f8e180055f0 2024-04-25T19:15:27.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7f8e2b7fe640 1 --2- 172.21.15.76:0/4172268895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e2c15d470 0x7f8e2c15d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:27.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7f198ac6e640 1 -- 172.21.15.76:0/1826037682 shutdown_connections 2024-04-25T19:15:27.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7f198ac6e640 1 --2- 172.21.15.76:0/1826037682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f198408d990 0x7f1984163a60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7f198ac6e640 1 --2- 172.21.15.76:0/1826037682 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19840cdd60 0x7f19840ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7f9215d9b640 1 --2- 172.21.15.76:0/555275107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92180c8520 0x7f92180c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7f198ac6e640 1 -- 172.21.15.76:0/1826037682 >> 172.21.15.76:0/1826037682 conn(0x7f1984070800 msgr2=0x7f1984070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7fa39259c640 1 -- 172.21.15.76:0/964138709 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa3940cdd60 msgr2=0x7fa394202030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7fa39259c640 1 --2- 172.21.15.76:0/964138709 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa3940cdd60 0x7fa394202030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7fa39259c640 1 -- 172.21.15.76:0/964138709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3940c8520 msgr2=0x7fa3940c2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7fa39259c640 1 --2- 172.21.15.76:0/964138709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3940c8520 0x7fa3940c2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7fa39259c640 1 -- 172.21.15.76:0/964138709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3941657a0 con 0x7fa39408d990 2024-04-25T19:15:27.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7fa26effd640 1 -- 172.21.15.76:0/2630922321 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2680074e0 con 0x7fa2780c8520 2024-04-25T19:15:27.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7f198ac6e640 1 -- 172.21.15.76:0/1826037682 shutdown_connections 2024-04-25T19:15:27.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7fa26effd640 1 -- 172.21.15.76:0/2630922321 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2680049e0 con 0x7fa2780c8520 2024-04-25T19:15:27.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7f198ac6e640 1 -- 172.21.15.76:0/1826037682 wait complete. 2024-04-25T19:15:27.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7f8e2affd640 1 --2- 172.21.15.76:0/4172268895 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e180055f0 0x7f8e2c164070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00f7a0 tx=0x7f8e1c00fc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.449+0000 7fa39259c640 1 --2- 172.21.15.76:0/964138709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa39408d990 0x7fa394165260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa384034040 tx=0x7fa3840046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.450+0000 7fa382ffd640 1 -- 172.21.15.76:0/964138709 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3840092e0 con 0x7fa39408d990 2024-04-25T19:15:27.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.450+0000 7fa382ffd640 1 -- 172.21.15.76:0/964138709 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa384043070 con 0x7fa39408d990 2024-04-25T19:15:27.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.450+0000 7fa382ffd640 1 -- 172.21.15.76:0/964138709 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38403c8a0 con 0x7fa39408d990 2024-04-25T19:15:27.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.450+0000 7fa3988e0640 1 -- 172.21.15.76:0/964138709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa394164b30 con 0x7fa39408d990 2024-04-25T19:15:27.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.450+0000 7fa3988e0640 1 -- 172.21.15.76:0/964138709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3942088b0 con 0x7fa39408d990 2024-04-25T19:15:27.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.450+0000 7f8e237fe640 1 -- 172.21.15.76:0/4172268895 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c0052e0 con 0x7f8e180055f0 2024-04-25T19:15:27.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.450+0000 7f8e237fe640 1 -- 172.21.15.76:0/4172268895 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e1c018070 con 0x7f8e180055f0 2024-04-25T19:15:27.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.450+0000 7f8e31492640 1 -- 172.21.15.76:0/4172268895 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e2c2039e0 con 0x7f8e180055f0 2024-04-25T19:15:27.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.450+0000 7f8e31492640 1 -- 172.21.15.76:0/4172268895 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e2c16ae90 con 0x7f8e180055f0 2024-04-25T19:15:27.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.451+0000 7f88d5bff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88d008d990 0x7f88d02044d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.451+0000 7f8e237fe640 1 -- 172.21.15.76:0/4172268895 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c00d970 con 0x7f8e180055f0 2024-04-25T19:15:27.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.452+0000 7f8e31492640 1 -- 172.21.15.76:0/4172268895 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e2c06a390 con 0x7f8e180055f0 2024-04-25T19:15:27.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.451+0000 7fdb27ba8640 1 Processor -- start 2024-04-25T19:15:27.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.452+0000 7fdb27ba8640 1 -- start start 2024-04-25T19:15:27.493 INFO:teuthology.orchestra.run.smithi076.stdout:55834574853 2024-04-25T19:15:27.493 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T19:15:27.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.452+0000 7fa26effd640 1 -- 172.21.15.76:0/2630922321 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa26801b070 con 0x7fa2780c8520 2024-04-25T19:15:27.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.451+0000 7f88d5bff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88d00c8520 0x7f88d01649e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.452+0000 7f88d5bff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88d0163a60 0x7f88d0164f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.452+0000 7f88ceffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88d00c8520 0x7f88d01649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.452+0000 7f88ceffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88d00c8520 0x7f88d01649e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:45908/0 (socket says 172.21.15.76:45908) 2024-04-25T19:15:27.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.452+0000 7f88ceffd640 1 -- 172.21.15.76:0/138892728 learned_addr learned my addr 172.21.15.76:0/138892728 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:27.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.452+0000 7f88cffff640 1 --2- 172.21.15.76:0/138892728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88d0163a60 0x7f88d0164f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.452+0000 7f88d5bff640 1 -- 172.21.15.76:0/138892728 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f88d008dee0 con 0x7f88d008d990 2024-04-25T19:15:27.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.452+0000 7f88d5bff640 1 -- 172.21.15.76:0/138892728 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f88d00ccdc0 con 0x7f88d0163a60 2024-04-25T19:15:27.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.452+0000 7f88d5bff640 1 -- 172.21.15.76:0/138892728 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f88d008e060 con 0x7f88d00c8520 2024-04-25T19:15:27.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.453+0000 7fa382ffd640 1 -- 172.21.15.76:0/964138709 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa38403ca40 con 0x7fa39408d990 2024-04-25T19:15:27.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.453+0000 7fa26effd640 1 --2- 172.21.15.76:0/2630922321 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa25c0744b0 0x7fa25c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.453+0000 7fa26effd640 1 -- 172.21.15.76:0/2630922321 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa2680946d0 con 0x7fa2780c8520 2024-04-25T19:15:27.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.453+0000 7fa27e2d1640 1 --2- 172.21.15.76:0/2630922321 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa25c0744b0 0x7fa25c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.453+0000 7f198ac6e640 1 Processor -- start 2024-04-25T19:15:27.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.453+0000 7f198ac6e640 1 -- start start 2024-04-25T19:15:27.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.453+0000 7f88cf7fe640 1 --2- 172.21.15.76:0/138892728 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88d008d990 0x7f88d02044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.453+0000 7f88ceffd640 1 -- 172.21.15.76:0/138892728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88d0163a60 msgr2=0x7f88d0164f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.453+0000 7f88ceffd640 1 --2- 172.21.15.76:0/138892728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88d0163a60 0x7f88d0164f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.453+0000 7f88ceffd640 1 -- 172.21.15.76:0/138892728 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88d008d990 msgr2=0x7f88d02044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.453+0000 7f88ceffd640 1 --2- 172.21.15.76:0/138892728 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88d008d990 0x7f88d02044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.453+0000 7f88ceffd640 1 -- 172.21.15.76:0/138892728 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d0204a10 con 0x7f88d00c8520 2024-04-25T19:15:27.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.453+0000 7f88ceffd640 1 --2- 172.21.15.76:0/138892728 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88d00c8520 0x7f88d01649e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f88c400c410 tx=0x7f88c400c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.453+0000 7fdb27ba8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb20165810 0x7fdb2015d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.453+0000 7fdb27ba8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb200ca810 0x7fdb200cac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.453+0000 7fdb27ba8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb20202de0 0x7fdb202031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.453+0000 7fdb27ba8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2008dee0 con 0x7fdb20202de0 2024-04-25T19:15:27.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.453+0000 7fdb27ba8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdb200ccdc0 con 0x7fdb200ca810 2024-04-25T19:15:27.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.453+0000 7fdb27ba8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fdb2008e060 con 0x7fdb20165810 2024-04-25T19:15:27.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.451+0000 7f921659c640 1 -- 172.21.15.76:0/555275107 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92180cdd60 msgr2=0x7f9218073a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.451+0000 7f921659c640 1 --2- 172.21.15.76:0/555275107 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92180cdd60 0x7f9218073a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.451+0000 7f921659c640 1 -- 172.21.15.76:0/555275107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92180c8520 msgr2=0x7f92180c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.451+0000 7f921659c640 1 --2- 172.21.15.76:0/555275107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92180c8520 0x7f92180c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.451+0000 7f921659c640 1 -- 172.21.15.76:0/555275107 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9218162830 con 0x7f921808d990 2024-04-25T19:15:27.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.452+0000 7f921659c640 1 --2- 172.21.15.76:0/555275107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f921808d990 0x7f9218162a40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9200034040 tx=0x7f92000046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.453+0000 7f920effd640 1 -- 172.21.15.76:0/555275107 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200042020 con 0x7f921808d990 2024-04-25T19:15:27.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.453+0000 7f920effd640 1 -- 172.21.15.76:0/555275107 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f920003e880 con 0x7f921808d990 2024-04-25T19:15:27.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.454+0000 7fdb25143640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb200ca810 0x7fdb200cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.454+0000 7fdb25143640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb200ca810 0x7fdb200cac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:42666/0 (socket says 172.21.15.76:42666) 2024-04-25T19:15:27.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.453+0000 7f88b7fff640 1 -- 172.21.15.76:0/138892728 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4015e30 con 0x7f88d00c8520 2024-04-25T19:15:27.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.454+0000 7f88b7fff640 1 -- 172.21.15.76:0/138892728 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88c4019030 con 0x7f88d00c8520 2024-04-25T19:15:27.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.454+0000 7f88b7fff640 1 -- 172.21.15.76:0/138892728 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4005ab0 con 0x7f88d00c8520 2024-04-25T19:15:27.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.454+0000 7fdb25143640 1 -- 172.21.15.76:0/907215813 learned_addr learned my addr 172.21.15.76:0/907215813 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:27.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.454+0000 7fdb25944640 1 --2- 172.21.15.76:0/907215813 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb20165810 0x7fdb2015d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.454+0000 7f88d5bff640 1 -- 172.21.15.76:0/138892728 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d016c560 con 0x7f88d00c8520 2024-04-25T19:15:27.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.454+0000 7f88d5bff640 1 -- 172.21.15.76:0/138892728 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d0207d00 con 0x7f88d00c8520 2024-04-25T19:15:27.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.454+0000 7fa382ffd640 1 --2- 172.21.15.76:0/964138709 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa374074580 0x7fa374076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.454+0000 7fa382ffd640 1 -- 172.21.15.76:0/964138709 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa3840b5c70 con 0x7fa39408d990 2024-04-25T19:15:27.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.454+0000 7f920effd640 1 -- 172.21.15.76:0/555275107 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920003ec50 con 0x7f921808d990 2024-04-25T19:15:27.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.454+0000 7fa391d9b640 1 --2- 172.21.15.76:0/964138709 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa374074580 0x7fa374076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.454+0000 7f921cbc0640 1 -- 172.21.15.76:0/555275107 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92181622b0 con 0x7f921808d990 2024-04-25T19:15:27.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.454+0000 7fa3988e0640 1 -- 172.21.15.76:0/964138709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa378005290 con 0x7fa39408d990 2024-04-25T19:15:27.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.454+0000 7f921cbc0640 1 -- 172.21.15.76:0/555275107 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9218162550 con 0x7f921808d990 2024-04-25T19:15:27.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.454+0000 7fdb26145640 1 --2- 172.21.15.76:0/907215813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb20202de0 0x7fdb202031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.455+0000 7fdb26145640 1 -- 172.21.15.76:0/907215813 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb20165810 msgr2=0x7fdb2015d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.455+0000 7fdb26145640 1 --2- 172.21.15.76:0/907215813 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb20165810 0x7fdb2015d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.455+0000 7fdb26145640 1 -- 172.21.15.76:0/907215813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb200ca810 msgr2=0x7fdb200cac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.455+0000 7fdb26145640 1 --2- 172.21.15.76:0/907215813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb200ca810 0x7fdb200cac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.455+0000 7fdb26145640 1 -- 172.21.15.76:0/907215813 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb20166070 con 0x7fdb20202de0 2024-04-25T19:15:27.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.455+0000 7f198ac6e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f198408d990 0x7f1984168d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.455+0000 7fa27e2d1640 1 --2- 172.21.15.76:0/2630922321 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa25c0744b0 0x7fa25c0769c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa274035000 tx=0x7fa274008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.455+0000 7fdb26145640 1 --2- 172.21.15.76:0/907215813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb20202de0 0x7fdb202031f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdb18031000 tx=0x7fdb18004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.455+0000 7f198ac6e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19840c8520 0x7f19840c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.455+0000 7f8e237fe640 1 -- 172.21.15.76:0/4172268895 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8e1c00db10 con 0x7f8e180055f0 2024-04-25T19:15:27.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.455+0000 7f1988a0a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f198408d990 0x7f1984168d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.456+0000 7f1988a0a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f198408d990 0x7f1984168d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:45934/0 (socket says 172.21.15.76:45934) 2024-04-25T19:15:27.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.455+0000 7fa391d9b640 1 --2- 172.21.15.76:0/964138709 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa374074580 0x7fa374076a90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa39415e5c0 tx=0x7fa388009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.456+0000 7f198ac6e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19840cdd60 0x7f198415d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.456+0000 7f198ac6e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f198408e060 con 0x7f19840c8520 2024-04-25T19:15:27.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.456+0000 7f198ac6e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f198408dee0 con 0x7f19840cdd60 2024-04-25T19:15:27.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.456+0000 7f198ac6e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f19840cc950 con 0x7f198408d990 2024-04-25T19:15:27.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.456+0000 7f1988a0a640 1 -- 172.21.15.76:0/1581027039 learned_addr learned my addr 172.21.15.76:0/1581027039 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:27.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.456+0000 7f88d5bff640 1 -- 172.21.15.76:0/138892728 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88d006a390 con 0x7f88d00c8520 2024-04-25T19:15:27.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.456+0000 7f921cbc0640 1 -- 172.21.15.76:0/555275107 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f921806a390 con 0x7f921808d990 2024-04-25T19:15:27.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.456+0000 7f1983fff640 1 --2- 172.21.15.76:0/1581027039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19840c8520 0x7f19840c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.456+0000 7f8e237fe640 1 --2- 172.21.15.76:0/4172268895 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8e0c074530 0x7f8e0c076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.456+0000 7f198920b640 1 --2- 172.21.15.76:0/1581027039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19840cdd60 0x7f198415d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.456+0000 7f1988a0a640 1 -- 172.21.15.76:0/1581027039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19840cdd60 msgr2=0x7f198415d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.456+0000 7f1988a0a640 1 --2- 172.21.15.76:0/1581027039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19840cdd60 0x7f198415d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.456+0000 7f1988a0a640 1 -- 172.21.15.76:0/1581027039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19840c8520 msgr2=0x7f19840c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.456+0000 7f8e23fff640 1 --2- 172.21.15.76:0/4172268895 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8e0c074530 0x7f8e0c076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.456+0000 7f8e237fe640 1 -- 172.21.15.76:0/4172268895 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8e1c095080 con 0x7f8e180055f0 2024-04-25T19:15:27.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.456+0000 7f1988a0a640 1 --2- 172.21.15.76:0/1581027039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19840c8520 0x7f19840c9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.456+0000 7f1988a0a640 1 -- 172.21.15.76:0/1581027039 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1984169270 con 0x7f198408d990 2024-04-25T19:15:27.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.456+0000 7f88b7fff640 1 -- 172.21.15.76:0/138892728 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f88c4005c50 con 0x7f88d00c8520 2024-04-25T19:15:27.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.456+0000 7f1983fff640 1 --2- 172.21.15.76:0/1581027039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19840c8520 0x7f19840c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:27.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.457+0000 7f8e23fff640 1 --2- 172.21.15.76:0/4172268895 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8e0c074530 0x7f8e0c076a40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8e14033000 tx=0x7f8e14007000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.457+0000 7f1988a0a640 1 --2- 172.21.15.76:0/1581027039 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f198408d990 0x7f1984168d30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f197800c430 tx=0x7f197800c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.457+0000 7f88b7fff640 1 --2- 172.21.15.76:0/138892728 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f88bc0744b0 0x7f88bc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.457+0000 7f88b7fff640 1 -- 172.21.15.76:0/138892728 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f88c409ce00 con 0x7f88d00c8520 2024-04-25T19:15:27.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.457+0000 7f88cf7fe640 1 --2- 172.21.15.76:0/138892728 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f88bc0744b0 0x7f88bc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.457+0000 7f920effd640 1 -- 172.21.15.76:0/555275107 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9200045020 con 0x7f921808d990 2024-04-25T19:15:27.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.457+0000 7f88cf7fe640 1 --2- 172.21.15.76:0/138892728 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f88bc0744b0 0x7f88bc0769c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f88c00011e0 tx=0x7f88c0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.458+0000 7f920effd640 1 --2- 172.21.15.76:0/555275107 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f91f8074500 0x7f91f8076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.458+0000 7f9215d9b640 1 --2- 172.21.15.76:0/555275107 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f91f8074500 0x7f91f8076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.458+0000 7f9215d9b640 1 --2- 172.21.15.76:0/555275107 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f91f8074500 0x7f91f8076a10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9208001a90 tx=0x7f9208001a20 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.459+0000 7fdb167fc640 1 -- 172.21.15.76:0/907215813 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb18043070 con 0x7fdb20202de0 2024-04-25T19:15:27.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.459+0000 7fdb167fc640 1 -- 172.21.15.76:0/907215813 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb18007bc0 con 0x7fdb20202de0 2024-04-25T19:15:27.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.459+0000 7fdb167fc640 1 -- 172.21.15.76:0/907215813 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c007d70 con 0x7fdb20202de0 2024-04-25T19:15:27.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.459+0000 7fdb27ba8640 1 -- 172.21.15.76:0/907215813 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb2015fe30 con 0x7fdb20202de0 2024-04-25T19:15:27.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.459+0000 7fa26effd640 1 -- 172.21.15.76:0/2630922321 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa26805eda0 con 0x7fa2780c8520 2024-04-25T19:15:27.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.459+0000 7fdb27ba8640 1 -- 172.21.15.76:0/907215813 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb201603f0 con 0x7fdb20202de0 2024-04-25T19:15:27.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.461+0000 7f920effd640 1 -- 172.21.15.76:0/555275107 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f92000b4300 con 0x7f921808d990 2024-04-25T19:15:27.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.461+0000 7fa382ffd640 1 -- 172.21.15.76:0/964138709 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa3840803e0 con 0x7fa39408d990 2024-04-25T19:15:27.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.461+0000 7fdb27ba8640 1 -- 172.21.15.76:0/907215813 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb2006a390 con 0x7fdb20202de0 2024-04-25T19:15:27.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.461+0000 7fdb167fc640 1 -- 172.21.15.76:0/907215813 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdb18046070 con 0x7fdb20202de0 2024-04-25T19:15:27.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.462+0000 7f8e237fe640 1 -- 172.21.15.76:0/4172268895 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8e1c05f870 con 0x7f8e180055f0 2024-04-25T19:15:27.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.462+0000 7fdb167fc640 1 --2- 172.21.15.76:0/907215813 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fdb040744b0 0x7fdb040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.462+0000 7fdb167fc640 1 -- 172.21.15.76:0/907215813 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdb180b5670 con 0x7fdb20202de0 2024-04-25T19:15:27.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.463+0000 7fdb25944640 1 --2- 172.21.15.76:0/907215813 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fdb040744b0 0x7fdb040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.463+0000 7fdb25944640 1 --2- 172.21.15.76:0/907215813 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fdb040744b0 0x7fdb040769c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c001bb0 tx=0x7fdb1c001b20 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.466+0000 7f920effd640 1 -- 172.21.15.76:0/555275107 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f920004c030 con 0x7f921808d990 2024-04-25T19:15:27.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.467+0000 7f88b7fff640 1 -- 172.21.15.76:0/138892728 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f88c4066f40 con 0x7f88d00c8520 2024-04-25T19:15:27.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.467+0000 7f19817fa640 1 -- 172.21.15.76:0/1581027039 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978017070 con 0x7f198408d990 2024-04-25T19:15:27.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.467+0000 7f19817fa640 1 -- 172.21.15.76:0/1581027039 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19780052f0 con 0x7f198408d990 2024-04-25T19:15:27.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.467+0000 7f19817fa640 1 -- 172.21.15.76:0/1581027039 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978003830 con 0x7f198408d990 2024-04-25T19:15:27.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.467+0000 7f198ac6e640 1 -- 172.21.15.76:0/1581027039 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1984167c90 con 0x7f198408d990 2024-04-25T19:15:27.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.467+0000 7f198ac6e640 1 -- 172.21.15.76:0/1581027039 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1984168250 con 0x7f198408d990 2024-04-25T19:15:27.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.469+0000 7f198ac6e640 1 -- 172.21.15.76:0/1581027039 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f198406a390 con 0x7f198408d990 2024-04-25T19:15:27.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.469+0000 7f19817fa640 1 -- 172.21.15.76:0/1581027039 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f197801a070 con 0x7f198408d990 2024-04-25T19:15:27.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.470+0000 7f19817fa640 1 --2- 172.21.15.76:0/1581027039 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f19680744b0 0x7f19680769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.470+0000 7f19817fa640 1 -- 172.21.15.76:0/1581027039 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f19780944e0 con 0x7f198408d990 2024-04-25T19:15:27.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.470+0000 7f1983fff640 1 --2- 172.21.15.76:0/1581027039 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f19680744b0 0x7f19680769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.471+0000 7f1983fff640 1 --2- 172.21.15.76:0/1581027039 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f19680744b0 0x7f19680769c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f197003e000 tx=0x7f1970039000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.472+0000 7fdb167fc640 1 -- 172.21.15.76:0/907215813 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdb1800c040 con 0x7fdb20202de0 2024-04-25T19:15:27.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.475+0000 7f19817fa640 1 -- 172.21.15.76:0/1581027039 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f197805e6b0 con 0x7f198408d990 2024-04-25T19:15:27.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.692+0000 7fdd4d6bd640 1 Processor -- start 2024-04-25T19:15:27.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.692+0000 7fdd4d6bd640 1 -- start start 2024-04-25T19:15:27.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.695+0000 7fdd4d6bd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdd480c9540 0x7fdd480ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.695+0000 7fdd4d6bd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd48152f20 0x7fdd48158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.695+0000 7fdd4d6bd640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd480cde50 con 0x7fdd480c9b90 2024-04-25T19:15:27.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.695+0000 7fdd4d6bd640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fdd480c5780 con 0x7fdd480c9540 2024-04-25T19:15:27.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.695+0000 7fdd4d6bd640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fdd480c8c10 con 0x7fdd48152f20 2024-04-25T19:15:27.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.695+0000 7fdd467fc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd48152f20 0x7fdd48158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.695+0000 7fdd467fc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd48152f20 0x7fdd48158ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:45944/0 (socket says 172.21.15.76:45944) 2024-04-25T19:15:27.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.695+0000 7fdd467fc640 1 -- 172.21.15.76:0/638247635 learned_addr learned my addr 172.21.15.76:0/638247635 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:27.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.695+0000 7fdd46ffd640 1 --2- 172.21.15.76:0/638247635 >> v2:172.21.15.105:3300/0 conn(0x7fdd480c9540 0x7fdd480ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.696+0000 7fdd467fc640 1 -- 172.21.15.76:0/638247635 >> v2:172.21.15.105:3300/0 conn(0x7fdd480c9540 msgr2=0x7fdd480ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:27.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.696+0000 7fdd467fc640 1 --2- 172.21.15.76:0/638247635 >> v2:172.21.15.105:3300/0 conn(0x7fdd480c9540 0x7fdd480ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.696+0000 7fdd467fc640 1 -- 172.21.15.76:0/638247635 >> v1:172.21.15.105:6789/0 conn(0x7fdd480c9b90 legacy=0x7fdd480cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:27.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.696+0000 7fdd467fc640 1 -- 172.21.15.76:0/638247635 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd480c8ef0 con 0x7fdd48152f20 2024-04-25T19:15:27.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.697+0000 7fdd467fc640 1 --2- 172.21.15.76:0/638247635 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd48152f20 0x7fdd48158ad0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdd40009bd0 tx=0x7fdd4002cd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93f09e7ee4ae033 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.697+0000 7fdd45ffb640 1 -- 172.21.15.76:0/638247635 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4002d810 con 0x7fdd48152f20 2024-04-25T19:15:27.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.697+0000 7fdd45ffb640 1 -- 172.21.15.76:0/638247635 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd4002d9b0 con 0x7fdd48152f20 2024-04-25T19:15:27.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.699+0000 7fdd4d6bd640 1 -- 172.21.15.76:0/638247635 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd48152f20 msgr2=0x7fdd48158ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.699+0000 7fdd4d6bd640 1 --2- 172.21.15.76:0/638247635 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd48152f20 0x7fdd48158ad0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdd40009bd0 tx=0x7fdd4002cd10 comp rx=0 tx=0).stop 2024-04-25T19:15:27.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.699+0000 7fdd4d6bd640 1 -- 172.21.15.76:0/638247635 shutdown_connections 2024-04-25T19:15:27.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.699+0000 7fdd4d6bd640 1 --2- 172.21.15.76:0/638247635 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd48152f20 0x7fdd48158ad0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.699+0000 7fdd4d6bd640 1 --2- 172.21.15.76:0/638247635 >> v2:172.21.15.105:3300/0 conn(0x7fdd480c9540 0x7fdd480ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.699+0000 7fdd4d6bd640 1 -- 172.21.15.76:0/638247635 >> 172.21.15.76:0/638247635 conn(0x7fdd48070e70 msgr2=0x7fdd480ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.699+0000 7fdd4d6bd640 1 -- 172.21.15.76:0/638247635 shutdown_connections 2024-04-25T19:15:27.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.699+0000 7fdd4d6bd640 1 -- 172.21.15.76:0/638247635 wait complete. 2024-04-25T19:15:27.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.702+0000 7fdd4d6bd640 1 Processor -- start 2024-04-25T19:15:27.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.702+0000 7fdd4d6bd640 1 -- start start 2024-04-25T19:15:27.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.705+0000 7fdd4d6bd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd480c9540 0x7fdd48202cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.705+0000 7fdd4d6bd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd480c9b90 0x7fdd48153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.705+0000 7fdd4d6bd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd481f16c0 0x7fdd4814f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.705+0000 7fdd46ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd480c9540 0x7fdd48202cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.705+0000 7fdd46ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd480c9540 0x7fdd48202cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:42696/0 (socket says 172.21.15.76:42696) 2024-04-25T19:15:27.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.705+0000 7fdd477fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd481f16c0 0x7fdd4814f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.705+0000 7fdd477fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd481f16c0 0x7fdd4814f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59268/0 (socket says 172.21.15.76:59268) 2024-04-25T19:15:27.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.706+0000 7fdd477fe640 1 -- 172.21.15.76:0/801402944 learned_addr learned my addr 172.21.15.76:0/801402944 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:27.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.706+0000 7fdd4d6bd640 1 -- 172.21.15.76:0/801402944 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdd48077fb0 con 0x7fdd481f16c0 2024-04-25T19:15:27.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.706+0000 7fdd4d6bd640 1 -- 172.21.15.76:0/801402944 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdd48077e30 con 0x7fdd480c9540 2024-04-25T19:15:27.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.706+0000 7fdd4d6bd640 1 -- 172.21.15.76:0/801402944 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fdd480c7990 con 0x7fdd480c9b90 2024-04-25T19:15:27.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.706+0000 7fdd477fe640 1 -- 172.21.15.76:0/801402944 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd480c9b90 msgr2=0x7fdd48153230 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T19:15:27.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.706+0000 7fdd477fe640 1 --2- 172.21.15.76:0/801402944 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd480c9b90 0x7fdd48153230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.706+0000 7fdd477fe640 1 -- 172.21.15.76:0/801402944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd480c9540 msgr2=0x7fdd48202cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.706+0000 7fdd477fe640 1 --2- 172.21.15.76:0/801402944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd480c9540 0x7fdd48202cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.706+0000 7fdd477fe640 1 -- 172.21.15.76:0/801402944 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd48203210 con 0x7fdd481f16c0 2024-04-25T19:15:27.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.707+0000 7fdd477fe640 1 --2- 172.21.15.76:0/801402944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd481f16c0 0x7fdd4814f730 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800b780 tx=0x7fdd3800bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.708+0000 7fdd46ffd640 1 --2- 172.21.15.76:0/801402944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd480c9540 0x7fdd48202cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:15:27.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.708+0000 7fdd2f7fe640 1 -- 172.21.15.76:0/801402944 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd38004220 con 0x7fdd481f16c0 2024-04-25T19:15:27.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.708+0000 7fdd4d6bd640 1 -- 172.21.15.76:0/801402944 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd48154d80 con 0x7fdd481f16c0 2024-04-25T19:15:27.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.708+0000 7fdd4d6bd640 1 -- 172.21.15.76:0/801402944 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd4815a8e0 con 0x7fdd481f16c0 2024-04-25T19:15:27.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.713+0000 7fdd2f7fe640 1 -- 172.21.15.76:0/801402944 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd380043c0 con 0x7fdd481f16c0 2024-04-25T19:15:27.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.713+0000 7fdd2f7fe640 1 -- 172.21.15.76:0/801402944 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd380037f0 con 0x7fdd481f16c0 2024-04-25T19:15:27.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.713+0000 7fdd2f7fe640 1 -- 172.21.15.76:0/801402944 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdd38003a10 con 0x7fdd481f16c0 2024-04-25T19:15:27.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.713+0000 7fdd2f7fe640 1 --2- 172.21.15.76:0/801402944 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fdd30074720 0x7fdd30076c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:27.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.714+0000 7fdd4d6bd640 1 -- 172.21.15.76:0/801402944 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd34005290 con 0x7fdd481f16c0 2024-04-25T19:15:27.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.724+0000 7fdd46ffd640 1 --2- 172.21.15.76:0/801402944 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fdd30074720 0x7fdd30076c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:27.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.724+0000 7fdd2f7fe640 1 -- 172.21.15.76:0/801402944 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdd380948c0 con 0x7fdd481f16c0 2024-04-25T19:15:27.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.724+0000 7fdd46ffd640 1 --2- 172.21.15.76:0/801402944 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fdd30074720 0x7fdd30076c30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c006fd0 tx=0x7fdd3c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:27.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.725+0000 7fdd2f7fe640 1 -- 172.21.15.76:0/801402944 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdd3805f030 con 0x7fdd481f16c0 2024-04-25T19:15:27.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.827+0000 7f198ac6e640 1 -- 172.21.15.76:0/1581027039 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f19840cd0d0 con 0x7f198408d990 2024-04-25T19:15:27.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.828+0000 7f19817fa640 1 -- 172.21.15.76:0/1581027039 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f19780629b0 con 0x7f198408d990 2024-04-25T19:15:27.832 INFO:teuthology.orchestra.run.smithi076.stdout:55834574852 2024-04-25T19:15:27.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.832+0000 7f198ac6e640 1 -- 172.21.15.76:0/1581027039 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f19680744b0 msgr2=0x7f19680769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.832+0000 7f198ac6e640 1 --2- 172.21.15.76:0/1581027039 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f19680744b0 0x7f19680769c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f197003e000 tx=0x7f1970039000 comp rx=0 tx=0).stop 2024-04-25T19:15:27.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.832+0000 7f198ac6e640 1 -- 172.21.15.76:0/1581027039 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f198408d990 msgr2=0x7f1984168d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.832+0000 7f198ac6e640 1 --2- 172.21.15.76:0/1581027039 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f198408d990 0x7f1984168d30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f197800c430 tx=0x7f197800c8f0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.833+0000 7f198ac6e640 1 -- 172.21.15.76:0/1581027039 shutdown_connections 2024-04-25T19:15:27.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.833+0000 7f198ac6e640 1 --2- 172.21.15.76:0/1581027039 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f19680744b0 0x7f19680769c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.833+0000 7f198ac6e640 1 --2- 172.21.15.76:0/1581027039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19840cdd60 0x7f198415d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.833+0000 7f198ac6e640 1 --2- 172.21.15.76:0/1581027039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19840c8520 0x7f19840c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.833+0000 7f198ac6e640 1 --2- 172.21.15.76:0/1581027039 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f198408d990 0x7f1984168d30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.833+0000 7f198ac6e640 1 -- 172.21.15.76:0/1581027039 >> 172.21.15.76:0/1581027039 conn(0x7f1984070800 msgr2=0x7f198415f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.834+0000 7f198ac6e640 1 -- 172.21.15.76:0/1581027039 shutdown_connections 2024-04-25T19:15:27.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.834+0000 7f198ac6e640 1 -- 172.21.15.76:0/1581027039 wait complete. 2024-04-25T19:15:27.853 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2024-04-25T19:15:27.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.864+0000 7fa280535640 1 -- 172.21.15.76:0/2630922321 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fa2780cd0d0 con 0x7fa2780c8520 2024-04-25T19:15:27.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.864+0000 7fa26effd640 1 -- 172.21.15.76:0/2630922321 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa26805e6a0 con 0x7fa2780c8520 2024-04-25T19:15:27.866 INFO:teuthology.orchestra.run.smithi076.stdout:55834574852 2024-04-25T19:15:27.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.866+0000 7fa280535640 1 -- 172.21.15.76:0/2630922321 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa25c0744b0 msgr2=0x7fa25c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.866+0000 7fa280535640 1 --2- 172.21.15.76:0/2630922321 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa25c0744b0 0x7fa25c0769c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa274035000 tx=0x7fa274008040 comp rx=0 tx=0).stop 2024-04-25T19:15:27.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.866+0000 7fa280535640 1 -- 172.21.15.76:0/2630922321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2780c8520 msgr2=0x7fa278163e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.866+0000 7fa280535640 1 --2- 172.21.15.76:0/2630922321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2780c8520 0x7fa278163e70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa26800c460 tx=0x7fa26800c920 comp rx=0 tx=0).stop 2024-04-25T19:15:27.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.866+0000 7fa280535640 1 -- 172.21.15.76:0/2630922321 shutdown_connections 2024-04-25T19:15:27.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.866+0000 7fa280535640 1 --2- 172.21.15.76:0/2630922321 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa25c0744b0 0x7fa25c0769c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.866+0000 7fa280535640 1 --2- 172.21.15.76:0/2630922321 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2780cdd60 0x7fa278162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.866+0000 7fa280535640 1 --2- 172.21.15.76:0/2630922321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2780c8520 0x7fa278163e70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.866+0000 7fa280535640 1 --2- 172.21.15.76:0/2630922321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa27808d990 0x7fa278203c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.866+0000 7fa280535640 1 -- 172.21.15.76:0/2630922321 >> 172.21.15.76:0/2630922321 conn(0x7fa278070800 msgr2=0x7fa2780ce170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.867+0000 7fa280535640 1 -- 172.21.15.76:0/2630922321 shutdown_connections 2024-04-25T19:15:27.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.867+0000 7fa3988e0640 1 -- 172.21.15.76:0/964138709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fa378005580 con 0x7fa39408d990 2024-04-25T19:15:27.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.867+0000 7fa280535640 1 -- 172.21.15.76:0/2630922321 wait complete. 2024-04-25T19:15:27.870 INFO:teuthology.orchestra.run.smithi076.stdout:55834574852 2024-04-25T19:15:27.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.867+0000 7fa382ffd640 1 -- 172.21.15.76:0/964138709 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa384007e20 con 0x7fa39408d990 2024-04-25T19:15:27.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.870+0000 7fa3988e0640 1 -- 172.21.15.76:0/964138709 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa374074580 msgr2=0x7fa374076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.870+0000 7fa3988e0640 1 --2- 172.21.15.76:0/964138709 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa374074580 0x7fa374076a90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa39415e5c0 tx=0x7fa388009040 comp rx=0 tx=0).stop 2024-04-25T19:15:27.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.870+0000 7fa3988e0640 1 -- 172.21.15.76:0/964138709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa39408d990 msgr2=0x7fa394165260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.870+0000 7fa3988e0640 1 --2- 172.21.15.76:0/964138709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa39408d990 0x7fa394165260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa384034040 tx=0x7fa3840046c0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.870+0000 7fa3988e0640 1 -- 172.21.15.76:0/964138709 shutdown_connections 2024-04-25T19:15:27.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.870+0000 7fa3988e0640 1 --2- 172.21.15.76:0/964138709 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa374074580 0x7fa374076a90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.870+0000 7fa3988e0640 1 --2- 172.21.15.76:0/964138709 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa3940cdd60 0x7fa394202030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.870+0000 7fa3988e0640 1 --2- 172.21.15.76:0/964138709 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3940c8520 0x7fa3940c2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.870+0000 7fa3988e0640 1 --2- 172.21.15.76:0/964138709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa39408d990 0x7fa394165260 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.870+0000 7fa3988e0640 1 -- 172.21.15.76:0/964138709 >> 172.21.15.76:0/964138709 conn(0x7fa394070800 msgr2=0x7fa394070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.870+0000 7fa3988e0640 1 -- 172.21.15.76:0/964138709 shutdown_connections 2024-04-25T19:15:27.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.870+0000 7fa3988e0640 1 -- 172.21.15.76:0/964138709 wait complete. 2024-04-25T19:15:27.884 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.0 2024-04-25T19:15:27.886 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.5 2024-04-25T19:15:27.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.890+0000 7f88d5bff640 1 -- 172.21.15.76:0/138892728 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f88d00cd0d0 con 0x7f88d00c8520 2024-04-25T19:15:27.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.890+0000 7f88b7fff640 1 -- 172.21.15.76:0/138892728 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f88c406b240 con 0x7f88d00c8520 2024-04-25T19:15:27.892 INFO:teuthology.orchestra.run.smithi076.stdout:55834574852 2024-04-25T19:15:27.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.892+0000 7f88d5bff640 1 -- 172.21.15.76:0/138892728 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f88bc0744b0 msgr2=0x7f88bc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.892+0000 7f88d5bff640 1 --2- 172.21.15.76:0/138892728 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f88bc0744b0 0x7f88bc0769c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f88c00011e0 tx=0x7f88c0009040 comp rx=0 tx=0).stop 2024-04-25T19:15:27.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.892+0000 7f88d5bff640 1 -- 172.21.15.76:0/138892728 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88d00c8520 msgr2=0x7f88d01649e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.892+0000 7f88d5bff640 1 --2- 172.21.15.76:0/138892728 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88d00c8520 0x7f88d01649e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f88c400c410 tx=0x7f88c400c8d0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.892+0000 7f88d5bff640 1 -- 172.21.15.76:0/138892728 shutdown_connections 2024-04-25T19:15:27.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.892+0000 7f88d5bff640 1 --2- 172.21.15.76:0/138892728 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f88bc0744b0 0x7f88bc0769c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.892+0000 7f88d5bff640 1 --2- 172.21.15.76:0/138892728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88d0163a60 0x7f88d0164f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.892+0000 7f88d5bff640 1 --2- 172.21.15.76:0/138892728 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88d00c8520 0x7f88d01649e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.892+0000 7f88d5bff640 1 --2- 172.21.15.76:0/138892728 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88d008d990 0x7f88d02044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.892+0000 7f88d5bff640 1 -- 172.21.15.76:0/138892728 >> 172.21.15.76:0/138892728 conn(0x7f88d0070800 msgr2=0x7f88d0070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.893+0000 7f88d5bff640 1 -- 172.21.15.76:0/138892728 shutdown_connections 2024-04-25T19:15:27.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.893+0000 7f88d5bff640 1 -- 172.21.15.76:0/138892728 wait complete. 2024-04-25T19:15:27.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.893+0000 7f8e31492640 1 -- 172.21.15.76:0/4172268895 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f8e2c0507e0 con 0x7f8e180055f0 2024-04-25T19:15:27.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.894+0000 7f8e237fe640 1 -- 172.21.15.76:0/4172268895 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8e1c05f170 con 0x7f8e180055f0 2024-04-25T19:15:27.896 INFO:teuthology.orchestra.run.smithi076.stdout:55834574852 2024-04-25T19:15:27.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.894+0000 7f921cbc0640 1 -- 172.21.15.76:0/555275107 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f92180cd0d0 con 0x7f921808d990 2024-04-25T19:15:27.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.894+0000 7f920effd640 1 -- 172.21.15.76:0/555275107 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f920007ff50 con 0x7f921808d990 2024-04-25T19:15:27.897 INFO:teuthology.orchestra.run.smithi076.stdout:55834574852 2024-04-25T19:15:27.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.896+0000 7f8e31492640 1 -- 172.21.15.76:0/4172268895 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8e0c074530 msgr2=0x7f8e0c076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.896+0000 7f8e31492640 1 --2- 172.21.15.76:0/4172268895 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8e0c074530 0x7f8e0c076a40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8e14033000 tx=0x7f8e14007000 comp rx=0 tx=0).stop 2024-04-25T19:15:27.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.896+0000 7f8e31492640 1 -- 172.21.15.76:0/4172268895 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e180055f0 msgr2=0x7f8e2c164070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.896+0000 7f8e31492640 1 --2- 172.21.15.76:0/4172268895 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e180055f0 0x7f8e2c164070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00f7a0 tx=0x7f8e1c00fc60 comp rx=0 tx=0).stop 2024-04-25T19:15:27.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.896+0000 7f8e31492640 1 -- 172.21.15.76:0/4172268895 shutdown_connections 2024-04-25T19:15:27.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.896+0000 7f8e31492640 1 --2- 172.21.15.76:0/4172268895 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8e0c074530 0x7f8e0c076a40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.896+0000 7f8e31492640 1 --2- 172.21.15.76:0/4172268895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e2c15d470 0x7f8e2c15d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.896+0000 7f8e31492640 1 --2- 172.21.15.76:0/4172268895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e2c165050 0x7f8e2c165db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.896+0000 7f8e31492640 1 --2- 172.21.15.76:0/4172268895 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e180055f0 0x7f8e2c164070 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.896+0000 7f8e31492640 1 -- 172.21.15.76:0/4172268895 >> 172.21.15.76:0/4172268895 conn(0x7f8e2c070800 msgr2=0x7f8e2c15edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.896+0000 7f8e31492640 1 -- 172.21.15.76:0/4172268895 shutdown_connections 2024-04-25T19:15:27.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.896+0000 7f8e31492640 1 -- 172.21.15.76:0/4172268895 wait complete. 2024-04-25T19:15:27.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.897+0000 7f921cbc0640 1 -- 172.21.15.76:0/555275107 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f91f8074500 msgr2=0x7f91f8076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.897+0000 7f921cbc0640 1 --2- 172.21.15.76:0/555275107 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f91f8074500 0x7f91f8076a10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9208001a90 tx=0x7f9208001a20 comp rx=0 tx=0).stop 2024-04-25T19:15:27.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.897+0000 7f921cbc0640 1 -- 172.21.15.76:0/555275107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f921808d990 msgr2=0x7f9218162a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.897+0000 7f921cbc0640 1 --2- 172.21.15.76:0/555275107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f921808d990 0x7f9218162a40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9200034040 tx=0x7f92000046c0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.897+0000 7f921cbc0640 1 -- 172.21.15.76:0/555275107 shutdown_connections 2024-04-25T19:15:27.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.897+0000 7f921cbc0640 1 --2- 172.21.15.76:0/555275107 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f91f8074500 0x7f91f8076a10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.897+0000 7f921cbc0640 1 --2- 172.21.15.76:0/555275107 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92180cdd60 0x7f9218073a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.897+0000 7f921cbc0640 1 --2- 172.21.15.76:0/555275107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92180c8520 0x7f92180c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.897+0000 7f921cbc0640 1 --2- 172.21.15.76:0/555275107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f921808d990 0x7f9218162a40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.897+0000 7f921cbc0640 1 -- 172.21.15.76:0/555275107 >> 172.21.15.76:0/555275107 conn(0x7f9218070800 msgr2=0x7f9218070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.897+0000 7f921cbc0640 1 -- 172.21.15.76:0/555275107 shutdown_connections 2024-04-25T19:15:27.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.897+0000 7f921cbc0640 1 -- 172.21.15.76:0/555275107 wait complete. 2024-04-25T19:15:27.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.899+0000 7fdb27ba8640 1 -- 172.21.15.76:0/907215813 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fdb2008d710 con 0x7fdb20202de0 2024-04-25T19:15:27.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.899+0000 7fdb167fc640 1 -- 172.21.15.76:0/907215813 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdb1807f7b0 con 0x7fdb20202de0 2024-04-25T19:15:27.901 INFO:teuthology.orchestra.run.smithi076.stdout:55834574852 2024-04-25T19:15:27.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.903+0000 7fdb27ba8640 1 -- 172.21.15.76:0/907215813 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fdb040744b0 msgr2=0x7fdb040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.903+0000 7fdb27ba8640 1 --2- 172.21.15.76:0/907215813 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fdb040744b0 0x7fdb040769c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c001bb0 tx=0x7fdb1c001b20 comp rx=0 tx=0).stop 2024-04-25T19:15:27.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.903+0000 7fdb27ba8640 1 -- 172.21.15.76:0/907215813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb20202de0 msgr2=0x7fdb202031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:27.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.903+0000 7fdb27ba8640 1 --2- 172.21.15.76:0/907215813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb20202de0 0x7fdb202031f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdb18031000 tx=0x7fdb18004ab0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.903+0000 7fdb27ba8640 1 -- 172.21.15.76:0/907215813 shutdown_connections 2024-04-25T19:15:27.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.903+0000 7fdb27ba8640 1 --2- 172.21.15.76:0/907215813 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fdb040744b0 0x7fdb040769c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.903+0000 7fdb27ba8640 1 --2- 172.21.15.76:0/907215813 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb20202de0 0x7fdb202031f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.903+0000 7fdb27ba8640 1 --2- 172.21.15.76:0/907215813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdb200ca810 0x7fdb200cac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.903+0000 7fdb27ba8640 1 --2- 172.21.15.76:0/907215813 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdb20165810 0x7fdb2015d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:27.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.903+0000 7fdb27ba8640 1 -- 172.21.15.76:0/907215813 >> 172.21.15.76:0/907215813 conn(0x7fdb20070800 msgr2=0x7fdb20169460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:27.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.903+0000 7fdb27ba8640 1 -- 172.21.15.76:0/907215813 shutdown_connections 2024-04-25T19:15:27.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:27.903+0000 7fdb27ba8640 1 -- 172.21.15.76:0/907215813 wait complete. 2024-04-25T19:15:27.908 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.7 2024-04-25T19:15:27.913 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.4 2024-04-25T19:15:27.917 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2024-04-25T19:15:27.920 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.2 2024-04-25T19:15:28.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:28.000+0000 7fdd4d6bd640 1 -- 172.21.15.76:0/801402944 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fdd34005580 con 0x7fdd481f16c0 2024-04-25T19:15:28.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:28.000+0000 7fdd2f7fe640 1 -- 172.21.15.76:0/801402944 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdd38094070 con 0x7fdd481f16c0 2024-04-25T19:15:28.002 INFO:teuthology.orchestra.run.smithi076.stdout:55834574852 2024-04-25T19:15:28.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:28.002+0000 7fdd4d6bd640 1 -- 172.21.15.76:0/801402944 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fdd30074720 msgr2=0x7fdd30076c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:28.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:28.002+0000 7fdd4d6bd640 1 --2- 172.21.15.76:0/801402944 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fdd30074720 0x7fdd30076c30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c006fd0 tx=0x7fdd3c008040 comp rx=0 tx=0).stop 2024-04-25T19:15:28.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:28.002+0000 7fdd4d6bd640 1 -- 172.21.15.76:0/801402944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd481f16c0 msgr2=0x7fdd4814f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:28.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:28.002+0000 7fdd4d6bd640 1 --2- 172.21.15.76:0/801402944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd481f16c0 0x7fdd4814f730 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800b780 tx=0x7fdd3800bc40 comp rx=0 tx=0).stop 2024-04-25T19:15:28.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:28.002+0000 7fdd4d6bd640 1 -- 172.21.15.76:0/801402944 shutdown_connections 2024-04-25T19:15:28.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:28.002+0000 7fdd4d6bd640 1 --2- 172.21.15.76:0/801402944 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fdd30074720 0x7fdd30076c30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:28.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:28.002+0000 7fdd4d6bd640 1 --2- 172.21.15.76:0/801402944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd481f16c0 0x7fdd4814f730 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:28.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:28.002+0000 7fdd4d6bd640 1 --2- 172.21.15.76:0/801402944 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd480c9b90 0x7fdd48153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:28.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:28.002+0000 7fdd4d6bd640 1 --2- 172.21.15.76:0/801402944 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd480c9540 0x7fdd48202cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:28.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:28.002+0000 7fdd4d6bd640 1 -- 172.21.15.76:0/801402944 >> 172.21.15.76:0/801402944 conn(0x7fdd48070e70 msgr2=0x7fdd48072d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:28.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:28.002+0000 7fdd4d6bd640 1 -- 172.21.15.76:0/801402944 shutdown_connections 2024-04-25T19:15:28.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:28.002+0000 7fdd4d6bd640 1 -- 172.21.15.76:0/801402944 wait complete. 2024-04-25T19:15:28.015 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.1 2024-04-25T19:15:28.854 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T19:15:28.885 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T19:15:28.886 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T19:15:28.909 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T19:15:28.914 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T19:15:28.918 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T19:15:28.921 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T19:15:29.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:28.999+0000 7fa300185640 1 Processor -- start 2024-04-25T19:15:29.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:28.999+0000 7fa300185640 1 -- start start 2024-04-25T19:15:29.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.002+0000 7fa300185640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2f80c9e20 0x7fa2f80c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.002+0000 7fa300185640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2f80c3c10 con 0x7fa2f806fc50 2024-04-25T19:15:29.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.002+0000 7fa300185640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2f80c3950 con 0x7fa2f80c3480 2024-04-25T19:15:29.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.002+0000 7fa300185640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa2f80c8e60 con 0x7fa2f80c9e20 2024-04-25T19:15:29.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.002+0000 7fa2fd720640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2f80c9e20 0x7fa2f80c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.002+0000 7fa2fd720640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2f80c9e20 0x7fa2f80c5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:45952/0 (socket says 172.21.15.76:45952) 2024-04-25T19:15:29.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.002+0000 7fa2fd720640 1 -- 172.21.15.76:0/2803397271 learned_addr learned my addr 172.21.15.76:0/2803397271 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:29.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.003+0000 7fa2fcf1f640 1 -- 172.21.15.76:0/2803397271 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2475863684 0 0) 0x7fa2f80c3c10 con 0x7fa2f806fc50 2024-04-25T19:15:29.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.003+0000 7fa2fcf1f640 1 -- 172.21.15.76:0/2803397271 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2e4003520 con 0x7fa2f806fc50 2024-04-25T19:15:29.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.004+0000 7fa2fcf1f640 1 -- 172.21.15.76:0/2803397271 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3758847569 0 0) 0x7fa2e4003520 con 0x7fa2f806fc50 2024-04-25T19:15:29.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.004+0000 7fa2fd720640 1 -- 172.21.15.76:0/2803397271 >> v1:172.21.15.105:6789/0 conn(0x7fa2f80c3480 legacy=0x7fa2f8153210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T19:15:29.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.004+0000 7fa2fd720640 1 -- 172.21.15.76:0/2803397271 >> v1:172.21.15.76:6789/0 conn(0x7fa2f806fc50 legacy=0x7fa2f8074130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.004+0000 7fa2fd720640 1 -- 172.21.15.76:0/2803397271 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f80ca510 con 0x7fa2f80c9e20 2024-04-25T19:15:29.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.005+0000 7fa2fd720640 1 --2- 172.21.15.76:0/2803397271 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2f80c9e20 0x7fa2f80c5180 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa2f400bb30 tx=0x7fa2f402c6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=526477db3a615310 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.005+0000 7fa2fcf1f640 1 -- 172.21.15.76:0/2803397271 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f403a070 con 0x7fa2f80c9e20 2024-04-25T19:15:29.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.005+0000 7fa2fcf1f640 1 -- 172.21.15.76:0/2803397271 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2f4032480 con 0x7fa2f80c9e20 2024-04-25T19:15:29.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.005+0000 7fa2fcf1f640 1 -- 172.21.15.76:0/2803397271 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f00019f0 con 0x7fa2f80c9e20 2024-04-25T19:15:29.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.006+0000 7fa300185640 1 -- 172.21.15.76:0/2803397271 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2f80c9e20 msgr2=0x7fa2f80c5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.006+0000 7fa300185640 1 --2- 172.21.15.76:0/2803397271 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2f80c9e20 0x7fa2f80c5180 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa2f400bb30 tx=0x7fa2f402c6b0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.006+0000 7fa300185640 1 -- 172.21.15.76:0/2803397271 shutdown_connections 2024-04-25T19:15:29.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.006+0000 7fa300185640 1 --2- 172.21.15.76:0/2803397271 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2f80c9e20 0x7fa2f80c5180 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.006+0000 7fa300185640 1 -- 172.21.15.76:0/2803397271 >> 172.21.15.76:0/2803397271 conn(0x7fa2f806c5c0 msgr2=0x7fa2f806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:29.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.007+0000 7fa300185640 1 -- 172.21.15.76:0/2803397271 shutdown_connections 2024-04-25T19:15:29.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.007+0000 7fa300185640 1 -- 172.21.15.76:0/2803397271 wait complete. 2024-04-25T19:15:29.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.010+0000 7fa300185640 1 Processor -- start 2024-04-25T19:15:29.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.010+0000 7fa300185640 1 -- start start 2024-04-25T19:15:29.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.012+0000 7fa300185640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2f806fc50 0x7fa2f814dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.012+0000 7fa300185640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2f80c3480 0x7fa2f8159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.013+0000 7fa2fdf21640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2f806fc50 0x7fa2f814dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.013+0000 7fa2fdf21640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2f806fc50 0x7fa2f814dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:45968/0 (socket says 172.21.15.76:45968) 2024-04-25T19:15:29.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.013+0000 7fa2fd720640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2f80c3480 0x7fa2f8159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.013+0000 7fa2fd720640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2f80c3480 0x7fa2f8159670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59280/0 (socket says 172.21.15.76:59280) 2024-04-25T19:15:29.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.013+0000 7fa2fdf21640 1 -- 172.21.15.76:0/3672441069 learned_addr learned my addr 172.21.15.76:0/3672441069 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:29.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.013+0000 7fa300185640 1 --2- 172.21.15.76:0/3672441069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2f80c9e20 0x7fa2f814ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.013+0000 7fa300185640 1 -- 172.21.15.76:0/3672441069 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f80c56c0 con 0x7fa2f80c3480 2024-04-25T19:15:29.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.013+0000 7fa300185640 1 -- 172.21.15.76:0/3672441069 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f80caad0 con 0x7fa2f80c9e20 2024-04-25T19:15:29.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.013+0000 7fa300185640 1 -- 172.21.15.76:0/3672441069 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa2f8074840 con 0x7fa2f806fc50 2024-04-25T19:15:29.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.013+0000 7fa2fe722640 1 --2- 172.21.15.76:0/3672441069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2f80c9e20 0x7fa2f814ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.014+0000 7fa2fdf21640 1 -- 172.21.15.76:0/3672441069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2f80c9e20 msgr2=0x7fa2f814ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.014+0000 7fa2fdf21640 1 --2- 172.21.15.76:0/3672441069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2f80c9e20 0x7fa2f814ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.014+0000 7fa2fdf21640 1 -- 172.21.15.76:0/3672441069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2f80c3480 msgr2=0x7fa2f8159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.014+0000 7fa2fdf21640 1 --2- 172.21.15.76:0/3672441069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2f80c3480 0x7fa2f8159670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.014+0000 7fa2fdf21640 1 -- 172.21.15.76:0/3672441069 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f80c7a70 con 0x7fa2f806fc50 2024-04-25T19:15:29.017 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T19:15:29.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.014+0000 7fa2fd720640 1 --2- 172.21.15.76:0/3672441069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2f80c3480 0x7fa2f8159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:29.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.014+0000 7fa2fdf21640 1 --2- 172.21.15.76:0/3672441069 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2f806fc50 0x7fa2f814dd50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800b410 tx=0x7fa2e800b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.015+0000 7fa2ee7fc640 1 -- 172.21.15.76:0/3672441069 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8015020 con 0x7fa2f806fc50 2024-04-25T19:15:29.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.015+0000 7fa2ee7fc640 1 -- 172.21.15.76:0/3672441069 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2e8011830 con 0x7fa2f806fc50 2024-04-25T19:15:29.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.015+0000 7fa2ee7fc640 1 -- 172.21.15.76:0/3672441069 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8011c20 con 0x7fa2f806fc50 2024-04-25T19:15:29.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.015+0000 7fa300185640 1 -- 172.21.15.76:0/3672441069 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2f81ee790 con 0x7fa2f806fc50 2024-04-25T19:15:29.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.015+0000 7fa300185640 1 -- 172.21.15.76:0/3672441069 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2f81eed50 con 0x7fa2f806fc50 2024-04-25T19:15:29.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.017+0000 7fa300185640 1 -- 172.21.15.76:0/3672441069 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2f806d6a0 con 0x7fa2f806fc50 2024-04-25T19:15:29.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.018+0000 7fa2ee7fc640 1 -- 172.21.15.76:0/3672441069 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa2e8018020 con 0x7fa2f806fc50 2024-04-25T19:15:29.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.018+0000 7fa2ee7fc640 1 --2- 172.21.15.76:0/3672441069 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa2dc0744b0 0x7fa2dc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.018+0000 7fa2ee7fc640 1 -- 172.21.15.76:0/3672441069 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa2e8095180 con 0x7fa2f806fc50 2024-04-25T19:15:29.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.019+0000 7fa2fd720640 1 --2- 172.21.15.76:0/3672441069 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa2dc0744b0 0x7fa2dc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.020+0000 7fa2fd720640 1 --2- 172.21.15.76:0/3672441069 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa2dc0744b0 0x7fa2dc0769c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4037000 tx=0x7fa2f4030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.023+0000 7fa2ee7fc640 1 -- 172.21.15.76:0/3672441069 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa2e801f020 con 0x7fa2f806fc50 2024-04-25T19:15:29.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.042+0000 7fbbad6e7640 1 Processor -- start 2024-04-25T19:15:29.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.042+0000 7fbbad6e7640 1 -- start start 2024-04-25T19:15:29.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.045+0000 7fbbad6e7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbba80c9e20 0x7fbba80c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.045+0000 7fbbad6e7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbba806fc50 0x7fbba8070060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.045+0000 7fbbad6e7640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbba80c3c10 con 0x7fbba80c3480 2024-04-25T19:15:29.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.045+0000 7fbbad6e7640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fbba80c8e60 con 0x7fbba806fc50 2024-04-25T19:15:29.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.045+0000 7fbbad6e7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fbba8153920 con 0x7fbba80c9e20 2024-04-25T19:15:29.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.045+0000 7fbba67fc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbba80c9e20 0x7fbba80c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.045+0000 7fbba77fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbba806fc50 0x7fbba8070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.045+0000 7fbba77fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbba806fc50 0x7fbba8070060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:42722/0 (socket says 172.21.15.76:42722) 2024-04-25T19:15:29.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.045+0000 7fbba67fc640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbba80c9e20 0x7fbba80c5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:45978/0 (socket says 172.21.15.76:45978) 2024-04-25T19:15:29.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.045+0000 7fbba77fe640 1 -- 172.21.15.76:0/2615699152 learned_addr learned my addr 172.21.15.76:0/2615699152 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:29.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.046+0000 7fbba5ffb640 1 -- 172.21.15.76:0/2615699152 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3318929980 0 0) 0x7fbba80c3c10 con 0x7fbba80c3480 2024-04-25T19:15:29.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.046+0000 7fbba5ffb640 1 -- 172.21.15.76:0/2615699152 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb90003440 con 0x7fbba80c3480 2024-04-25T19:15:29.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.047+0000 7fbba77fe640 1 -- 172.21.15.76:0/2615699152 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbba80c9e20 msgr2=0x7fbba80c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:29.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.047+0000 7fbba77fe640 1 --2- 172.21.15.76:0/2615699152 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbba80c9e20 0x7fbba80c5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.047+0000 7fbba77fe640 1 -- 172.21.15.76:0/2615699152 >> v1:172.21.15.105:6789/0 conn(0x7fbba80c3480 legacy=0x7fbba8153210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.047+0000 7fbba77fe640 1 -- 172.21.15.76:0/2615699152 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba80c3950 con 0x7fbba806fc50 2024-04-25T19:15:29.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.048+0000 7fbba77fe640 1 --2- 172.21.15.76:0/2615699152 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbba806fc50 0x7fbba8070060 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009e50 tx=0x7fbba002a750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=306260bad3f05bfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.048+0000 7fbba5ffb640 1 -- 172.21.15.76:0/2615699152 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0036070 con 0x7fbba806fc50 2024-04-25T19:15:29.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.048+0000 7fbba5ffb640 1 -- 172.21.15.76:0/2615699152 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbba0031600 con 0x7fbba806fc50 2024-04-25T19:15:29.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.050+0000 7fbbad6e7640 1 -- 172.21.15.76:0/2615699152 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbba806fc50 msgr2=0x7fbba8070060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.050+0000 7fbbad6e7640 1 --2- 172.21.15.76:0/2615699152 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbba806fc50 0x7fbba8070060 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009e50 tx=0x7fbba002a750 comp rx=0 tx=0).stop 2024-04-25T19:15:29.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.050+0000 7f1907146640 1 Processor -- start 2024-04-25T19:15:29.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.050+0000 7fbbad6e7640 1 -- 172.21.15.76:0/2615699152 shutdown_connections 2024-04-25T19:15:29.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.050+0000 7fbbad6e7640 1 --2- 172.21.15.76:0/2615699152 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbba806fc50 0x7fbba8070060 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.050+0000 7fbbad6e7640 1 --2- 172.21.15.76:0/2615699152 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbba80c9e20 0x7fbba80c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.050+0000 7fbbad6e7640 1 -- 172.21.15.76:0/2615699152 >> 172.21.15.76:0/2615699152 conn(0x7fbba806c5c0 msgr2=0x7fbba806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:29.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.050+0000 7fbbad6e7640 1 -- 172.21.15.76:0/2615699152 shutdown_connections 2024-04-25T19:15:29.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.051+0000 7fbbad6e7640 1 -- 172.21.15.76:0/2615699152 wait complete. 2024-04-25T19:15:29.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.052+0000 7f1907146640 1 -- start start 2024-04-25T19:15:29.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.054+0000 7f1907146640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1900153020 0x7f19000c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.054+0000 7f1907146640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f19000c5020 0x7f19000c7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.054+0000 7f1907146640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19000c85c0 con 0x7f19001590f0 2024-04-25T19:15:29.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.054+0000 7f1907146640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f19000c4d60 con 0x7f1900153020 2024-04-25T19:15:29.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.054+0000 7f1907146640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f19000c8880 con 0x7f19000c5020 2024-04-25T19:15:29.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.054+0000 7f19056e3640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f19001590f0 0x7f1900159500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.76:59650/0 (socket says 172.21.15.76:59650) 2024-04-25T19:15:29.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.054+0000 7f19056e3640 1 -- 172.21.15.76:0/3558525734 learned_addr learned my addr 172.21.15.76:0/3558525734 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:29.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.055+0000 7f1904ee2640 1 --2- 172.21.15.76:0/3558525734 >> v2:172.21.15.76:3300/0 conn(0x7f1900153020 0x7f19000c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.055+0000 7f1904ee2640 1 -- 172.21.15.76:0/3558525734 >> v2:172.21.15.105:3300/0 conn(0x7f19000c5020 msgr2=0x7f19000c7010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:15:29.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.055+0000 7f1904ee2640 1 --2- 172.21.15.76:0/3558525734 >> v2:172.21.15.105:3300/0 conn(0x7f19000c5020 0x7f19000c7010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.055+0000 7f1904ee2640 1 -- 172.21.15.76:0/3558525734 >> v1:172.21.15.105:6789/0 conn(0x7f19001590f0 legacy=0x7f1900159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.055+0000 7f1904ee2640 1 -- 172.21.15.76:0/3558525734 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19000c7940 con 0x7f1900153020 2024-04-25T19:15:29.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.057+0000 7f1904ee2640 1 --2- 172.21.15.76:0/3558525734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1900153020 0x7f19000c6ad0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f190006e830 tx=0x7f18f402a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131aafb4ada07cec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.055+0000 7fbbad6e7640 1 Processor -- start 2024-04-25T19:15:29.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.055+0000 7fbbad6e7640 1 -- start start 2024-04-25T19:15:29.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.057+0000 7fbbad6e7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbba806fc50 0x7fbba8154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.057+0000 7fbbad6e7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbba80c3480 0x7fbba81563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.058+0000 7fbbad6e7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbba80c9e20 0x7fbba814f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.058+0000 7fbbad6e7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbba80ca7f0 con 0x7fbba80c3480 2024-04-25T19:15:29.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.058+0000 7fbbad6e7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbba80c56c0 con 0x7fbba806fc50 2024-04-25T19:15:29.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.058+0000 7fbbad6e7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fbba80caad0 con 0x7fbba80c9e20 2024-04-25T19:15:29.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.059+0000 7fbba67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbba80c3480 0x7fbba81563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.059+0000 7fbba6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbba806fc50 0x7fbba8154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.059+0000 7f18ff7fe640 1 -- 172.21.15.76:0/3558525734 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4037070 con 0x7f1900153020 2024-04-25T19:15:29.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.059+0000 7f18ff7fe640 1 -- 172.21.15.76:0/3558525734 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18f402ae40 con 0x7f1900153020 2024-04-25T19:15:29.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.060+0000 7f1907146640 1 -- 172.21.15.76:0/3558525734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1900153020 msgr2=0x7f19000c6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.059+0000 7fbba67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbba80c3480 0x7fbba81563e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59288/0 (socket says 172.21.15.76:59288) 2024-04-25T19:15:29.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.059+0000 7fbba67fc640 1 -- 172.21.15.76:0/430735071 learned_addr learned my addr 172.21.15.76:0/430735071 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:29.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.060+0000 7fbba67fc640 1 -- 172.21.15.76:0/430735071 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbba80c9e20 msgr2=0x7fbba814f5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T19:15:29.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.060+0000 7fbba67fc640 1 --2- 172.21.15.76:0/430735071 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbba80c9e20 0x7fbba814f5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.060+0000 7fbba67fc640 1 -- 172.21.15.76:0/430735071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbba806fc50 msgr2=0x7fbba8154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.060+0000 7fbba67fc640 1 --2- 172.21.15.76:0/430735071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbba806fc50 0x7fbba8154480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.060+0000 7fbba67fc640 1 -- 172.21.15.76:0/430735071 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba81549c0 con 0x7fbba80c3480 2024-04-25T19:15:29.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.061+0000 7fbba67fc640 1 --2- 172.21.15.76:0/430735071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbba80c3480 0x7fbba81563e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbb9800d5f0 tx=0x7fbb9800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.061+0000 7f18ff7fe640 1 -- 172.21.15.76:0/3558525734 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4037070 con 0x7f1900153020 2024-04-25T19:15:29.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.061+0000 7f1907146640 1 --2- 172.21.15.76:0/3558525734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1900153020 0x7f19000c6ad0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f190006e830 tx=0x7f18f402a3c0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.062+0000 7f1907146640 1 -- 172.21.15.76:0/3558525734 shutdown_connections 2024-04-25T19:15:29.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.062+0000 7fbb8f7fe640 1 -- 172.21.15.76:0/430735071 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb980125f0 con 0x7fbba80c3480 2024-04-25T19:15:29.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.062+0000 7f1907146640 1 --2- 172.21.15.76:0/3558525734 >> v2:172.21.15.105:3300/0 conn(0x7f19000c5020 0x7f19000c7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.062+0000 7f1907146640 1 --2- 172.21.15.76:0/3558525734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1900153020 0x7f19000c6ad0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.062+0000 7f1907146640 1 -- 172.21.15.76:0/3558525734 >> 172.21.15.76:0/3558525734 conn(0x7f1900070800 msgr2=0x7f1900070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:29.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.062+0000 7fbbad6e7640 1 -- 172.21.15.76:0/430735071 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba82111a0 con 0x7fbba80c3480 2024-04-25T19:15:29.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.062+0000 7fbbad6e7640 1 -- 172.21.15.76:0/430735071 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba8159830 con 0x7fbba80c3480 2024-04-25T19:15:29.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.065+0000 7fbbad6e7640 1 -- 172.21.15.76:0/430735071 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbba806d6a0 con 0x7fbba80c3480 2024-04-25T19:15:29.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.065+0000 7fbb8f7fe640 1 -- 172.21.15.76:0/430735071 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb98019070 con 0x7fbba80c3480 2024-04-25T19:15:29.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.065+0000 7fbb8f7fe640 1 -- 172.21.15.76:0/430735071 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98012990 con 0x7fbba80c3480 2024-04-25T19:15:29.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.068+0000 7fbb8f7fe640 1 -- 172.21.15.76:0/430735071 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbb98012b30 con 0x7fbba80c3480 2024-04-25T19:15:29.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.069+0000 7fbb8f7fe640 1 --2- 172.21.15.76:0/430735071 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbb94074580 0x7fbb94076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.069+0000 7fbb8f7fe640 1 -- 172.21.15.76:0/430735071 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbb98096280 con 0x7fbba80c3480 2024-04-25T19:15:29.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.066+0000 7f1907146640 1 -- 172.21.15.76:0/3558525734 shutdown_connections 2024-04-25T19:15:29.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.066+0000 7f1907146640 1 -- 172.21.15.76:0/3558525734 wait complete. 2024-04-25T19:15:29.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.069+0000 7f1907146640 1 Processor -- start 2024-04-25T19:15:29.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.070+0000 7fbba6ffd640 1 --2- 172.21.15.76:0/430735071 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbb94074580 0x7fbb94076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.070+0000 7f1907146640 1 -- start start 2024-04-25T19:15:29.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.072+0000 7f1907146640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19000c5020 0x7f19001f19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.073+0000 7f1907146640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1900153020 0x7f190014fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.073+0000 7f1907146640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19001590f0 0x7f1900203370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.073+0000 7f1907146640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f19000c8b60 con 0x7f19000c5020 2024-04-25T19:15:29.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.073+0000 7f1907146640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f19000c7f00 con 0x7f1900153020 2024-04-25T19:15:29.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.073+0000 7f1907146640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f1900158b60 con 0x7f19001590f0 2024-04-25T19:15:29.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.073+0000 7f19056e3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19001590f0 0x7f1900203370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.073+0000 7f19056e3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19001590f0 0x7f1900203370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:45994/0 (socket says 172.21.15.76:45994) 2024-04-25T19:15:29.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.074+0000 7f19056e3640 1 -- 172.21.15.76:0/1002291964 learned_addr learned my addr 172.21.15.76:0/1002291964 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:29.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.074+0000 7f1904ee2640 1 --2- 172.21.15.76:0/1002291964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19000c5020 0x7f19001f19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.074+0000 7f18fffff640 1 --2- 172.21.15.76:0/1002291964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1900153020 0x7f190014fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.075+0000 7fbba6ffd640 1 --2- 172.21.15.76:0/430735071 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbb94074580 0x7fbb94076a90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0010e0 tx=0x7fbb9c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.075+0000 7f19056e3640 1 -- 172.21.15.76:0/1002291964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1900153020 msgr2=0x7f190014fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.075+0000 7f19056e3640 1 --2- 172.21.15.76:0/1002291964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1900153020 0x7f190014fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.075+0000 7f19056e3640 1 -- 172.21.15.76:0/1002291964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19000c5020 msgr2=0x7f19001f19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.075+0000 7f19056e3640 1 --2- 172.21.15.76:0/1002291964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19000c5020 0x7f19001f19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.075+0000 7f19056e3640 1 -- 172.21.15.76:0/1002291964 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19001f1f30 con 0x7f19001590f0 2024-04-25T19:15:29.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.075+0000 7f19056e3640 1 --2- 172.21.15.76:0/1002291964 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19001590f0 0x7f1900203370 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f18f000c4a0 tx=0x7f18f000c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.076+0000 7f18fd7fa640 1 -- 172.21.15.76:0/1002291964 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f0018070 con 0x7f19001590f0 2024-04-25T19:15:29.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.076+0000 7f1907146640 1 -- 172.21.15.76:0/1002291964 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19001515d0 con 0x7f19001590f0 2024-04-25T19:15:29.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.076+0000 7f1907146640 1 -- 172.21.15.76:0/1002291964 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1900151fd0 con 0x7f19001590f0 2024-04-25T19:15:29.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.076+0000 7f18fd7fa640 1 -- 172.21.15.76:0/1002291964 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18f00074e0 con 0x7f19001590f0 2024-04-25T19:15:29.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.076+0000 7f18fd7fa640 1 -- 172.21.15.76:0/1002291964 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f0007690 con 0x7f19001590f0 2024-04-25T19:15:29.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.079+0000 7f1907146640 1 -- 172.21.15.76:0/1002291964 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f190006a390 con 0x7f19001590f0 2024-04-25T19:15:29.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.081+0000 7fbb8f7fe640 1 -- 172.21.15.76:0/430735071 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbb980609f0 con 0x7fbba80c3480 2024-04-25T19:15:29.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.081+0000 7f18fd7fa640 1 -- 172.21.15.76:0/1002291964 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f18f001b070 con 0x7f19001590f0 2024-04-25T19:15:29.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.081+0000 7f18fd7fa640 1 --2- 172.21.15.76:0/1002291964 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f18ec074500 0x7f18ec076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.082+0000 7f18fd7fa640 1 -- 172.21.15.76:0/1002291964 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f18f005b0d0 con 0x7f19001590f0 2024-04-25T19:15:29.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.082+0000 7f1904ee2640 1 --2- 172.21.15.76:0/1002291964 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f18ec074500 0x7f18ec076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.083+0000 7f1904ee2640 1 --2- 172.21.15.76:0/1002291964 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f18ec074500 0x7f18ec076a10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f18f4034000 tx=0x7f18f402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.085+0000 7f18fd7fa640 1 -- 172.21.15.76:0/1002291964 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f18f005eae0 con 0x7f19001590f0 2024-04-25T19:15:29.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.122+0000 7fd2cd2bd640 1 Processor -- start 2024-04-25T19:15:29.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.122+0000 7fd2cd2bd640 1 -- start start 2024-04-25T19:15:29.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.125+0000 7fd2cd2bd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2c0006a90 0x7fd2c0045e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.125+0000 7fd2cd2bd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2c0034fe0 con 0x7fd2c0050ed0 2024-04-25T19:15:29.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.125+0000 7fd2cd2bd640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2c0035840 con 0x7fd2c00334e0 2024-04-25T19:15:29.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.125+0000 7fd2cd2bd640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fd2c00d5c20 con 0x7fd2c0006a90 2024-04-25T19:15:29.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.125+0000 7fd2c7fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd2c0050ed0 0x7fd2c0046d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35318/0 (socket says 172.21.15.76:35318) 2024-04-25T19:15:29.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.125+0000 7fd2c7fff640 1 -- 172.21.15.76:0/593831716 learned_addr learned my addr 172.21.15.76:0/593831716 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:29.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.126+0000 7fd2ccabc640 1 --2- 172.21.15.76:0/593831716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2c0006a90 0x7fd2c0045e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.127+0000 7fd2c6ffd640 1 -- 172.21.15.76:0/593831716 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2031860543 0 0) 0x7fd2c0034fe0 con 0x7fd2c0050ed0 2024-04-25T19:15:29.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.127+0000 7fd2c6ffd640 1 -- 172.21.15.76:0/593831716 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2b8003520 con 0x7fd2c0050ed0 2024-04-25T19:15:29.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.128+0000 7fd2c6ffd640 1 -- 172.21.15.76:0/593831716 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3281066235 0 0) 0x7fd2c0035840 con 0x7fd2c00334e0 2024-04-25T19:15:29.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.128+0000 7fd2c6ffd640 1 -- 172.21.15.76:0/593831716 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2c0034fe0 con 0x7fd2c00334e0 2024-04-25T19:15:29.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.128+0000 7fd2c6ffd640 1 -- 172.21.15.76:0/593831716 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 344532282 0 0) 0x7fd2b8003520 con 0x7fd2c0050ed0 2024-04-25T19:15:29.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.128+0000 7fd2c6ffd640 1 -- 172.21.15.76:0/593831716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2c0006a90 msgr2=0x7fd2c0045e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:29.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.128+0000 7fd2c6ffd640 1 --2- 172.21.15.76:0/593831716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2c0006a90 0x7fd2c0045e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.128+0000 7fd2c6ffd640 1 -- 172.21.15.76:0/593831716 >> v1:172.21.15.105:6789/0 conn(0x7fd2c00334e0 legacy=0x7fd2c00d5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.128+0000 7fd2c6ffd640 1 -- 172.21.15.76:0/593831716 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c0035280 con 0x7fd2c0050ed0 2024-04-25T19:15:29.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.128+0000 7fd2c6ffd640 1 -- 172.21.15.76:0/593831716 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7fd2b40015f0 con 0x7fd2c0050ed0 2024-04-25T19:15:29.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.128+0000 7fd2c6ffd640 1 -- 172.21.15.76:0/593831716 >> v1:172.21.15.76:6789/0 conn(0x7fd2c0050ed0 legacy=0x7fd2c0046d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.127+0000 7f0a64734640 1 Processor -- start 2024-04-25T19:15:29.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.128+0000 7f0a64734640 1 -- start start 2024-04-25T19:15:29.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.129+0000 7fd2c6ffd640 1 --2- 172.21.15.76:0/593831716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2b8003b70 0x7fd2b8003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.129+0000 7f880e99d640 1 Processor -- start 2024-04-25T19:15:29.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.129+0000 7fd2c6ffd640 1 --2- 172.21.15.76:0/593831716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b8004800 0x7fd2b8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.129+0000 7fd2c7fff640 1 --2- 172.21.15.76:0/593831716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2b8003b70 0x7fd2b8003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.129+0000 7fd2c77fe640 1 --2- 172.21.15.76:0/593831716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b8004800 0x7fd2b8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.130+0000 7f0a64734640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0a5c0c9e20 0x7f0a5c0c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.130+0000 7f0a64734640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a5c06fc50 0x7f0a5c070060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.129+0000 7fd2c6ffd640 1 --2- 172.21.15.76:0/593831716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2b8005270 0x7fd2b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.129+0000 7fd2c6ffd640 1 -- 172.21.15.76:0/593831716 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c00d5c20 con 0x7fd2b8004800 2024-04-25T19:15:29.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.129+0000 7fd2c6ffd640 1 -- 172.21.15.76:0/593831716 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b800c1e0 con 0x7fd2b8005270 2024-04-25T19:15:29.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.129+0000 7fd2c6ffd640 1 -- 172.21.15.76:0/593831716 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fd2b800c3e0 con 0x7fd2b8003b70 2024-04-25T19:15:29.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.129+0000 7fd2ccabc640 1 --2- 172.21.15.76:0/593831716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2c0006a90 0x7fd2c0045e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:29.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.130+0000 7fd2ccabc640 1 --2- 172.21.15.76:0/593831716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2b8005270 0x7fd2b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.130+0000 7fd2c77fe640 1 -- 172.21.15.76:0/593831716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2b8003b70 msgr2=0x7fd2b8003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:29.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.130+0000 7fd2c77fe640 1 --2- 172.21.15.76:0/593831716 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2b8003b70 0x7fd2b8003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.130+0000 7fd2c77fe640 1 -- 172.21.15.76:0/593831716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2b8005270 msgr2=0x7fd2b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:29.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.130+0000 7fd2c77fe640 1 --2- 172.21.15.76:0/593831716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2b8005270 0x7fd2b800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.130+0000 7fd2c77fe640 1 -- 172.21.15.76:0/593831716 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b8003520 con 0x7fd2b8004800 2024-04-25T19:15:29.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.130+0000 7f0a64734640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a5c0c3c10 con 0x7f0a5c0c3480 2024-04-25T19:15:29.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.130+0000 7f0a64734640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f0a5c0c8e60 con 0x7f0a5c0c9e20 2024-04-25T19:15:29.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.130+0000 7f0a64734640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f0a5c153920 con 0x7f0a5c06fc50 2024-04-25T19:15:29.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.130+0000 7f0a624d0640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0a5c0c3480 0x7f0a5c153210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35326/0 (socket says 172.21.15.76:35326) 2024-04-25T19:15:29.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.130+0000 7f0a624d0640 1 -- 172.21.15.76:0/3331340872 learned_addr learned my addr 172.21.15.76:0/3331340872 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:29.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.130+0000 7f0a62cd1640 1 --2- 172.21.15.76:0/3331340872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a5c06fc50 0x7f0a5c070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.131+0000 7f14423d5640 1 Processor -- start 2024-04-25T19:15:29.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.131+0000 7f0a61ccf640 1 --2- 172.21.15.76:0/3331340872 >> v2:172.21.15.105:3300/0 conn(0x7f0a5c0c9e20 0x7f0a5c0c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.129+0000 7f880e99d640 1 -- start start 2024-04-25T19:15:29.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.131+0000 7f0a614ce640 1 -- 172.21.15.76:0/3331340872 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3685463683 0 0) 0x7f0a5c0c3c10 con 0x7f0a5c0c3480 2024-04-25T19:15:29.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.131+0000 7f880e99d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88080cf560 0x7f88080c9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.132+0000 7f0a614ce640 1 -- 172.21.15.76:0/3331340872 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a48003520 con 0x7f0a5c0c3480 2024-04-25T19:15:29.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.131+0000 7f880e99d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f880806d6d0 0x7f880806dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.131+0000 7f880e99d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88080ca9c0 con 0x7f8808153770 2024-04-25T19:15:29.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.132+0000 7f0a61ccf640 1 -- 172.21.15.76:0/3331340872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a5c06fc50 msgr2=0x7f0a5c070060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:29.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.132+0000 7f0a61ccf640 1 --2- 172.21.15.76:0/3331340872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a5c06fc50 0x7f0a5c070060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.132+0000 7f0a61ccf640 1 -- 172.21.15.76:0/3331340872 >> v1:172.21.15.76:6789/0 conn(0x7f0a5c0c3480 legacy=0x7f0a5c153210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.131+0000 7f880e99d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f88080ca790 con 0x7f880806d6d0 2024-04-25T19:15:29.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.131+0000 7f880e99d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f88080cac80 con 0x7f88080cf560 2024-04-25T19:15:29.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.132+0000 7f0a61ccf640 1 -- 172.21.15.76:0/3331340872 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5c0c3950 con 0x7f0a5c0c9e20 2024-04-25T19:15:29.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.132+0000 7f880d99b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8808153770 0x7f8808159d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35336/0 (socket says 172.21.15.76:35336) 2024-04-25T19:15:29.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.132+0000 7f880d99b640 1 -- 172.21.15.76:0/1096290984 learned_addr learned my addr 172.21.15.76:0/1096290984 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:29.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.131+0000 7fd2c77fe640 1 --2- 172.21.15.76:0/593831716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b8004800 0x7fd2b8004c10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc001bb0 tx=0x7fd2b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3239968cfcb06293 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.132+0000 7fd2c6ffd640 1 -- 172.21.15.76:0/593831716 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc036070 con 0x7fd2b8004800 2024-04-25T19:15:29.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.132+0000 7fd2c6ffd640 1 -- 172.21.15.76:0/593831716 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2bc031600 con 0x7fd2b8004800 2024-04-25T19:15:29.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.133+0000 7f880d19a640 1 --2- 172.21.15.76:0/1096290984 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88080cf560 0x7f88080c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.133+0000 7f880c999640 1 -- 172.21.15.76:0/1096290984 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1242619704 0 0) 0x7f88080ca9c0 con 0x7f8808153770 2024-04-25T19:15:29.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.133+0000 7f880c999640 1 -- 172.21.15.76:0/1096290984 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87f4005730 con 0x7f8808153770 2024-04-25T19:15:29.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.133+0000 7f0a61ccf640 1 --2- 172.21.15.76:0/3331340872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a5c0c9e20 0x7f0a5c0c5180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0097a0 tx=0x7f0a4c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d45341f7fad663f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.131+0000 7f14423d5640 1 -- start start 2024-04-25T19:15:29.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.133+0000 7f0a614ce640 1 -- 172.21.15.76:0/3331340872 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c037070 con 0x7f0a5c0c9e20 2024-04-25T19:15:29.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.133+0000 7f0a614ce640 1 -- 172.21.15.76:0/3331340872 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a4c032440 con 0x7f0a5c0c9e20 2024-04-25T19:15:29.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.134+0000 7f0a614ce640 1 -- 172.21.15.76:0/3331340872 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c0327f0 con 0x7f0a5c0c9e20 2024-04-25T19:15:29.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.133+0000 7f14423d5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f143c0c5020 0x7f143c1590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.133+0000 7f14423d5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f143c159630 0x7f143c163400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.134+0000 7f880e19c640 1 --2- 172.21.15.76:0/1096290984 >> v2:172.21.15.76:3300/0 conn(0x7f880806d6d0 0x7f880806dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.133+0000 7f14423d5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f143c0c85c0 con 0x7f143c153020 2024-04-25T19:15:29.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.133+0000 7f14423d5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f143c0c4d60 con 0x7f143c0c5020 2024-04-25T19:15:29.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.133+0000 7f14423d5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f143c0c8880 con 0x7f143c159630 2024-04-25T19:15:29.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.134+0000 7f880c999640 1 -- 172.21.15.76:0/1096290984 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3483068852 0 0) 0x7f87f4005730 con 0x7f8808153770 2024-04-25T19:15:29.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.134+0000 7f880c999640 1 -- 172.21.15.76:0/1096290984 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88080cf560 msgr2=0x7f88080c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:29.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.134+0000 7f880c999640 1 --2- 172.21.15.76:0/1096290984 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88080cf560 0x7f88080c9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.134+0000 7f880c999640 1 -- 172.21.15.76:0/1096290984 >> v2:172.21.15.76:3300/0 conn(0x7f880806d6d0 msgr2=0x7f880806dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:29.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.134+0000 7f880c999640 1 --2- 172.21.15.76:0/1096290984 >> v2:172.21.15.76:3300/0 conn(0x7f880806d6d0 0x7f880806dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.134+0000 7f880c999640 1 -- 172.21.15.76:0/1096290984 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88080caf60 con 0x7f8808153770 2024-04-25T19:15:29.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.134+0000 7f880c999640 1 -- 172.21.15.76:0/1096290984 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f8800001a10 con 0x7f8808153770 2024-04-25T19:15:29.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.134+0000 7f880c999640 1 -- 172.21.15.76:0/1096290984 >> v1:172.21.15.76:6789/0 conn(0x7f8808153770 legacy=0x7f8808159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.134+0000 7f880c999640 1 --2- 172.21.15.76:0/1096290984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f4005bc0 0x7f87f4006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.134+0000 7f880c999640 1 --2- 172.21.15.76:0/1096290984 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88080cf560 0x7f87f4006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.134+0000 7f880c999640 1 --2- 172.21.15.76:0/1096290984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87f4006f50 0x7f87f4007360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.134+0000 7f880c999640 1 -- 172.21.15.76:0/1096290984 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f88080cac80 con 0x7f87f4006f50 2024-04-25T19:15:29.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.134+0000 7f880c999640 1 -- 172.21.15.76:0/1096290984 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f87f4003520 con 0x7f87f4005bc0 2024-04-25T19:15:29.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.134+0000 7f880c999640 1 -- 172.21.15.76:0/1096290984 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f87f40036d0 con 0x7f88080cf560 2024-04-25T19:15:29.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.134+0000 7f143b7fe640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f143c0c5020 0x7f143c1590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.134+0000 7f1440972640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f143c159630 0x7f143c163400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.134+0000 7f1440972640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f143c159630 0x7f143c163400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:46016/0 (socket says 172.21.15.76:46016) 2024-04-25T19:15:29.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.134+0000 7f1440972640 1 -- 172.21.15.76:0/3166307117 learned_addr learned my addr 172.21.15.76:0/3166307117 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:29.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.134+0000 7fd2cd2bd640 1 -- 172.21.15.76:0/593831716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b8004800 msgr2=0x7fd2b8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.134+0000 7fd2cd2bd640 1 --2- 172.21.15.76:0/593831716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2b8004800 0x7fd2b8004c10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc001bb0 tx=0x7fd2b8005830 comp rx=0 tx=0).stop 2024-04-25T19:15:29.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.134+0000 7fd2ccabc640 1 -- 172.21.15.76:0/593831716 reap_dead start 2024-04-25T19:15:29.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.134+0000 7f880d19a640 1 --2- 172.21.15.76:0/1096290984 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88080cf560 0x7f87f4006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.135+0000 7fd2cd2bd640 1 -- 172.21.15.76:0/593831716 shutdown_connections 2024-04-25T19:15:29.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.135+0000 7fd2cd2bd640 1 -- 172.21.15.76:0/593831716 >> 172.21.15.76:0/593831716 conn(0x7fd2c00503b0 msgr2=0x7fd2c00507c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:29.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.135+0000 7f880d99b640 1 --2- 172.21.15.76:0/1096290984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f4005bc0 0x7f87f4006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.135+0000 7fd2cd2bd640 1 -- 172.21.15.76:0/593831716 shutdown_connections 2024-04-25T19:15:29.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.135+0000 7f880d19a640 1 -- 172.21.15.76:0/1096290984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f4005bc0 msgr2=0x7f87f4006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:29.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.135+0000 7f880d19a640 1 --2- 172.21.15.76:0/1096290984 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f87f4005bc0 0x7f87f4006030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.135+0000 7f880d19a640 1 -- 172.21.15.76:0/1096290984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87f4006f50 msgr2=0x7f87f4007360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:29.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.135+0000 7f880d19a640 1 --2- 172.21.15.76:0/1096290984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87f4006f50 0x7f87f4007360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.135+0000 7f880d19a640 1 -- 172.21.15.76:0/1096290984 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f4005730 con 0x7f88080cf560 2024-04-25T19:15:29.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.135+0000 7f143affd640 1 -- 172.21.15.76:0/3166307117 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1959153012 0 0) 0x7f143c0c85c0 con 0x7f143c153020 2024-04-25T19:15:29.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.136+0000 7f143affd640 1 -- 172.21.15.76:0/3166307117 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1424003520 con 0x7f143c153020 2024-04-25T19:15:29.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.135+0000 7f0a64734640 1 -- 172.21.15.76:0/3331340872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a5c0c9e20 msgr2=0x7f0a5c0c5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.135+0000 7f0a64734640 1 --2- 172.21.15.76:0/3331340872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a5c0c9e20 0x7f0a5c0c5180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0097a0 tx=0x7f0a4c02a1a0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.136+0000 7f0a64734640 1 -- 172.21.15.76:0/3331340872 shutdown_connections 2024-04-25T19:15:29.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.136+0000 7f0a64734640 1 --2- 172.21.15.76:0/3331340872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a5c0c9e20 0x7f0a5c0c5180 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.136+0000 7f0a64734640 1 --2- 172.21.15.76:0/3331340872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a5c06fc50 0x7f0a5c070060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.136+0000 7f0a64734640 1 -- 172.21.15.76:0/3331340872 >> 172.21.15.76:0/3331340872 conn(0x7f0a5c06c5c0 msgr2=0x7f0a5c06c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:29.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.135+0000 7fd2cd2bd640 1 -- 172.21.15.76:0/593831716 wait complete. 2024-04-25T19:15:29.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.136+0000 7f0a64734640 1 -- 172.21.15.76:0/3331340872 shutdown_connections 2024-04-25T19:15:29.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.136+0000 7f143affd640 1 -- 172.21.15.76:0/3166307117 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2044504715 0 0) 0x7f1424003520 con 0x7f143c153020 2024-04-25T19:15:29.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.136+0000 7f1440972640 1 -- 172.21.15.76:0/3166307117 >> v2:172.21.15.105:3300/0 conn(0x7f143c0c5020 msgr2=0x7f143c1590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:29.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.136+0000 7f1440972640 1 --2- 172.21.15.76:0/3166307117 >> v2:172.21.15.105:3300/0 conn(0x7f143c0c5020 0x7f143c1590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.136+0000 7f1440972640 1 -- 172.21.15.76:0/3166307117 >> v1:172.21.15.76:6789/0 conn(0x7f143c153020 legacy=0x7f143c0c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.136+0000 7f0a64734640 1 -- 172.21.15.76:0/3331340872 wait complete. 2024-04-25T19:15:29.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.136+0000 7f1440972640 1 -- 172.21.15.76:0/3166307117 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f143c0c7940 con 0x7f143c159630 2024-04-25T19:15:29.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.137+0000 7f880d19a640 1 --2- 172.21.15.76:0/1096290984 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88080cf560 0x7f87f4006a10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8804009ac0 tx=0x7f87f4007a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac02e1ca32626005 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.137+0000 7f880c999640 1 -- 172.21.15.76:0/1096290984 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804036070 con 0x7f88080cf560 2024-04-25T19:15:29.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.137+0000 7f880c999640 1 -- 172.21.15.76:0/1096290984 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8804031400 con 0x7f88080cf560 2024-04-25T19:15:29.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.137+0000 7f880c999640 1 -- 172.21.15.76:0/1096290984 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804009690 con 0x7f88080cf560 2024-04-25T19:15:29.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.138+0000 7f1440972640 1 --2- 172.21.15.76:0/3166307117 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f143c159630 0x7f143c163400 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1434009d40 tx=0x7f143402b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee22c6c621c88da3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.138+0000 7f143affd640 1 -- 172.21.15.76:0/3166307117 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1434037070 con 0x7f143c159630 2024-04-25T19:15:29.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.138+0000 7fd2cd2bd640 1 Processor -- start 2024-04-25T19:15:29.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.139+0000 7f143affd640 1 -- 172.21.15.76:0/3166307117 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1434032400 con 0x7f143c159630 2024-04-25T19:15:29.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.139+0000 7fd2cd2bd640 1 -- start start 2024-04-25T19:15:29.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.140+0000 7f0a64734640 1 Processor -- start 2024-04-25T19:15:29.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.139+0000 7f880e99d640 1 -- 172.21.15.76:0/1096290984 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88080cf560 msgr2=0x7f87f4006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.139+0000 7f880e99d640 1 --2- 172.21.15.76:0/1096290984 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88080cf560 0x7f87f4006a10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8804009ac0 tx=0x7f87f4007a40 comp rx=0 tx=0).stop 2024-04-25T19:15:29.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.140+0000 7f14423d5640 1 -- 172.21.15.76:0/3166307117 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f143c159630 msgr2=0x7f143c163400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.140+0000 7f14423d5640 1 --2- 172.21.15.76:0/3166307117 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f143c159630 0x7f143c163400 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1434009d40 tx=0x7f143402b860 comp rx=0 tx=0).stop 2024-04-25T19:15:29.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.139+0000 7f880e19c640 1 -- 172.21.15.76:0/1096290984 reap_dead start 2024-04-25T19:15:29.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.139+0000 7f880e99d640 1 -- 172.21.15.76:0/1096290984 shutdown_connections 2024-04-25T19:15:29.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.139+0000 7f880e99d640 1 -- 172.21.15.76:0/1096290984 >> 172.21.15.76:0/1096290984 conn(0x7f88080719f0 msgr2=0x7f88080cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:29.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.140+0000 7f14423d5640 1 -- 172.21.15.76:0/3166307117 shutdown_connections 2024-04-25T19:15:29.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.140+0000 7f14423d5640 1 --2- 172.21.15.76:0/3166307117 >> v2:172.21.15.105:3300/0 conn(0x7f143c0c5020 0x7f143c1590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.140+0000 7f14423d5640 1 --2- 172.21.15.76:0/3166307117 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f143c159630 0x7f143c163400 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.140+0000 7f14423d5640 1 -- 172.21.15.76:0/3166307117 >> 172.21.15.76:0/3166307117 conn(0x7f143c070800 msgr2=0x7f143c070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:29.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.139+0000 7f880e99d640 1 -- 172.21.15.76:0/1096290984 shutdown_connections 2024-04-25T19:15:29.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.139+0000 7f880e99d640 1 -- 172.21.15.76:0/1096290984 wait complete. 2024-04-25T19:15:29.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.140+0000 7f14423d5640 1 -- 172.21.15.76:0/3166307117 shutdown_connections 2024-04-25T19:15:29.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.140+0000 7f14423d5640 1 -- 172.21.15.76:0/3166307117 wait complete. 2024-04-25T19:15:29.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.141+0000 7fd2cd2bd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2c0178d10 0x7fd2c004f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.142+0000 7fd2cd2bd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2c00db3f0 0x7fd2c00db800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.142+0000 7f880e99d640 1 Processor -- start 2024-04-25T19:15:29.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.141+0000 7f0a64734640 1 -- start start 2024-04-25T19:15:29.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.142+0000 7f880e99d640 1 -- start start 2024-04-25T19:15:29.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.142+0000 7fd2c7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2c0178d10 0x7fd2c004f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.142+0000 7fd2c7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2c0178d10 0x7fd2c004f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:42786/0 (socket says 172.21.15.76:42786) 2024-04-25T19:15:29.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.142+0000 7fd2c7fff640 1 -- 172.21.15.76:0/1211562043 learned_addr learned my addr 172.21.15.76:0/1211562043 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:29.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.142+0000 7fd2c77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2c00db3f0 0x7fd2c00db800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.143+0000 7fd2cd2bd640 1 --2- 172.21.15.76:0/1211562043 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2c00d3fd0 0x7fd2c00d43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.143+0000 7fd2cd2bd640 1 -- 172.21.15.76:0/1211562043 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c0045350 con 0x7fd2c00db3f0 2024-04-25T19:15:29.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.143+0000 7fd2cd2bd640 1 -- 172.21.15.76:0/1211562043 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c0006570 con 0x7fd2c0178d10 2024-04-25T19:15:29.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.143+0000 7fd2cd2bd640 1 -- 172.21.15.76:0/1211562043 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fd2c0045e70 con 0x7fd2c00d3fd0 2024-04-25T19:15:29.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.143+0000 7f0a64734640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a5c06fc50 0x7f0a5c15b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.143+0000 7f14423d5640 1 Processor -- start 2024-04-25T19:15:29.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.144+0000 7fd2ccabc640 1 --2- 172.21.15.76:0/1211562043 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2c00d3fd0 0x7fd2c00d43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.144+0000 7f0a624d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a5c06fc50 0x7f0a5c15b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.144+0000 7f0a624d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a5c06fc50 0x7f0a5c15b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59330/0 (socket says 172.21.15.76:59330) 2024-04-25T19:15:29.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.144+0000 7f0a624d0640 1 -- 172.21.15.76:0/3569605265 learned_addr learned my addr 172.21.15.76:0/3569605265 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:29.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.144+0000 7f0a64734640 1 --2- 172.21.15.76:0/3569605265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a5c0c3480 0x7f0a5c0c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.144+0000 7f0a64734640 1 --2- 172.21.15.76:0/3569605265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a5c0c9e20 0x7f0a5c150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.144+0000 7f0a64734640 1 -- 172.21.15.76:0/3569605265 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c0ca7f0 con 0x7f0a5c06fc50 2024-04-25T19:15:29.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.144+0000 7f0a64734640 1 -- 172.21.15.76:0/3569605265 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5c0c56c0 con 0x7f0a5c0c3480 2024-04-25T19:15:29.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.144+0000 7f0a64734640 1 -- 172.21.15.76:0/3569605265 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f0a5c0caad0 con 0x7f0a5c0c9e20 2024-04-25T19:15:29.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.144+0000 7fd2c77fe640 1 -- 172.21.15.76:0/1211562043 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2c00d3fd0 msgr2=0x7fd2c00d43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.144+0000 7fd2c77fe640 1 --2- 172.21.15.76:0/1211562043 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2c00d3fd0 0x7fd2c00d43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.144+0000 7fd2c77fe640 1 -- 172.21.15.76:0/1211562043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2c0178d10 msgr2=0x7fd2c004f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.144+0000 7fd2c77fe640 1 --2- 172.21.15.76:0/1211562043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2c0178d10 0x7fd2c004f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.144+0000 7fd2c77fe640 1 -- 172.21.15.76:0/1211562043 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c0050000 con 0x7fd2c00db3f0 2024-04-25T19:15:29.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.144+0000 7fd2ccabc640 1 --2- 172.21.15.76:0/1211562043 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2c00d3fd0 0x7fd2c00d43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:29.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.144+0000 7f0a62cd1640 1 --2- 172.21.15.76:0/3569605265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a5c0c9e20 0x7f0a5c150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.144+0000 7f0a61ccf640 1 --2- 172.21.15.76:0/3569605265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a5c0c3480 0x7f0a5c0c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.144+0000 7f880e99d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88080cf560 0x7f88080cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.145+0000 7f14423d5640 1 -- start start 2024-04-25T19:15:29.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.145+0000 7f880e99d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f880815a6c0 0x7f880814b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.145+0000 7fd2c77fe640 1 --2- 172.21.15.76:0/1211562043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2c00db3f0 0x7fd2c00db800 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800c1b0 tx=0x7fd2bc0012b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.145+0000 7f0a62cd1640 1 -- 172.21.15.76:0/3569605265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a5c0c3480 msgr2=0x7f0a5c0c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.145+0000 7f0a62cd1640 1 --2- 172.21.15.76:0/3569605265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a5c0c3480 0x7f0a5c0c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.145+0000 7f0a62cd1640 1 -- 172.21.15.76:0/3569605265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a5c06fc50 msgr2=0x7f0a5c15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.145+0000 7f880d19a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f880815a6c0 0x7f880814b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.145+0000 7f0a62cd1640 1 --2- 172.21.15.76:0/3569605265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a5c06fc50 0x7f0a5c15b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.145+0000 7f880d19a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f880815a6c0 0x7f880814b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59342/0 (socket says 172.21.15.76:59342) 2024-04-25T19:15:29.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.145+0000 7f0a62cd1640 1 -- 172.21.15.76:0/3569605265 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5c15b7b0 con 0x7f0a5c0c9e20 2024-04-25T19:15:29.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.145+0000 7f880d19a640 1 -- 172.21.15.76:0/1302138543 learned_addr learned my addr 172.21.15.76:0/1302138543 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:29.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.145+0000 7f880e99d640 1 --2- 172.21.15.76:0/1302138543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f880815cb40 0x7f880814b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.145+0000 7f0a624d0640 1 --2- 172.21.15.76:0/3569605265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a5c06fc50 0x7f0a5c15b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:29.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.145+0000 7fd2c4ff9640 1 -- 172.21.15.76:0/1211562043 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc036070 con 0x7fd2c00db3f0 2024-04-25T19:15:29.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.145+0000 7f880d99b640 1 --2- 172.21.15.76:0/1302138543 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88080cf560 0x7f88080cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.145+0000 7f880e99d640 1 -- 172.21.15.76:0/1302138543 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8808159500 con 0x7f880815a6c0 2024-04-25T19:15:29.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.145+0000 7f880e99d640 1 -- 172.21.15.76:0/1302138543 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8808159850 con 0x7f880815cb40 2024-04-25T19:15:29.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.145+0000 7f880e99d640 1 -- 172.21.15.76:0/1302138543 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f88080c8e60 con 0x7f88080cf560 2024-04-25T19:15:29.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.145+0000 7fd2cd2bd640 1 -- 172.21.15.76:0/1211562043 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c00d08e0 con 0x7fd2c00db3f0 2024-04-25T19:15:29.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.145+0000 7fd2c4ff9640 1 -- 172.21.15.76:0/1211562043 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2bc037040 con 0x7fd2c00db3f0 2024-04-25T19:15:29.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.145+0000 7f880e19c640 1 --2- 172.21.15.76:0/1302138543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f880815cb40 0x7f880814b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.146+0000 7f0a62cd1640 1 --2- 172.21.15.76:0/3569605265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a5c0c9e20 0x7f0a5c150270 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009db0 tx=0x7f0a540075c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.146+0000 7f880d99b640 1 -- 172.21.15.76:0/1302138543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f880815cb40 msgr2=0x7f880814b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.146+0000 7f880d99b640 1 --2- 172.21.15.76:0/1302138543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f880815cb40 0x7f880814b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.146+0000 7f880d99b640 1 -- 172.21.15.76:0/1302138543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f880815a6c0 msgr2=0x7f880814b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.146+0000 7f880d99b640 1 --2- 172.21.15.76:0/1302138543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f880815a6c0 0x7f880814b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.146+0000 7f880d99b640 1 -- 172.21.15.76:0/1302138543 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880814ca70 con 0x7f88080cf560 2024-04-25T19:15:29.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.146+0000 7f0a52ffd640 1 -- 172.21.15.76:0/3569605265 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a54019070 con 0x7f0a5c0c9e20 2024-04-25T19:15:29.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.146+0000 7f0a52ffd640 1 -- 172.21.15.76:0/3569605265 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a54007e50 con 0x7f0a5c0c9e20 2024-04-25T19:15:29.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.146+0000 7f0a52ffd640 1 -- 172.21.15.76:0/3569605265 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a54003bd0 con 0x7f0a5c0c9e20 2024-04-25T19:15:29.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.146+0000 7f880d99b640 1 --2- 172.21.15.76:0/1302138543 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88080cf560 0x7f88080cdbb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8800002230 tx=0x7f880000a6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.146+0000 7fd2c4ff9640 1 -- 172.21.15.76:0/1211562043 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0069f0 con 0x7fd2c00db3f0 2024-04-25T19:15:29.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.147+0000 7fd2cd2bd640 1 -- 172.21.15.76:0/1211562043 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c00da910 con 0x7fd2c00db3f0 2024-04-25T19:15:29.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.147+0000 7f14423d5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f143c0c5020 0x7f143c152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.147+0000 7f14423d5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f143c153020 0x7f143c1441a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.148+0000 7f143b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f143c153020 0x7f143c1441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.148+0000 7f143b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f143c153020 0x7f143c1441a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59354/0 (socket says 172.21.15.76:59354) 2024-04-25T19:15:29.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.148+0000 7f143b7fe640 1 -- 172.21.15.76:0/4241563541 learned_addr learned my addr 172.21.15.76:0/4241563541 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:29.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.148+0000 7f14423d5640 1 --2- 172.21.15.76:0/4241563541 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f143c159630 0x7f143c1446e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.148+0000 7f14423d5640 1 -- 172.21.15.76:0/4241563541 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f143c0c8b60 con 0x7f143c153020 2024-04-25T19:15:29.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.148+0000 7f14423d5640 1 -- 172.21.15.76:0/4241563541 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f143c0c7f00 con 0x7f143c0c5020 2024-04-25T19:15:29.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.148+0000 7f14423d5640 1 -- 172.21.15.76:0/4241563541 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f143c159a40 con 0x7f143c159630 2024-04-25T19:15:29.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.147+0000 7f0a64734640 1 -- 172.21.15.76:0/3569605265 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a5c156720 con 0x7f0a5c0c9e20 2024-04-25T19:15:29.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.147+0000 7f0a64734640 1 -- 172.21.15.76:0/3569605265 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a5c15a790 con 0x7f0a5c0c9e20 2024-04-25T19:15:29.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.148+0000 7f880d19a640 1 --2- 172.21.15.76:0/1302138543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f880815a6c0 0x7f880814b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:15:29.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.148+0000 7fd2b27fc640 1 -- 172.21.15.76:0/1211562043 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2a0005290 con 0x7fd2c00db3f0 2024-04-25T19:15:29.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.148+0000 7f1440972640 1 --2- 172.21.15.76:0/4241563541 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f143c159630 0x7f143c1446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.148+0000 7f143bfff640 1 --2- 172.21.15.76:0/4241563541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f143c0c5020 0x7f143c152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.148+0000 7f143b7fe640 1 -- 172.21.15.76:0/4241563541 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f143c159630 msgr2=0x7f143c1446e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.149+0000 7f143b7fe640 1 --2- 172.21.15.76:0/4241563541 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f143c159630 0x7f143c1446e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.149+0000 7f143b7fe640 1 -- 172.21.15.76:0/4241563541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f143c0c5020 msgr2=0x7f143c152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.149+0000 7f143b7fe640 1 --2- 172.21.15.76:0/4241563541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f143c0c5020 0x7f143c152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.149+0000 7f143b7fe640 1 -- 172.21.15.76:0/4241563541 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f143c151b00 con 0x7f143c153020 2024-04-25T19:15:29.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.149+0000 7f143bfff640 1 --2- 172.21.15.76:0/4241563541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f143c0c5020 0x7f143c152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:29.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.149+0000 7f1440972640 1 --2- 172.21.15.76:0/4241563541 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f143c159630 0x7f143c1446e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:29.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.149+0000 7f143b7fe640 1 --2- 172.21.15.76:0/4241563541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f143c153020 0x7f143c1441a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f142c00ddc0 tx=0x7f142c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.150+0000 7f0a52ffd640 1 -- 172.21.15.76:0/3569605265 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0a54005d10 con 0x7f0a5c0c9e20 2024-04-25T19:15:29.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.150+0000 7f0a50ff9640 1 -- 172.21.15.76:0/3569605265 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a5c06d6a0 con 0x7f0a5c0c9e20 2024-04-25T19:15:29.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.150+0000 7f1438ff9640 1 -- 172.21.15.76:0/4241563541 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c007df0 con 0x7f143c153020 2024-04-25T19:15:29.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.150+0000 7f0a52ffd640 1 --2- 172.21.15.76:0/3569605265 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0a400745b0 0x7f0a40076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.148+0000 7f87fe7fc640 1 -- 172.21.15.76:0/1302138543 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800017020 con 0x7f88080cf560 2024-04-25T19:15:29.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.148+0000 7f87fe7fc640 1 -- 172.21.15.76:0/1302138543 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f880000ad60 con 0x7f88080cf560 2024-04-25T19:15:29.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.150+0000 7f87fe7fc640 1 -- 172.21.15.76:0/1302138543 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800013610 con 0x7f88080cf560 2024-04-25T19:15:29.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.150+0000 7f880e99d640 1 -- 172.21.15.76:0/1302138543 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880814cd50 con 0x7f88080cf560 2024-04-25T19:15:29.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.150+0000 7f880e99d640 1 -- 172.21.15.76:0/1302138543 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f880814d310 con 0x7f88080cf560 2024-04-25T19:15:29.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.151+0000 7f0a52ffd640 1 -- 172.21.15.76:0/3569605265 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0a540950c0 con 0x7f0a5c0c9e20 2024-04-25T19:15:29.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.151+0000 7f0a624d0640 1 --2- 172.21.15.76:0/3569605265 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0a400745b0 0x7f0a40076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.151+0000 7fd2c4ff9640 1 -- 172.21.15.76:0/1211562043 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd2bc031600 con 0x7fd2c00db3f0 2024-04-25T19:15:29.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.150+0000 7f1438ff9640 1 -- 172.21.15.76:0/4241563541 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f142c00a440 con 0x7f143c153020 2024-04-25T19:15:29.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.150+0000 7f14423d5640 1 -- 172.21.15.76:0/4241563541 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f143c151de0 con 0x7f143c153020 2024-04-25T19:15:29.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.150+0000 7f14423d5640 1 -- 172.21.15.76:0/4241563541 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f143c156f90 con 0x7f143c153020 2024-04-25T19:15:29.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.151+0000 7f1438ff9640 1 -- 172.21.15.76:0/4241563541 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c007df0 con 0x7f143c153020 2024-04-25T19:15:29.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.152+0000 7fd2c4ff9640 1 --2- 172.21.15.76:0/1211562043 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fd2ac074530 0x7fd2ac076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.152+0000 7fd2c4ff9640 1 -- 172.21.15.76:0/1211562043 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd2bc0b6d30 con 0x7fd2c00db3f0 2024-04-25T19:15:29.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.152+0000 7f880e99d640 1 -- 172.21.15.76:0/1302138543 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88080c9a30 con 0x7f88080cf560 2024-04-25T19:15:29.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.152+0000 7f14423d5640 1 -- 172.21.15.76:0/4241563541 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f143c06a390 con 0x7f143c153020 2024-04-25T19:15:29.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.152+0000 7f0a624d0640 1 --2- 172.21.15.76:0/3569605265 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0a400745b0 0x7f0a40076ac0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0a58005880 tx=0x7f0a58009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.152+0000 7fd2c7fff640 1 --2- 172.21.15.76:0/1211562043 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fd2ac074530 0x7fd2ac076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.153+0000 7fd2c7fff640 1 --2- 172.21.15.76:0/1211562043 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fd2ac074530 0x7fd2ac076a40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4008a50 tx=0x7fd2b4001650 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.156+0000 7f0a52ffd640 1 -- 172.21.15.76:0/3569605265 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0a54011070 con 0x7f0a5c0c9e20 2024-04-25T19:15:29.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.157+0000 7fd2c4ff9640 1 -- 172.21.15.76:0/1211562043 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd2c0050000 con 0x7fd2c00db3f0 2024-04-25T19:15:29.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.158+0000 7f87fe7fc640 1 -- 172.21.15.76:0/1302138543 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f880001a020 con 0x7f88080cf560 2024-04-25T19:15:29.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.158+0000 7f87fe7fc640 1 --2- 172.21.15.76:0/1302138543 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f87ec074580 0x7f87ec076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.158+0000 7f87fe7fc640 1 -- 172.21.15.76:0/1302138543 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8800096250 con 0x7f88080cf560 2024-04-25T19:15:29.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.161+0000 7f87fe7fc640 1 -- 172.21.15.76:0/1302138543 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8800021020 con 0x7f88080cf560 2024-04-25T19:15:29.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.165+0000 7f880d19a640 1 --2- 172.21.15.76:0/1302138543 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f87ec074580 0x7f87ec076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.165+0000 7f1438ff9640 1 -- 172.21.15.76:0/4241563541 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f142c00a5f0 con 0x7f143c153020 2024-04-25T19:15:29.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.165+0000 7f1438ff9640 1 --2- 172.21.15.76:0/4241563541 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f14280748b0 0x7f1428076dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.166+0000 7f1438ff9640 1 -- 172.21.15.76:0/4241563541 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f142c013070 con 0x7f143c153020 2024-04-25T19:15:29.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.167+0000 7f143bfff640 1 --2- 172.21.15.76:0/4241563541 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f14280748b0 0x7f1428076dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.169+0000 7f143bfff640 1 --2- 172.21.15.76:0/4241563541 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f14280748b0 0x7f1428076dc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1430001eb0 tx=0x7f1430014040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.170+0000 7f880d19a640 1 --2- 172.21.15.76:0/1302138543 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f87ec074580 0x7f87ec076a90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8804000c00 tx=0x7f8804008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.170+0000 7f1438ff9640 1 -- 172.21.15.76:0/4241563541 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f142c05fcb0 con 0x7f143c153020 2024-04-25T19:15:29.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.203+0000 7fe9434d1640 1 Processor -- start 2024-04-25T19:15:29.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.204+0000 7fe9434d1640 1 -- start start 2024-04-25T19:15:29.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.206+0000 7fe9434d1640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe93c0cb920 0x7fe93c15b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.206+0000 7fe9434d1640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe93c0c7340 con 0x7fe93c1548b0 2024-04-25T19:15:29.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.206+0000 7fe9434d1640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe93c0ca390 con 0x7fe93c0c8ba0 2024-04-25T19:15:29.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.206+0000 7fe9434d1640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fe93c0c7600 con 0x7fe93c0cb920 2024-04-25T19:15:29.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.206+0000 7fe94126d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe93c0cb920 0x7fe93c15b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.206+0000 7fe940a6c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe93c1548b0 0x7fe93c0cab70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35354/0 (socket says 172.21.15.76:35354) 2024-04-25T19:15:29.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.207+0000 7fe94126d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe93c0cb920 0x7fe93c15b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:42816/0 (socket says 172.21.15.76:42816) 2024-04-25T19:15:29.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.207+0000 7fe940a6c640 1 -- 172.21.15.76:0/2022272402 learned_addr learned my addr 172.21.15.76:0/2022272402 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:29.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.207+0000 7fe933fff640 1 -- 172.21.15.76:0/2022272402 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1025586047 0 0) 0x7fe93c0c7340 con 0x7fe93c1548b0 2024-04-25T19:15:29.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.207+0000 7fe933fff640 1 -- 172.21.15.76:0/2022272402 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe928003520 con 0x7fe93c1548b0 2024-04-25T19:15:29.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.208+0000 7fe933fff640 1 -- 172.21.15.76:0/2022272402 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3819556482 0 0) 0x7fe928003520 con 0x7fe93c1548b0 2024-04-25T19:15:29.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.208+0000 7fe933fff640 1 -- 172.21.15.76:0/2022272402 >> v2:172.21.15.105:3300/0 conn(0x7fe93c0cb920 msgr2=0x7fe93c15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:29.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.208+0000 7fe933fff640 1 --2- 172.21.15.76:0/2022272402 >> v2:172.21.15.105:3300/0 conn(0x7fe93c0cb920 0x7fe93c15b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.208+0000 7fe933fff640 1 -- 172.21.15.76:0/2022272402 >> v1:172.21.15.105:6789/0 conn(0x7fe93c0c8ba0 legacy=0x7fe93c165280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:29.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.208+0000 7fe933fff640 1 -- 172.21.15.76:0/2022272402 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93c0c78e0 con 0x7fe93c1548b0 2024-04-25T19:15:29.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.208+0000 7fe933fff640 1 -- 172.21.15.76:0/2022272402 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7fe938001650 con 0x7fe93c1548b0 2024-04-25T19:15:29.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.208+0000 7fe933fff640 1 -- 172.21.15.76:0/2022272402 >> v1:172.21.15.76:6789/0 conn(0x7fe93c1548b0 legacy=0x7fe93c0cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.208+0000 7fe933fff640 1 --2- 172.21.15.76:0/2022272402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9280039f0 0x7fe928003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.208+0000 7fe933fff640 1 --2- 172.21.15.76:0/2022272402 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe928004800 0x7fe928004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.209+0000 7fe933fff640 1 --2- 172.21.15.76:0/2022272402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe928005300 0x7fe92800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.209+0000 7fe933fff640 1 -- 172.21.15.76:0/2022272402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c0c7600 con 0x7fe9280039f0 2024-04-25T19:15:29.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.209+0000 7fe933fff640 1 -- 172.21.15.76:0/2022272402 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe92800c290 con 0x7fe928005300 2024-04-25T19:15:29.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.209+0000 7fe933fff640 1 -- 172.21.15.76:0/2022272402 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fe92800c490 con 0x7fe928004800 2024-04-25T19:15:29.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.209+0000 7fe940a6c640 1 --2- 172.21.15.76:0/2022272402 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe928004800 0x7fe928004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.209+0000 7fe94126d640 1 --2- 172.21.15.76:0/2022272402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9280039f0 0x7fe928003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.210+0000 7fe941a6e640 1 --2- 172.21.15.76:0/2022272402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe928005300 0x7fe92800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.210+0000 7fe94126d640 1 -- 172.21.15.76:0/2022272402 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe928004800 msgr2=0x7fe928004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:29.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.210+0000 7fe94126d640 1 --2- 172.21.15.76:0/2022272402 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe928004800 0x7fe928004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.210+0000 7fe94126d640 1 -- 172.21.15.76:0/2022272402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe928005300 msgr2=0x7fe92800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:29.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.210+0000 7fe94126d640 1 --2- 172.21.15.76:0/2022272402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe928005300 0x7fe92800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.210+0000 7fe94126d640 1 -- 172.21.15.76:0/2022272402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93c0c78e0 con 0x7fe9280039f0 2024-04-25T19:15:29.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.211+0000 7fe941a6e640 1 -- 172.21.15.76:0/2022272402 reap_dead start 2024-04-25T19:15:29.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.211+0000 7fe94126d640 1 --2- 172.21.15.76:0/2022272402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9280039f0 0x7fe928003e80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe92c00b5a0 tx=0x7fe928005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98bcf14256152cf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.212+0000 7fe933fff640 1 -- 172.21.15.76:0/2022272402 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c03c070 con 0x7fe9280039f0 2024-04-25T19:15:29.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.212+0000 7fe933fff640 1 -- 172.21.15.76:0/2022272402 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe92c037410 con 0x7fe9280039f0 2024-04-25T19:15:29.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.212+0000 7fe933fff640 1 -- 172.21.15.76:0/2022272402 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c00b5e0 con 0x7fe9280039f0 2024-04-25T19:15:29.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.213+0000 7fe9434d1640 1 -- 172.21.15.76:0/2022272402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9280039f0 msgr2=0x7fe928003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.213+0000 7fe9434d1640 1 --2- 172.21.15.76:0/2022272402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9280039f0 0x7fe928003e80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe92c00b5a0 tx=0x7fe928005830 comp rx=0 tx=0).stop 2024-04-25T19:15:29.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.213+0000 7fe9434d1640 1 -- 172.21.15.76:0/2022272402 shutdown_connections 2024-04-25T19:15:29.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.213+0000 7fe9434d1640 1 --2- 172.21.15.76:0/2022272402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9280039f0 0x7fe928003e80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.213+0000 7fe9434d1640 1 -- 172.21.15.76:0/2022272402 >> 172.21.15.76:0/2022272402 conn(0x7fe93c070800 msgr2=0x7fe93c070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:29.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.214+0000 7fe9434d1640 1 -- 172.21.15.76:0/2022272402 shutdown_connections 2024-04-25T19:15:29.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.214+0000 7fe9434d1640 1 -- 172.21.15.76:0/2022272402 wait complete. 2024-04-25T19:15:29.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.217+0000 7fe9434d1640 1 Processor -- start 2024-04-25T19:15:29.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.217+0000 7fe9434d1640 1 -- start start 2024-04-25T19:15:29.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.220+0000 7fe9434d1640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9280039f0 0x7fe93c1f2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.220+0000 7fe9434d1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe93c158b90 0x7fe93c146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.220+0000 7fe9434d1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe93c1432b0 0x7fe93c1436c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.220+0000 7fe94126d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9280039f0 0x7fe93c1f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.220+0000 7fe94126d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9280039f0 0x7fe93c1f2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:46104/0 (socket says 172.21.15.76:46104) 2024-04-25T19:15:29.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.220+0000 7fe94126d640 1 -- 172.21.15.76:0/1315371948 learned_addr learned my addr 172.21.15.76:0/1315371948 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:29.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.220+0000 7fe9434d1640 1 -- 172.21.15.76:0/1315371948 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c154cc0 con 0x7fe93c1432b0 2024-04-25T19:15:29.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.220+0000 7fe941a6e640 1 --2- 172.21.15.76:0/1315371948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe93c1432b0 0x7fe93c1436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.220+0000 7fe9434d1640 1 -- 172.21.15.76:0/1315371948 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c0ca650 con 0x7fe93c158b90 2024-04-25T19:15:29.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.221+0000 7fe9434d1640 1 -- 172.21.15.76:0/1315371948 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fe93c154e40 con 0x7fe9280039f0 2024-04-25T19:15:29.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.221+0000 7fe940a6c640 1 --2- 172.21.15.76:0/1315371948 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe93c158b90 0x7fe93c146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.221+0000 7fe94126d640 1 -- 172.21.15.76:0/1315371948 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe93c158b90 msgr2=0x7fe93c146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.221+0000 7fe94126d640 1 --2- 172.21.15.76:0/1315371948 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe93c158b90 0x7fe93c146ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.221+0000 7fe94126d640 1 -- 172.21.15.76:0/1315371948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe93c1432b0 msgr2=0x7fe93c1436c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.221+0000 7fe94126d640 1 --2- 172.21.15.76:0/1315371948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe93c1432b0 0x7fe93c1436c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.221+0000 7fe94126d640 1 -- 172.21.15.76:0/1315371948 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93c1541f0 con 0x7fe9280039f0 2024-04-25T19:15:29.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.222+0000 7fe940a6c640 1 --2- 172.21.15.76:0/1315371948 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe93c158b90 0x7fe93c146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:15:29.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.223+0000 7fe94126d640 1 --2- 172.21.15.76:0/1315371948 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9280039f0 0x7fe93c1f2670 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe92c00bad0 tx=0x7fe92c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.223+0000 7fe931ffb640 1 -- 172.21.15.76:0/1315371948 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c044070 con 0x7fe9280039f0 2024-04-25T19:15:29.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.223+0000 7fe9434d1640 1 -- 172.21.15.76:0/1315371948 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe93c153950 con 0x7fe9280039f0 2024-04-25T19:15:29.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.223+0000 7fe9434d1640 1 -- 172.21.15.76:0/1315371948 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe93c153f10 con 0x7fe9280039f0 2024-04-25T19:15:29.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.224+0000 7fe931ffb640 1 -- 172.21.15.76:0/1315371948 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe92c037410 con 0x7fe9280039f0 2024-04-25T19:15:29.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.224+0000 7fe931ffb640 1 -- 172.21.15.76:0/1315371948 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c03c040 con 0x7fe9280039f0 2024-04-25T19:15:29.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.228+0000 7fe931ffb640 1 -- 172.21.15.76:0/1315371948 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe92c047070 con 0x7fe9280039f0 2024-04-25T19:15:29.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.228+0000 7fe931ffb640 1 --2- 172.21.15.76:0/1315371948 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe9200744b0 0x7fe9200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.229+0000 7fe9434d1640 1 -- 172.21.15.76:0/1315371948 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe91c005290 con 0x7fe9280039f0 2024-04-25T19:15:29.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.229+0000 7fe931ffb640 1 -- 172.21.15.76:0/1315371948 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe92c0b9880 con 0x7fe9280039f0 2024-04-25T19:15:29.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.229+0000 7fe940a6c640 1 --2- 172.21.15.76:0/1315371948 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe9200744b0 0x7fe9200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.230+0000 7fe940a6c640 1 --2- 172.21.15.76:0/1315371948 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe9200744b0 0x7fe9200769c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe93c0ba600 tx=0x7fe938009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.234+0000 7fe931ffb640 1 -- 172.21.15.76:0/1315371948 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe92c0839c0 con 0x7fe9280039f0 2024-04-25T19:15:29.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.414+0000 7fa300185640 1 -- 172.21.15.76:0/3672441069 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fa2f806a390 con 0x7fa2f806fc50 2024-04-25T19:15:29.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.415+0000 7fa2ee7fc640 1 -- 172.21.15.76:0/3672441069 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa2e805f870 con 0x7fa2f806fc50 2024-04-25T19:15:29.419 INFO:teuthology.orchestra.run.smithi076.stdout:55834574853 2024-04-25T19:15:29.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.418+0000 7fa300185640 1 -- 172.21.15.76:0/3672441069 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa2dc0744b0 msgr2=0x7fa2dc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.418+0000 7fa300185640 1 --2- 172.21.15.76:0/3672441069 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa2dc0744b0 0x7fa2dc0769c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4037000 tx=0x7fa2f4030000 comp rx=0 tx=0).stop 2024-04-25T19:15:29.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.418+0000 7fa300185640 1 -- 172.21.15.76:0/3672441069 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2f806fc50 msgr2=0x7fa2f814dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.418+0000 7fa300185640 1 --2- 172.21.15.76:0/3672441069 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2f806fc50 0x7fa2f814dd50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800b410 tx=0x7fa2e800b8d0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.418+0000 7fa300185640 1 -- 172.21.15.76:0/3672441069 shutdown_connections 2024-04-25T19:15:29.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.418+0000 7fa300185640 1 --2- 172.21.15.76:0/3672441069 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa2dc0744b0 0x7fa2dc0769c0 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4037000 tx=0x7fa2f4030000 comp rx=0 tx=0).stop 2024-04-25T19:15:29.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.418+0000 7fa300185640 1 --2- 172.21.15.76:0/3672441069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2f80c9e20 0x7fa2f814ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.418+0000 7fa300185640 1 --2- 172.21.15.76:0/3672441069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2f80c3480 0x7fa2f8159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.418+0000 7fa300185640 1 --2- 172.21.15.76:0/3672441069 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa2f806fc50 0x7fa2f814dd50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.418+0000 7fa300185640 1 -- 172.21.15.76:0/3672441069 >> 172.21.15.76:0/3672441069 conn(0x7fa2f806c5c0 msgr2=0x7fa2f8150970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:29.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.418+0000 7fa300185640 1 -- 172.21.15.76:0/3672441069 shutdown_connections 2024-04-25T19:15:29.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.418+0000 7fa300185640 1 -- 172.21.15.76:0/3672441069 wait complete. 2024-04-25T19:15:29.436 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2024-04-25T19:15:29.436 DEBUG:teuthology.parallel:result is None 2024-04-25T19:15:29.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.487+0000 7fbbad6e7640 1 -- 172.21.15.76:0/430735071 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fbba81515d0 con 0x7fbba80c3480 2024-04-25T19:15:29.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.488+0000 7fbb8f7fe640 1 -- 172.21.15.76:0/430735071 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbba81515d0 con 0x7fbba80c3480 2024-04-25T19:15:29.493 INFO:teuthology.orchestra.run.smithi076.stdout:55834574853 2024-04-25T19:15:29.493 INFO:teuthology.orchestra.run.smithi076.stdout:55834574853 2024-04-25T19:15:29.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.489+0000 7f1907146640 1 -- 172.21.15.76:0/1002291964 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f19000cd5e0 con 0x7f19001590f0 2024-04-25T19:15:29.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.489+0000 7f18fd7fa640 1 -- 172.21.15.76:0/1002291964 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f18f0067120 con 0x7f19001590f0 2024-04-25T19:15:29.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.492+0000 7f18e6ffd640 1 -- 172.21.15.76:0/1002291964 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f18ec074500 msgr2=0x7f18ec076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.492+0000 7f18e6ffd640 1 --2- 172.21.15.76:0/1002291964 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f18ec074500 0x7f18ec076a10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f18f4034000 tx=0x7f18f402e000 comp rx=0 tx=0).stop 2024-04-25T19:15:29.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.493+0000 7f18e6ffd640 1 -- 172.21.15.76:0/1002291964 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19001590f0 msgr2=0x7f1900203370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.493+0000 7f18e6ffd640 1 --2- 172.21.15.76:0/1002291964 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19001590f0 0x7f1900203370 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f18f000c4a0 tx=0x7f18f000c960 comp rx=0 tx=0).stop 2024-04-25T19:15:29.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.493+0000 7fbbad6e7640 1 -- 172.21.15.76:0/430735071 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbb94074580 msgr2=0x7fbb94076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.493+0000 7fbbad6e7640 1 --2- 172.21.15.76:0/430735071 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbb94074580 0x7fbb94076a90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0010e0 tx=0x7fbb9c009040 comp rx=0 tx=0).stop 2024-04-25T19:15:29.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.493+0000 7fbbad6e7640 1 -- 172.21.15.76:0/430735071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbba80c3480 msgr2=0x7fbba81563e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.493+0000 7fbbad6e7640 1 --2- 172.21.15.76:0/430735071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbba80c3480 0x7fbba81563e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbb9800d5f0 tx=0x7fbb9800dab0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.493+0000 7f18e6ffd640 1 -- 172.21.15.76:0/1002291964 shutdown_connections 2024-04-25T19:15:29.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.493+0000 7f18e6ffd640 1 --2- 172.21.15.76:0/1002291964 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f18ec074500 0x7f18ec076a10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.493+0000 7f18e6ffd640 1 --2- 172.21.15.76:0/1002291964 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f19001590f0 0x7f1900203370 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.493+0000 7f18e6ffd640 1 --2- 172.21.15.76:0/1002291964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1900153020 0x7f190014fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.493+0000 7f18e6ffd640 1 --2- 172.21.15.76:0/1002291964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19000c5020 0x7f19001f19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.493+0000 7f18e6ffd640 1 -- 172.21.15.76:0/1002291964 >> 172.21.15.76:0/1002291964 conn(0x7f1900070800 msgr2=0x7f19000c71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:29.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.493+0000 7f18e6ffd640 1 -- 172.21.15.76:0/1002291964 shutdown_connections 2024-04-25T19:15:29.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.493+0000 7f18e6ffd640 1 -- 172.21.15.76:0/1002291964 wait complete. 2024-04-25T19:15:29.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.493+0000 7fbbad6e7640 1 -- 172.21.15.76:0/430735071 shutdown_connections 2024-04-25T19:15:29.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.493+0000 7fbbad6e7640 1 --2- 172.21.15.76:0/430735071 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbb94074580 0x7fbb94076a90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.493+0000 7fbbad6e7640 1 --2- 172.21.15.76:0/430735071 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbba80c9e20 0x7fbba814f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.493+0000 7fbbad6e7640 1 --2- 172.21.15.76:0/430735071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbba80c3480 0x7fbba81563e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.493+0000 7fbbad6e7640 1 --2- 172.21.15.76:0/430735071 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbba806fc50 0x7fbba8154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.493+0000 7fbbad6e7640 1 -- 172.21.15.76:0/430735071 >> 172.21.15.76:0/430735071 conn(0x7fbba806c5c0 msgr2=0x7fbba80c7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:29.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.493+0000 7fbbad6e7640 1 -- 172.21.15.76:0/430735071 shutdown_connections 2024-04-25T19:15:29.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.493+0000 7fbbad6e7640 1 -- 172.21.15.76:0/430735071 wait complete. 2024-04-25T19:15:29.514 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.5 2024-04-25T19:15:29.514 DEBUG:teuthology.parallel:result is None 2024-04-25T19:15:29.514 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.0 2024-04-25T19:15:29.514 DEBUG:teuthology.parallel:result is None 2024-04-25T19:15:29.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.529+0000 7fd2b27fc640 1 -- 172.21.15.76:0/1211562043 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fd2a0005850 con 0x7fd2c00db3f0 2024-04-25T19:15:29.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.529+0000 7fd2c4ff9640 1 -- 172.21.15.76:0/1211562043 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd2bc087370 con 0x7fd2c00db3f0 2024-04-25T19:15:29.531 INFO:teuthology.orchestra.run.smithi076.stdout:55834574853 2024-04-25T19:15:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.531+0000 7fd2b27fc640 1 -- 172.21.15.76:0/1211562043 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fd2ac074530 msgr2=0x7fd2ac076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.531+0000 7fd2b27fc640 1 --2- 172.21.15.76:0/1211562043 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fd2ac074530 0x7fd2ac076a40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4008a50 tx=0x7fd2b4001650 comp rx=0 tx=0).stop 2024-04-25T19:15:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.531+0000 7fd2b27fc640 1 -- 172.21.15.76:0/1211562043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2c00db3f0 msgr2=0x7fd2c00db800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.531+0000 7fd2b27fc640 1 --2- 172.21.15.76:0/1211562043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2c00db3f0 0x7fd2c00db800 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800c1b0 tx=0x7fd2bc0012b0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.531+0000 7fd2b27fc640 1 -- 172.21.15.76:0/1211562043 shutdown_connections 2024-04-25T19:15:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.531+0000 7fd2b27fc640 1 --2- 172.21.15.76:0/1211562043 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fd2ac074530 0x7fd2ac076a40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.531+0000 7fd2b27fc640 1 --2- 172.21.15.76:0/1211562043 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd2c00d3fd0 0x7fd2c00d43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.531+0000 7fd2b27fc640 1 --2- 172.21.15.76:0/1211562043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2c00db3f0 0x7fd2c00db800 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.531+0000 7fd2b27fc640 1 --2- 172.21.15.76:0/1211562043 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd2c0178d10 0x7fd2c004f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.531+0000 7fd2b27fc640 1 -- 172.21.15.76:0/1211562043 >> 172.21.15.76:0/1211562043 conn(0x7fd2c00503b0 msgr2=0x7fd2c00072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:29.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.531+0000 7fd2b27fc640 1 -- 172.21.15.76:0/1211562043 shutdown_connections 2024-04-25T19:15:29.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.531+0000 7fd2b27fc640 1 -- 172.21.15.76:0/1211562043 wait complete. 2024-04-25T19:15:29.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.544+0000 7f14423d5640 1 -- 172.21.15.76:0/4241563541 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f143c0cd5e0 con 0x7f143c153020 2024-04-25T19:15:29.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.544+0000 7f1438ff9640 1 -- 172.21.15.76:0/4241563541 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f142c063fb0 con 0x7f143c153020 2024-04-25T19:15:29.546 INFO:teuthology.orchestra.run.smithi076.stdout:55834574853 2024-04-25T19:15:29.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.547+0000 7f14423d5640 1 -- 172.21.15.76:0/4241563541 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f14280748b0 msgr2=0x7f1428076dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.547+0000 7f14423d5640 1 --2- 172.21.15.76:0/4241563541 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f14280748b0 0x7f1428076dc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1430001eb0 tx=0x7f1430014040 comp rx=0 tx=0).stop 2024-04-25T19:15:29.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.547+0000 7f14423d5640 1 -- 172.21.15.76:0/4241563541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f143c153020 msgr2=0x7f143c1441a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.547+0000 7f14423d5640 1 --2- 172.21.15.76:0/4241563541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f143c153020 0x7f143c1441a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f142c00ddc0 tx=0x7f142c00c7f0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.550 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2024-04-25T19:15:29.550 DEBUG:teuthology.parallel:result is None 2024-04-25T19:15:29.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.547+0000 7f14423d5640 1 -- 172.21.15.76:0/4241563541 shutdown_connections 2024-04-25T19:15:29.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.547+0000 7f14423d5640 1 --2- 172.21.15.76:0/4241563541 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f14280748b0 0x7f1428076dc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.547+0000 7f14423d5640 1 --2- 172.21.15.76:0/4241563541 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f143c159630 0x7f143c1446e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.547+0000 7f14423d5640 1 --2- 172.21.15.76:0/4241563541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f143c153020 0x7f143c1441a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.547+0000 7f14423d5640 1 --2- 172.21.15.76:0/4241563541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f143c0c5020 0x7f143c152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.547+0000 7f14423d5640 1 -- 172.21.15.76:0/4241563541 >> 172.21.15.76:0/4241563541 conn(0x7f143c070800 msgr2=0x7f143c1e47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:29.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.547+0000 7f14423d5640 1 -- 172.21.15.76:0/4241563541 shutdown_connections 2024-04-25T19:15:29.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.547+0000 7f14423d5640 1 -- 172.21.15.76:0/4241563541 wait complete. 2024-04-25T19:15:29.560 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2024-04-25T19:15:29.560 DEBUG:teuthology.parallel:result is None 2024-04-25T19:15:29.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.570+0000 7fe9434d1640 1 -- 172.21.15.76:0/1315371948 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe91c005580 con 0x7fe9280039f0 2024-04-25T19:15:29.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.571+0000 7fe931ffb640 1 -- 172.21.15.76:0/1315371948 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe92c087cc0 con 0x7fe9280039f0 2024-04-25T19:15:29.573 INFO:teuthology.orchestra.run.smithi076.stdout:55834574853 2024-04-25T19:15:29.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.573+0000 7f0a50ff9640 1 -- 172.21.15.76:0/3569605265 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f0a5c06a390 con 0x7f0a5c0c9e20 2024-04-25T19:15:29.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.573+0000 7fe9434d1640 1 -- 172.21.15.76:0/1315371948 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe9200744b0 msgr2=0x7fe9200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.573+0000 7fe9434d1640 1 --2- 172.21.15.76:0/1315371948 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe9200744b0 0x7fe9200769c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe93c0ba600 tx=0x7fe938009040 comp rx=0 tx=0).stop 2024-04-25T19:15:29.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.573+0000 7fe9434d1640 1 -- 172.21.15.76:0/1315371948 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9280039f0 msgr2=0x7fe93c1f2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.573+0000 7fe9434d1640 1 --2- 172.21.15.76:0/1315371948 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9280039f0 0x7fe93c1f2670 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe92c00bad0 tx=0x7fe92c004730 comp rx=0 tx=0).stop 2024-04-25T19:15:29.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.573+0000 7fe9434d1640 1 -- 172.21.15.76:0/1315371948 shutdown_connections 2024-04-25T19:15:29.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.573+0000 7fe9434d1640 1 --2- 172.21.15.76:0/1315371948 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe9200744b0 0x7fe9200769c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.573+0000 7fe9434d1640 1 --2- 172.21.15.76:0/1315371948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe93c1432b0 0x7fe93c1436c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.573+0000 7fe9434d1640 1 --2- 172.21.15.76:0/1315371948 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe93c158b90 0x7fe93c146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.573+0000 7fe9434d1640 1 --2- 172.21.15.76:0/1315371948 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe9280039f0 0x7fe93c1f2670 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.573+0000 7fe9434d1640 1 -- 172.21.15.76:0/1315371948 >> 172.21.15.76:0/1315371948 conn(0x7fe93c070800 msgr2=0x7fe93c1480c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:29.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.573+0000 7fe9434d1640 1 -- 172.21.15.76:0/1315371948 shutdown_connections 2024-04-25T19:15:29.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.573+0000 7fe9434d1640 1 -- 172.21.15.76:0/1315371948 wait complete. 2024-04-25T19:15:29.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.574+0000 7f0a52ffd640 1 -- 172.21.15.76:0/3569605265 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0a5405f1e0 con 0x7f0a5c0c9e20 2024-04-25T19:15:29.577 INFO:teuthology.orchestra.run.smithi076.stdout:55834574853 2024-04-25T19:15:29.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.576+0000 7f0a50ff9640 1 -- 172.21.15.76:0/3569605265 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0a400745b0 msgr2=0x7f0a40076ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.576+0000 7f0a50ff9640 1 --2- 172.21.15.76:0/3569605265 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0a400745b0 0x7f0a40076ac0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0a58005880 tx=0x7f0a58009040 comp rx=0 tx=0).stop 2024-04-25T19:15:29.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.576+0000 7f0a50ff9640 1 -- 172.21.15.76:0/3569605265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a5c0c9e20 msgr2=0x7f0a5c150270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.576+0000 7f0a50ff9640 1 --2- 172.21.15.76:0/3569605265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a5c0c9e20 0x7f0a5c150270 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009db0 tx=0x7f0a540075c0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.576+0000 7f0a50ff9640 1 -- 172.21.15.76:0/3569605265 shutdown_connections 2024-04-25T19:15:29.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.576+0000 7f0a50ff9640 1 --2- 172.21.15.76:0/3569605265 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0a400745b0 0x7f0a40076ac0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.576+0000 7f0a50ff9640 1 --2- 172.21.15.76:0/3569605265 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0a5c0c9e20 0x7f0a5c150270 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.576+0000 7f0a50ff9640 1 --2- 172.21.15.76:0/3569605265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a5c0c3480 0x7f0a5c0c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.576+0000 7f0a50ff9640 1 --2- 172.21.15.76:0/3569605265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a5c06fc50 0x7f0a5c15b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.576+0000 7f0a50ff9640 1 -- 172.21.15.76:0/3569605265 >> 172.21.15.76:0/3569605265 conn(0x7f0a5c06c5c0 msgr2=0x7f0a5c1e47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:29.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.576+0000 7f0a50ff9640 1 -- 172.21.15.76:0/3569605265 shutdown_connections 2024-04-25T19:15:29.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.576+0000 7f0a50ff9640 1 -- 172.21.15.76:0/3569605265 wait complete. 2024-04-25T19:15:29.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.582+0000 7f880e99d640 1 -- 172.21.15.76:0/1302138543 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f88081510f0 con 0x7f88080cf560 2024-04-25T19:15:29.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.583+0000 7f87fe7fc640 1 -- 172.21.15.76:0/1302138543 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8800060a50 con 0x7f88080cf560 2024-04-25T19:15:29.584 INFO:teuthology.orchestra.run.smithi076.stdout:55834574853 2024-04-25T19:15:29.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.585+0000 7f880e99d640 1 -- 172.21.15.76:0/1302138543 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f87ec074580 msgr2=0x7f87ec076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.585+0000 7f880e99d640 1 --2- 172.21.15.76:0/1302138543 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f87ec074580 0x7f87ec076a90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8804000c00 tx=0x7f8804008030 comp rx=0 tx=0).stop 2024-04-25T19:15:29.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.585+0000 7f880e99d640 1 -- 172.21.15.76:0/1302138543 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88080cf560 msgr2=0x7f88080cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.585+0000 7f880e99d640 1 --2- 172.21.15.76:0/1302138543 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88080cf560 0x7f88080cdbb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8800002230 tx=0x7f880000a6e0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.585+0000 7f880e99d640 1 -- 172.21.15.76:0/1302138543 shutdown_connections 2024-04-25T19:15:29.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.585+0000 7f880e99d640 1 --2- 172.21.15.76:0/1302138543 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f87ec074580 0x7f87ec076a90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.585+0000 7f880e99d640 1 --2- 172.21.15.76:0/1302138543 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f880815cb40 0x7f880814b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.585+0000 7f880e99d640 1 --2- 172.21.15.76:0/1302138543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f880815a6c0 0x7f880814b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.585+0000 7f880e99d640 1 --2- 172.21.15.76:0/1302138543 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f88080cf560 0x7f88080cdbb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.585+0000 7f880e99d640 1 -- 172.21.15.76:0/1302138543 >> 172.21.15.76:0/1302138543 conn(0x7f88080719f0 msgr2=0x7f880806cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:29.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.585+0000 7f880e99d640 1 -- 172.21.15.76:0/1302138543 shutdown_connections 2024-04-25T19:15:29.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.585+0000 7f880e99d640 1 -- 172.21.15.76:0/1302138543 wait complete. 2024-04-25T19:15:29.588 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2024-04-25T19:15:29.588 DEBUG:teuthology.parallel:result is None 2024-04-25T19:15:29.589 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2024-04-25T19:15:29.589 DEBUG:teuthology.parallel:result is None 2024-04-25T19:15:29.597 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.4 2024-04-25T19:15:29.598 DEBUG:teuthology.parallel:result is None 2024-04-25T19:15:29.598 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T19:15:29.598 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T19:15:29.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.744+0000 7f65e4b9d640 1 Processor -- start 2024-04-25T19:15:29.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.744+0000 7f65e4b9d640 1 -- start start 2024-04-25T19:15:29.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.745+0000 7f65e4b9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f65e00c9f60 0x7f65e015a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.745+0000 7f65e4b9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65e0154680 0x7f65e00c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.745+0000 7f65e4b9d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65e0073ce0 con 0x7f65e00cae90 2024-04-25T19:15:29.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.745+0000 7f65e4b9d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f65e00c9ad0 con 0x7f65e00c9f60 2024-04-25T19:15:29.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.745+0000 7f65e4b9d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f65e0073fc0 con 0x7f65e0154680 2024-04-25T19:15:29.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.745+0000 7f65deffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f65e00cae90 0x7f65e0070e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36310/0 (socket says 172.21.15.76:36310) 2024-04-25T19:15:29.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.745+0000 7f65deffd640 1 -- 172.21.15.76:0/358031361 learned_addr learned my addr 172.21.15.76:0/358031361 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:29.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.746+0000 7f65dffff640 1 --2- 172.21.15.76:0/358031361 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65e0154680 0x7f65e00c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.746+0000 7f65df7fe640 1 --2- 172.21.15.76:0/358031361 >> v2:172.21.15.105:3300/0 conn(0x7f65e00c9f60 0x7f65e015a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.746+0000 7f65de7fc640 1 -- 172.21.15.76:0/358031361 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 185773954 0 0) 0x7f65e0073ce0 con 0x7f65e00cae90 2024-04-25T19:15:29.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.746+0000 7f65de7fc640 1 -- 172.21.15.76:0/358031361 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65c8003520 con 0x7f65e00cae90 2024-04-25T19:15:29.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.746+0000 7f65de7fc640 1 -- 172.21.15.76:0/358031361 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1198387763 0 0) 0x7f65c8003520 con 0x7f65e00cae90 2024-04-25T19:15:29.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.746+0000 7f65de7fc640 1 -- 172.21.15.76:0/358031361 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65e0154680 msgr2=0x7f65e00c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:29.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.746+0000 7f65de7fc640 1 --2- 172.21.15.76:0/358031361 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65e0154680 0x7f65e00c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.746+0000 7f65de7fc640 1 -- 172.21.15.76:0/358031361 >> v2:172.21.15.105:3300/0 conn(0x7f65e00c9f60 msgr2=0x7f65e015a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:29.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.746+0000 7f65de7fc640 1 --2- 172.21.15.76:0/358031361 >> v2:172.21.15.105:3300/0 conn(0x7f65e00c9f60 0x7f65e015a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.746+0000 7f65de7fc640 1 -- 172.21.15.76:0/358031361 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e00c9480 con 0x7f65e00cae90 2024-04-25T19:15:29.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.746+0000 7f65de7fc640 1 -- 172.21.15.76:0/358031361 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f65d4001690 con 0x7f65e00cae90 2024-04-25T19:15:29.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.746+0000 7f65de7fc640 1 -- 172.21.15.76:0/358031361 >> v1:172.21.15.76:6789/0 conn(0x7f65e00cae90 legacy=0x7f65e0070e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.746+0000 7f65de7fc640 1 --2- 172.21.15.76:0/358031361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65c80039b0 0x7f65c8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.746+0000 7f65de7fc640 1 --2- 172.21.15.76:0/358031361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65c8004800 0x7f65c8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.746+0000 7f65de7fc640 1 --2- 172.21.15.76:0/358031361 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65e0154680 0x7f65c8005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.746+0000 7f65de7fc640 1 -- 172.21.15.76:0/358031361 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f65e00c9ad0 con 0x7f65c8004800 2024-04-25T19:15:29.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.746+0000 7f65de7fc640 1 -- 172.21.15.76:0/358031361 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f65e0073fc0 con 0x7f65c80039b0 2024-04-25T19:15:29.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.746+0000 7f65de7fc640 1 -- 172.21.15.76:0/358031361 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f65c800ba70 con 0x7f65e0154680 2024-04-25T19:15:29.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.746+0000 7f65deffd640 1 --2- 172.21.15.76:0/358031361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65c8004800 0x7f65c8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.746+0000 7f65df7fe640 1 --2- 172.21.15.76:0/358031361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65c80039b0 0x7f65c8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.746+0000 7f65dffff640 1 --2- 172.21.15.76:0/358031361 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65e0154680 0x7f65c8005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.746+0000 7f65deffd640 1 -- 172.21.15.76:0/358031361 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65e0154680 msgr2=0x7f65c8005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:29.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.746+0000 7f65deffd640 1 --2- 172.21.15.76:0/358031361 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65e0154680 0x7f65c8005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.746+0000 7f65deffd640 1 -- 172.21.15.76:0/358031361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65c80039b0 msgr2=0x7f65c8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:29.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.746+0000 7f65deffd640 1 --2- 172.21.15.76:0/358031361 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65c80039b0 0x7f65c8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.746+0000 7f65deffd640 1 -- 172.21.15.76:0/358031361 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c8003520 con 0x7f65c8004800 2024-04-25T19:15:29.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.747+0000 7f65deffd640 1 --2- 172.21.15.76:0/358031361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65c8004800 0x7f65c8004c10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f65d4007fd0 tx=0x7f65c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb3979d508030ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.747+0000 7f65de7fc640 1 -- 172.21.15.76:0/358031361 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d4007d50 con 0x7f65c8004800 2024-04-25T19:15:29.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.747+0000 7f65de7fc640 1 -- 172.21.15.76:0/358031361 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65d4005af0 con 0x7f65c8004800 2024-04-25T19:15:29.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.747+0000 7f65de7fc640 1 -- 172.21.15.76:0/358031361 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d4007070 con 0x7f65c8004800 2024-04-25T19:15:29.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.748+0000 7f65e4b9d640 1 -- 172.21.15.76:0/358031361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65c8004800 msgr2=0x7f65c8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.748+0000 7f65e4b9d640 1 --2- 172.21.15.76:0/358031361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65c8004800 0x7f65c8004c10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f65d4007fd0 tx=0x7f65c8005830 comp rx=0 tx=0).stop 2024-04-25T19:15:29.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.748+0000 7f65dffff640 1 -- 172.21.15.76:0/358031361 reap_dead start 2024-04-25T19:15:29.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.748+0000 7f65e4b9d640 1 -- 172.21.15.76:0/358031361 shutdown_connections 2024-04-25T19:15:29.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.748+0000 7f65e4b9d640 1 -- 172.21.15.76:0/358031361 >> 172.21.15.76:0/358031361 conn(0x7f65e00cb4e0 msgr2=0x7f65e00cb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:29.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.748+0000 7f65e4b9d640 1 -- 172.21.15.76:0/358031361 shutdown_connections 2024-04-25T19:15:29.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.748+0000 7f65e4b9d640 1 -- 172.21.15.76:0/358031361 wait complete. 2024-04-25T19:15:29.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.750+0000 7f65e4b9d640 1 Processor -- start 2024-04-25T19:15:29.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.750+0000 7f65e4b9d640 1 -- start start 2024-04-25T19:15:29.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.751+0000 7f65e4b9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65c8004800 0x7f65e015da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.751+0000 7f65e4b9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65e014f100 0x7f65e0148fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.751+0000 7f65e4b9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65e01494e0 0x7f65e0151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.751+0000 7f65e4b9d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f65e00749c0 con 0x7f65c8004800 2024-04-25T19:15:29.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.751+0000 7f65e4b9d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f65e00742a0 con 0x7f65e01494e0 2024-04-25T19:15:29.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.751+0000 7f65e4b9d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f65e0074b40 con 0x7f65e014f100 2024-04-25T19:15:29.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.751+0000 7f65df7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65c8004800 0x7f65e015da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.751+0000 7f65df7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65c8004800 0x7f65e015da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58876/0 (socket says 172.21.15.76:58876) 2024-04-25T19:15:29.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.751+0000 7f65df7fe640 1 -- 172.21.15.76:0/1954823540 learned_addr learned my addr 172.21.15.76:0/1954823540 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:29.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.751+0000 7f65deffd640 1 --2- 172.21.15.76:0/1954823540 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65e014f100 0x7f65e0148fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.751+0000 7f65dffff640 1 --2- 172.21.15.76:0/1954823540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65e01494e0 0x7f65e0151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.751+0000 7f65df7fe640 1 -- 172.21.15.76:0/1954823540 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65e014f100 msgr2=0x7f65e0148fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.751+0000 7f65df7fe640 1 --2- 172.21.15.76:0/1954823540 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65e014f100 0x7f65e0148fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.751+0000 7f65df7fe640 1 -- 172.21.15.76:0/1954823540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65e01494e0 msgr2=0x7f65e0151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.751+0000 7f65df7fe640 1 --2- 172.21.15.76:0/1954823540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65e01494e0 0x7f65e0151f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.751+0000 7f65df7fe640 1 -- 172.21.15.76:0/1954823540 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e015df40 con 0x7f65c8004800 2024-04-25T19:15:29.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.751+0000 7f65dffff640 1 --2- 172.21.15.76:0/1954823540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65e01494e0 0x7f65e0151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:29.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.752+0000 7f65df7fe640 1 --2- 172.21.15.76:0/1954823540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65c8004800 0x7f65e015da00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f65d0011910 tx=0x7f65d0011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.752+0000 7f65c7fff640 1 -- 172.21.15.76:0/1954823540 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d000faf0 con 0x7f65c8004800 2024-04-25T19:15:29.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.752+0000 7f65c7fff640 1 -- 172.21.15.76:0/1954823540 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65d000fc90 con 0x7f65c8004800 2024-04-25T19:15:29.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.752+0000 7f65e4b9d640 1 -- 172.21.15.76:0/1954823540 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e01f0210 con 0x7f65c8004800 2024-04-25T19:15:29.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.752+0000 7f65c7fff640 1 -- 172.21.15.76:0/1954823540 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d0014440 con 0x7f65c8004800 2024-04-25T19:15:29.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.752+0000 7f65e4b9d640 1 -- 172.21.15.76:0/1954823540 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e015cf20 con 0x7f65c8004800 2024-04-25T19:15:29.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.753+0000 7f65e4b9d640 1 -- 172.21.15.76:0/1954823540 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65e0050740 con 0x7f65c8004800 2024-04-25T19:15:29.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.753+0000 7f65c7fff640 1 -- 172.21.15.76:0/1954823540 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f65d0012040 con 0x7f65c8004800 2024-04-25T19:15:29.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.754+0000 7f65c7fff640 1 --2- 172.21.15.76:0/1954823540 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f65cc0744e0 0x7f65cc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:29.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.754+0000 7f65c7fff640 1 -- 172.21.15.76:0/1954823540 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f65d0095680 con 0x7f65c8004800 2024-04-25T19:15:29.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.754+0000 7f65deffd640 1 --2- 172.21.15.76:0/1954823540 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f65cc0744e0 0x7f65cc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:29.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.755+0000 7f65deffd640 1 --2- 172.21.15.76:0/1954823540 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f65cc0744e0 0x7f65cc0769f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f65d4007fd0 tx=0x7f65d4009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:29.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.758+0000 7f65c7fff640 1 -- 172.21.15.76:0/1954823540 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f65d0019030 con 0x7f65c8004800 2024-04-25T19:15:29.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.929+0000 7f65e4b9d640 1 -- 172.21.15.76:0/1954823540 --> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f65e00747a0 con 0x7f65cc0744e0 2024-04-25T19:15:29.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.936+0000 7f65c7fff640 1 -- 172.21.15.76:0/1954823540 <== mgr.4109 v2:172.21.15.105:6832/2585775531 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251902 (secure 0 0 0) 0x7f65e00747a0 con 0x7f65cc0744e0 2024-04-25T19:15:29.940 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:29.941 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-04-25T19:15:29.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.939+0000 7f65e4b9d640 1 -- 172.21.15.76:0/1954823540 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f65cc0744e0 msgr2=0x7f65cc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.939+0000 7f65e4b9d640 1 --2- 172.21.15.76:0/1954823540 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f65cc0744e0 0x7f65cc0769f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f65d4007fd0 tx=0x7f65d4009030 comp rx=0 tx=0).stop 2024-04-25T19:15:29.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.939+0000 7f65e4b9d640 1 -- 172.21.15.76:0/1954823540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65c8004800 msgr2=0x7f65e015da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:29.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.939+0000 7f65e4b9d640 1 --2- 172.21.15.76:0/1954823540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65c8004800 0x7f65e015da00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f65d0011910 tx=0x7f65d0011dd0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.939+0000 7f65e4b9d640 1 -- 172.21.15.76:0/1954823540 shutdown_connections 2024-04-25T19:15:29.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.939+0000 7f65e4b9d640 1 --2- 172.21.15.76:0/1954823540 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f65cc0744e0 0x7f65cc0769f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.939+0000 7f65e4b9d640 1 --2- 172.21.15.76:0/1954823540 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f65e01494e0 0x7f65e0151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.939+0000 7f65e4b9d640 1 --2- 172.21.15.76:0/1954823540 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f65e014f100 0x7f65e0148fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.939+0000 7f65e4b9d640 1 --2- 172.21.15.76:0/1954823540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65c8004800 0x7f65e015da00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:29.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.939+0000 7f65e4b9d640 1 -- 172.21.15.76:0/1954823540 >> 172.21.15.76:0/1954823540 conn(0x7f65e00cb4e0 msgr2=0x7f65e015ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:29.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.939+0000 7f65e4b9d640 1 -- 172.21.15.76:0/1954823540 shutdown_connections 2024-04-25T19:15:29.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:29.939+0000 7f65e4b9d640 1 -- 172.21.15.76:0/1954823540 wait complete. 2024-04-25T19:15:29.952 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-25T19:15:29.831096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218016,"kb_used_data":1888,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756704,"statfs":{"total":773094113280,"available":772870864896,"internally_reserved":0,"allocated":1933312,"data_stored":1790396,"data_compressed":64838,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002587"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425589+0000","last_change":"2024-04-25T19:15:19.433569+0000","last_active":"2024-04-25T19:15:20.425589+0000","last_peered":"2024-04-25T19:15:20.425589+0000","last_clean":"2024-04-25T19:15:20.425589+0000","last_became_active":"2024-04-25T19:15:19.431332+0000","last_became_peered":"2024-04-25T19:15:19.431332+0000","last_unstale":"2024-04-25T19:15:20.425589+0000","last_undegraded":"2024-04-25T19:15:20.425589+0000","last_fullsized":"2024-04-25T19:15:20.425589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:57:01.213834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404926+0000","last_change":"2024-04-25T19:15:19.418695+0000","last_active":"2024-04-25T19:15:20.404926+0000","last_peered":"2024-04-25T19:15:20.404926+0000","last_clean":"2024-04-25T19:15:20.404926+0000","last_became_active":"2024-04-25T19:15:19.418235+0000","last_became_peered":"2024-04-25T19:15:19.418235+0000","last_unstale":"2024-04-25T19:15:20.404926+0000","last_undegraded":"2024-04-25T19:15:20.404926+0000","last_fullsized":"2024-04-25T19:15:20.404926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:30.516472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804491+0000","last_change":"2024-04-25T19:15:19.416983+0000","last_active":"2024-04-25T19:15:20.804491+0000","last_peered":"2024-04-25T19:15:20.804491+0000","last_clean":"2024-04-25T19:15:20.804491+0000","last_became_active":"2024-04-25T19:15:19.416443+0000","last_became_peered":"2024-04-25T19:15:19.416443+0000","last_unstale":"2024-04-25T19:15:20.804491+0000","last_undegraded":"2024-04-25T19:15:20.804491+0000","last_fullsized":"2024-04-25T19:15:20.804491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:02.794164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848308+0000","last_change":"2024-04-25T19:15:19.429058+0000","last_active":"2024-04-25T19:15:20.848308+0000","last_peered":"2024-04-25T19:15:20.848308+0000","last_clean":"2024-04-25T19:15:20.848308+0000","last_became_active":"2024-04-25T19:15:19.428419+0000","last_became_peered":"2024-04-25T19:15:19.428419+0000","last_unstale":"2024-04-25T19:15:20.848308+0000","last_undegraded":"2024-04-25T19:15:20.848308+0000","last_fullsized":"2024-04-25T19:15:20.848308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:23:56.817616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425402+0000","last_change":"2024-04-25T19:15:19.427615+0000","last_active":"2024-04-25T19:15:20.425402+0000","last_peered":"2024-04-25T19:15:20.425402+0000","last_clean":"2024-04-25T19:15:20.425402+0000","last_became_active":"2024-04-25T19:15:19.426968+0000","last_became_peered":"2024-04-25T19:15:19.426968+0000","last_unstale":"2024-04-25T19:15:20.425402+0000","last_undegraded":"2024-04-25T19:15:20.425402+0000","last_fullsized":"2024-04-25T19:15:20.425402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:42:00.686712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847989+0000","last_change":"2024-04-25T19:15:19.417602+0000","last_active":"2024-04-25T19:15:20.847989+0000","last_peered":"2024-04-25T19:15:20.847989+0000","last_clean":"2024-04-25T19:15:20.847989+0000","last_became_active":"2024-04-25T19:15:19.417292+0000","last_became_peered":"2024-04-25T19:15:19.417292+0000","last_unstale":"2024-04-25T19:15:20.847989+0000","last_undegraded":"2024-04-25T19:15:20.847989+0000","last_fullsized":"2024-04-25T19:15:20.847989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:37:43.583809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425114+0000","last_change":"2024-04-25T19:15:19.428180+0000","last_active":"2024-04-25T19:15:20.425114+0000","last_peered":"2024-04-25T19:15:20.425114+0000","last_clean":"2024-04-25T19:15:20.425114+0000","last_became_active":"2024-04-25T19:15:19.427852+0000","last_became_peered":"2024-04-25T19:15:19.427852+0000","last_unstale":"2024-04-25T19:15:20.425114+0000","last_undegraded":"2024-04-25T19:15:20.425114+0000","last_fullsized":"2024-04-25T19:15:20.425114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:51.300880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405446+0000","last_change":"2024-04-25T19:15:19.418253+0000","last_active":"2024-04-25T19:15:20.405446+0000","last_peered":"2024-04-25T19:15:20.405446+0000","last_clean":"2024-04-25T19:15:20.405446+0000","last_became_active":"2024-04-25T19:15:19.417183+0000","last_became_peered":"2024-04-25T19:15:19.417183+0000","last_unstale":"2024-04-25T19:15:20.405446+0000","last_undegraded":"2024-04-25T19:15:20.405446+0000","last_fullsized":"2024-04-25T19:15:20.405446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:05:13.181646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425578+0000","last_change":"2024-04-25T19:15:19.431671+0000","last_active":"2024-04-25T19:15:20.425578+0000","last_peered":"2024-04-25T19:15:20.425578+0000","last_clean":"2024-04-25T19:15:20.425578+0000","last_became_active":"2024-04-25T19:15:19.431240+0000","last_became_peered":"2024-04-25T19:15:19.431240+0000","last_unstale":"2024-04-25T19:15:20.425578+0000","last_undegraded":"2024-04-25T19:15:20.425578+0000","last_fullsized":"2024-04-25T19:15:20.425578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:22.759462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.406080+0000","last_change":"2024-04-25T19:15:19.416304+0000","last_active":"2024-04-25T19:15:20.406080+0000","last_peered":"2024-04-25T19:15:20.406080+0000","last_clean":"2024-04-25T19:15:20.406080+0000","last_became_active":"2024-04-25T19:15:19.415967+0000","last_became_peered":"2024-04-25T19:15:19.415967+0000","last_unstale":"2024-04-25T19:15:20.406080+0000","last_undegraded":"2024-04-25T19:15:20.406080+0000","last_fullsized":"2024-04-25T19:15:20.406080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:30:52.958731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405773+0000","last_change":"2024-04-25T19:15:19.417596+0000","last_active":"2024-04-25T19:15:20.405773+0000","last_peered":"2024-04-25T19:15:20.405773+0000","last_clean":"2024-04-25T19:15:20.405773+0000","last_became_active":"2024-04-25T19:15:19.417230+0000","last_became_peered":"2024-04-25T19:15:19.417230+0000","last_unstale":"2024-04-25T19:15:20.405773+0000","last_undegraded":"2024-04-25T19:15:20.405773+0000","last_fullsized":"2024-04-25T19:15:20.405773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:28:12.387651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822843+0000","last_change":"2024-04-25T19:15:19.420564+0000","last_active":"2024-04-25T19:15:20.822843+0000","last_peered":"2024-04-25T19:15:20.822843+0000","last_clean":"2024-04-25T19:15:20.822843+0000","last_became_active":"2024-04-25T19:15:19.420044+0000","last_became_peered":"2024-04-25T19:15:19.420044+0000","last_unstale":"2024-04-25T19:15:20.822843+0000","last_undegraded":"2024-04-25T19:15:20.822843+0000","last_fullsized":"2024-04-25T19:15:20.822843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:04.066030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405269+0000","last_change":"2024-04-25T19:15:19.415500+0000","last_active":"2024-04-25T19:15:20.405269+0000","last_peered":"2024-04-25T19:15:20.405269+0000","last_clean":"2024-04-25T19:15:20.405269+0000","last_became_active":"2024-04-25T19:15:19.414007+0000","last_became_peered":"2024-04-25T19:15:19.414007+0000","last_unstale":"2024-04-25T19:15:20.405269+0000","last_undegraded":"2024-04-25T19:15:20.405269+0000","last_fullsized":"2024-04-25T19:15:20.405269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:56:12.320555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804235+0000","last_change":"2024-04-25T19:15:19.423884+0000","last_active":"2024-04-25T19:15:20.804235+0000","last_peered":"2024-04-25T19:15:20.804235+0000","last_clean":"2024-04-25T19:15:20.804235+0000","last_became_active":"2024-04-25T19:15:19.422575+0000","last_became_peered":"2024-04-25T19:15:19.422575+0000","last_unstale":"2024-04-25T19:15:20.804235+0000","last_undegraded":"2024-04-25T19:15:20.804235+0000","last_fullsized":"2024-04-25T19:15:20.804235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:30.988950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404396+0000","last_change":"2024-04-25T19:15:19.424267+0000","last_active":"2024-04-25T19:15:20.404396+0000","last_peered":"2024-04-25T19:15:20.404396+0000","last_clean":"2024-04-25T19:15:20.404396+0000","last_became_active":"2024-04-25T19:15:19.422953+0000","last_became_peered":"2024-04-25T19:15:19.422953+0000","last_unstale":"2024-04-25T19:15:20.404396+0000","last_undegraded":"2024-04-25T19:15:20.404396+0000","last_fullsized":"2024-04-25T19:15:20.404396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:42.823233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847983+0000","last_change":"2024-04-25T19:15:19.420374+0000","last_active":"2024-04-25T19:15:20.847983+0000","last_peered":"2024-04-25T19:15:20.847983+0000","last_clean":"2024-04-25T19:15:20.847983+0000","last_became_active":"2024-04-25T19:15:19.420015+0000","last_became_peered":"2024-04-25T19:15:19.420015+0000","last_unstale":"2024-04-25T19:15:20.847983+0000","last_undegraded":"2024-04-25T19:15:20.847983+0000","last_fullsized":"2024-04-25T19:15:20.847983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:04:26.719359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848705+0000","last_change":"2024-04-25T19:15:19.434476+0000","last_active":"2024-04-25T19:15:20.848705+0000","last_peered":"2024-04-25T19:15:20.848705+0000","last_clean":"2024-04-25T19:15:20.848705+0000","last_became_active":"2024-04-25T19:15:19.433836+0000","last_became_peered":"2024-04-25T19:15:19.433836+0000","last_unstale":"2024-04-25T19:15:20.848705+0000","last_undegraded":"2024-04-25T19:15:20.848705+0000","last_fullsized":"2024-04-25T19:15:20.848705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:13.234829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847359+0000","last_change":"2024-04-25T19:15:19.415521+0000","last_active":"2024-04-25T19:15:20.847359+0000","last_peered":"2024-04-25T19:15:20.847359+0000","last_clean":"2024-04-25T19:15:20.847359+0000","last_became_active":"2024-04-25T19:15:19.415177+0000","last_became_peered":"2024-04-25T19:15:19.415177+0000","last_unstale":"2024-04-25T19:15:20.847359+0000","last_undegraded":"2024-04-25T19:15:20.847359+0000","last_fullsized":"2024-04-25T19:15:20.847359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:24.355619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425590+0000","last_change":"2024-04-25T19:15:19.430291+0000","last_active":"2024-04-25T19:15:20.425590+0000","last_peered":"2024-04-25T19:15:20.425590+0000","last_clean":"2024-04-25T19:15:20.425590+0000","last_became_active":"2024-04-25T19:15:19.427226+0000","last_became_peered":"2024-04-25T19:15:19.427226+0000","last_unstale":"2024-04-25T19:15:20.425590+0000","last_undegraded":"2024-04-25T19:15:20.425590+0000","last_fullsized":"2024-04-25T19:15:20.425590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:38.464211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405750+0000","last_change":"2024-04-25T19:15:19.421696+0000","last_active":"2024-04-25T19:15:20.405750+0000","last_peered":"2024-04-25T19:15:20.405750+0000","last_clean":"2024-04-25T19:15:20.405750+0000","last_became_active":"2024-04-25T19:15:19.421300+0000","last_became_peered":"2024-04-25T19:15:19.421300+0000","last_unstale":"2024-04-25T19:15:20.405750+0000","last_undegraded":"2024-04-25T19:15:20.405750+0000","last_fullsized":"2024-04-25T19:15:20.405750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:15.226398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804410+0000","last_change":"2024-04-25T19:15:19.427880+0000","last_active":"2024-04-25T19:15:20.804410+0000","last_peered":"2024-04-25T19:15:20.804410+0000","last_clean":"2024-04-25T19:15:20.804410+0000","last_became_active":"2024-04-25T19:15:19.427579+0000","last_became_peered":"2024-04-25T19:15:19.427579+0000","last_unstale":"2024-04-25T19:15:20.804410+0000","last_undegraded":"2024-04-25T19:15:20.804410+0000","last_fullsized":"2024-04-25T19:15:20.804410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:47:10.657715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848145+0000","last_change":"2024-04-25T19:15:19.430409+0000","last_active":"2024-04-25T19:15:20.848145+0000","last_peered":"2024-04-25T19:15:20.848145+0000","last_clean":"2024-04-25T19:15:20.848145+0000","last_became_active":"2024-04-25T19:15:19.428305+0000","last_became_peered":"2024-04-25T19:15:19.428305+0000","last_unstale":"2024-04-25T19:15:20.848145+0000","last_undegraded":"2024-04-25T19:15:20.848145+0000","last_fullsized":"2024-04-25T19:15:20.848145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:10:01.268963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405396+0000","last_change":"2024-04-25T19:15:19.424435+0000","last_active":"2024-04-25T19:15:20.405396+0000","last_peered":"2024-04-25T19:15:20.405396+0000","last_clean":"2024-04-25T19:15:20.405396+0000","last_became_active":"2024-04-25T19:15:19.422763+0000","last_became_peered":"2024-04-25T19:15:19.422763+0000","last_unstale":"2024-04-25T19:15:20.405396+0000","last_undegraded":"2024-04-25T19:15:20.405396+0000","last_fullsized":"2024-04-25T19:15:20.405396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:58.092412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848072+0000","last_change":"2024-04-25T19:15:19.420275+0000","last_active":"2024-04-25T19:15:20.848072+0000","last_peered":"2024-04-25T19:15:20.848072+0000","last_clean":"2024-04-25T19:15:20.848072+0000","last_became_active":"2024-04-25T19:15:19.419936+0000","last_became_peered":"2024-04-25T19:15:19.419936+0000","last_unstale":"2024-04-25T19:15:20.848072+0000","last_undegraded":"2024-04-25T19:15:20.848072+0000","last_fullsized":"2024-04-25T19:15:20.848072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:55.710620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405087+0000","last_change":"2024-04-25T19:15:19.423408+0000","last_active":"2024-04-25T19:15:20.405087+0000","last_peered":"2024-04-25T19:15:20.405087+0000","last_clean":"2024-04-25T19:15:20.405087+0000","last_became_active":"2024-04-25T19:15:19.423116+0000","last_became_peered":"2024-04-25T19:15:19.423116+0000","last_unstale":"2024-04-25T19:15:20.405087+0000","last_undegraded":"2024-04-25T19:15:20.405087+0000","last_fullsized":"2024-04-25T19:15:20.405087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:18:57.787013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.406015+0000","last_change":"2024-04-25T19:15:19.428826+0000","last_active":"2024-04-25T19:15:20.406015+0000","last_peered":"2024-04-25T19:15:20.406015+0000","last_clean":"2024-04-25T19:15:20.406015+0000","last_became_active":"2024-04-25T19:15:19.428474+0000","last_became_peered":"2024-04-25T19:15:19.428474+0000","last_unstale":"2024-04-25T19:15:20.406015+0000","last_undegraded":"2024-04-25T19:15:20.406015+0000","last_fullsized":"2024-04-25T19:15:20.406015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:19.974851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404739+0000","last_change":"2024-04-25T19:15:19.423353+0000","last_active":"2024-04-25T19:15:20.404739+0000","last_peered":"2024-04-25T19:15:20.404739+0000","last_clean":"2024-04-25T19:15:20.404739+0000","last_became_active":"2024-04-25T19:15:19.422795+0000","last_became_peered":"2024-04-25T19:15:19.422795+0000","last_unstale":"2024-04-25T19:15:20.404739+0000","last_undegraded":"2024-04-25T19:15:20.404739+0000","last_fullsized":"2024-04-25T19:15:20.404739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:34.119976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405076+0000","last_change":"2024-04-25T19:15:19.421809+0000","last_active":"2024-04-25T19:15:20.405076+0000","last_peered":"2024-04-25T19:15:20.405076+0000","last_clean":"2024-04-25T19:15:20.405076+0000","last_became_active":"2024-04-25T19:15:19.421515+0000","last_became_peered":"2024-04-25T19:15:19.421515+0000","last_unstale":"2024-04-25T19:15:20.405076+0000","last_undegraded":"2024-04-25T19:15:20.405076+0000","last_fullsized":"2024-04-25T19:15:20.405076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:22.603291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804477+0000","last_change":"2024-04-25T19:15:19.410183+0000","last_active":"2024-04-25T19:15:20.804477+0000","last_peered":"2024-04-25T19:15:20.804477+0000","last_clean":"2024-04-25T19:15:20.804477+0000","last_became_active":"2024-04-25T19:15:19.409857+0000","last_became_peered":"2024-04-25T19:15:19.409857+0000","last_unstale":"2024-04-25T19:15:20.804477+0000","last_undegraded":"2024-04-25T19:15:20.804477+0000","last_fullsized":"2024-04-25T19:15:20.804477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:30.180786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848239+0000","last_change":"2024-04-25T19:15:19.425524+0000","last_active":"2024-04-25T19:15:20.848239+0000","last_peered":"2024-04-25T19:15:20.848239+0000","last_clean":"2024-04-25T19:15:20.848239+0000","last_became_active":"2024-04-25T19:15:19.424578+0000","last_became_peered":"2024-04-25T19:15:19.424578+0000","last_unstale":"2024-04-25T19:15:20.848239+0000","last_undegraded":"2024-04-25T19:15:20.848239+0000","last_fullsized":"2024-04-25T19:15:20.848239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:02.103879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847243+0000","last_change":"2024-04-25T19:15:19.426720+0000","last_active":"2024-04-25T19:15:20.847243+0000","last_peered":"2024-04-25T19:15:20.847243+0000","last_clean":"2024-04-25T19:15:20.847243+0000","last_became_active":"2024-04-25T19:15:19.426139+0000","last_became_peered":"2024-04-25T19:15:19.426139+0000","last_unstale":"2024-04-25T19:15:20.847243+0000","last_undegraded":"2024-04-25T19:15:20.847243+0000","last_fullsized":"2024-04-25T19:15:20.847243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:31.608993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405390+0000","last_change":"2024-04-25T19:15:19.413805+0000","last_active":"2024-04-25T19:15:20.405390+0000","last_peered":"2024-04-25T19:15:20.405390+0000","last_clean":"2024-04-25T19:15:20.405390+0000","last_became_active":"2024-04-25T19:15:19.413455+0000","last_became_peered":"2024-04-25T19:15:19.413455+0000","last_unstale":"2024-04-25T19:15:20.405390+0000","last_undegraded":"2024-04-25T19:15:20.405390+0000","last_fullsized":"2024-04-25T19:15:20.405390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:05:45.008076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822709+0000","last_change":"2024-04-25T19:15:19.423770+0000","last_active":"2024-04-25T19:15:20.822709+0000","last_peered":"2024-04-25T19:15:20.822709+0000","last_clean":"2024-04-25T19:15:20.822709+0000","last_became_active":"2024-04-25T19:15:19.423406+0000","last_became_peered":"2024-04-25T19:15:19.423406+0000","last_unstale":"2024-04-25T19:15:20.822709+0000","last_undegraded":"2024-04-25T19:15:20.822709+0000","last_fullsized":"2024-04-25T19:15:20.822709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:41:23.669832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405392+0000","last_change":"2024-04-25T19:15:18.422035+0000","last_active":"2024-04-25T19:15:20.405392+0000","last_peered":"2024-04-25T19:15:20.405392+0000","last_clean":"2024-04-25T19:15:20.405392+0000","last_became_active":"2024-04-25T19:15:18.421658+0000","last_became_peered":"2024-04-25T19:15:18.421658+0000","last_unstale":"2024-04-25T19:15:20.405392+0000","last_undegraded":"2024-04-25T19:15:20.405392+0000","last_fullsized":"2024-04-25T19:15:20.405392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:23.349126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804571+0000","last_change":"2024-04-25T19:15:19.421805+0000","last_active":"2024-04-25T19:15:20.804571+0000","last_peered":"2024-04-25T19:15:20.804571+0000","last_clean":"2024-04-25T19:15:20.804571+0000","last_became_active":"2024-04-25T19:15:19.421497+0000","last_became_peered":"2024-04-25T19:15:19.421497+0000","last_unstale":"2024-04-25T19:15:20.804571+0000","last_undegraded":"2024-04-25T19:15:20.804571+0000","last_fullsized":"2024-04-25T19:15:20.804571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:29:29.949508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405561+0000","last_change":"2024-04-25T19:15:18.416894+0000","last_active":"2024-04-25T19:15:20.405561+0000","last_peered":"2024-04-25T19:15:20.405561+0000","last_clean":"2024-04-25T19:15:20.405561+0000","last_became_active":"2024-04-25T19:15:18.416539+0000","last_became_peered":"2024-04-25T19:15:18.416539+0000","last_unstale":"2024-04-25T19:15:20.405561+0000","last_undegraded":"2024-04-25T19:15:20.405561+0000","last_fullsized":"2024-04-25T19:15:20.405561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:30.388606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405329+0000","last_change":"2024-04-25T19:15:18.420129+0000","last_active":"2024-04-25T19:15:20.405329+0000","last_peered":"2024-04-25T19:15:20.405329+0000","last_clean":"2024-04-25T19:15:20.405329+0000","last_became_active":"2024-04-25T19:15:18.419732+0000","last_became_peered":"2024-04-25T19:15:18.419732+0000","last_unstale":"2024-04-25T19:15:20.405329+0000","last_undegraded":"2024-04-25T19:15:20.405329+0000","last_fullsized":"2024-04-25T19:15:20.405329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:41:48.673989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405615+0000","last_change":"2024-04-25T19:15:19.415655+0000","last_active":"2024-04-25T19:15:20.405615+0000","last_peered":"2024-04-25T19:15:20.405615+0000","last_clean":"2024-04-25T19:15:20.405615+0000","last_became_active":"2024-04-25T19:15:19.415287+0000","last_became_peered":"2024-04-25T19:15:19.415287+0000","last_unstale":"2024-04-25T19:15:20.405615+0000","last_undegraded":"2024-04-25T19:15:20.405615+0000","last_fullsized":"2024-04-25T19:15:20.405615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:49.155289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848076+0000","last_change":"2024-04-25T19:15:18.420711+0000","last_active":"2024-04-25T19:15:20.848076+0000","last_peered":"2024-04-25T19:15:20.848076+0000","last_clean":"2024-04-25T19:15:20.848076+0000","last_became_active":"2024-04-25T19:15:18.420362+0000","last_became_peered":"2024-04-25T19:15:18.420362+0000","last_unstale":"2024-04-25T19:15:20.848076+0000","last_undegraded":"2024-04-25T19:15:20.848076+0000","last_fullsized":"2024-04-25T19:15:20.848076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:48.245193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404375+0000","last_change":"2024-04-25T19:15:19.424356+0000","last_active":"2024-04-25T19:15:20.404375+0000","last_peered":"2024-04-25T19:15:20.404375+0000","last_clean":"2024-04-25T19:15:20.404375+0000","last_became_active":"2024-04-25T19:15:19.423556+0000","last_became_peered":"2024-04-25T19:15:19.423556+0000","last_unstale":"2024-04-25T19:15:20.404375+0000","last_undegraded":"2024-04-25T19:15:20.404375+0000","last_fullsized":"2024-04-25T19:15:20.404375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:19:35.427261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405514+0000","last_change":"2024-04-25T19:15:18.419250+0000","last_active":"2024-04-25T19:15:20.405514+0000","last_peered":"2024-04-25T19:15:20.405514+0000","last_clean":"2024-04-25T19:15:20.405514+0000","last_became_active":"2024-04-25T19:15:18.418927+0000","last_became_peered":"2024-04-25T19:15:18.418927+0000","last_unstale":"2024-04-25T19:15:20.405514+0000","last_undegraded":"2024-04-25T19:15:20.405514+0000","last_fullsized":"2024-04-25T19:15:20.405514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:26.012965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848606+0000","last_change":"2024-04-25T19:15:19.430117+0000","last_active":"2024-04-25T19:15:20.848606+0000","last_peered":"2024-04-25T19:15:20.848606+0000","last_clean":"2024-04-25T19:15:20.848606+0000","last_became_active":"2024-04-25T19:15:19.428985+0000","last_became_peered":"2024-04-25T19:15:19.428985+0000","last_unstale":"2024-04-25T19:15:20.848606+0000","last_undegraded":"2024-04-25T19:15:20.848606+0000","last_fullsized":"2024-04-25T19:15:20.848606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:36:29.069541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848490+0000","last_change":"2024-04-25T19:15:18.422931+0000","last_active":"2024-04-25T19:15:20.848490+0000","last_peered":"2024-04-25T19:15:20.848490+0000","last_clean":"2024-04-25T19:15:20.848490+0000","last_became_active":"2024-04-25T19:15:18.422610+0000","last_became_peered":"2024-04-25T19:15:18.422610+0000","last_unstale":"2024-04-25T19:15:20.848490+0000","last_undegraded":"2024-04-25T19:15:20.848490+0000","last_fullsized":"2024-04-25T19:15:20.848490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:18.489484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804598+0000","last_change":"2024-04-25T19:15:19.416109+0000","last_active":"2024-04-25T19:15:20.804598+0000","last_peered":"2024-04-25T19:15:20.804598+0000","last_clean":"2024-04-25T19:15:20.804598+0000","last_became_active":"2024-04-25T19:15:19.415734+0000","last_became_peered":"2024-04-25T19:15:19.415734+0000","last_unstale":"2024-04-25T19:15:20.804598+0000","last_undegraded":"2024-04-25T19:15:20.804598+0000","last_fullsized":"2024-04-25T19:15:20.804598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:57:31.406303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:24.411207+0000","last_change":"2024-04-25T19:15:18.428677+0000","last_active":"2024-04-25T19:15:24.411207+0000","last_peered":"2024-04-25T19:15:24.411207+0000","last_clean":"2024-04-25T19:15:24.411207+0000","last_became_active":"2024-04-25T19:15:18.428432+0000","last_became_peered":"2024-04-25T19:15:18.428432+0000","last_unstale":"2024-04-25T19:15:24.411207+0000","last_undegraded":"2024-04-25T19:15:24.411207+0000","last_fullsized":"2024-04-25T19:15:24.411207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:38.478595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822814+0000","last_change":"2024-04-25T19:15:19.428245+0000","last_active":"2024-04-25T19:15:20.822814+0000","last_peered":"2024-04-25T19:15:20.822814+0000","last_clean":"2024-04-25T19:15:20.822814+0000","last_became_active":"2024-04-25T19:15:19.427941+0000","last_became_peered":"2024-04-25T19:15:19.427941+0000","last_unstale":"2024-04-25T19:15:20.822814+0000","last_undegraded":"2024-04-25T19:15:20.822814+0000","last_fullsized":"2024-04-25T19:15:20.822814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:09.947923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:24.414669+0000","last_change":"2024-04-25T19:15:18.420881+0000","last_active":"2024-04-25T19:15:24.414669+0000","last_peered":"2024-04-25T19:15:24.414669+0000","last_clean":"2024-04-25T19:15:24.414669+0000","last_became_active":"2024-04-25T19:15:18.420303+0000","last_became_peered":"2024-04-25T19:15:18.420303+0000","last_unstale":"2024-04-25T19:15:24.414669+0000","last_undegraded":"2024-04-25T19:15:24.414669+0000","last_fullsized":"2024-04-25T19:15:24.414669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:39:24.356298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425356+0000","last_change":"2024-04-25T19:15:19.432735+0000","last_active":"2024-04-25T19:15:20.425356+0000","last_peered":"2024-04-25T19:15:20.425356+0000","last_clean":"2024-04-25T19:15:20.425356+0000","last_became_active":"2024-04-25T19:15:19.431828+0000","last_became_peered":"2024-04-25T19:15:19.431828+0000","last_unstale":"2024-04-25T19:15:20.425356+0000","last_undegraded":"2024-04-25T19:15:20.425356+0000","last_fullsized":"2024-04-25T19:15:20.425356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:20.189230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:24.414315+0000","last_change":"2024-04-25T19:15:18.421008+0000","last_active":"2024-04-25T19:15:24.414315+0000","last_peered":"2024-04-25T19:15:24.414315+0000","last_clean":"2024-04-25T19:15:24.414315+0000","last_became_active":"2024-04-25T19:15:18.420402+0000","last_became_peered":"2024-04-25T19:15:18.420402+0000","last_unstale":"2024-04-25T19:15:24.414315+0000","last_undegraded":"2024-04-25T19:15:24.414315+0000","last_fullsized":"2024-04-25T19:15:24.414315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:05:28.723958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804304+0000","last_change":"2024-04-25T19:15:19.424805+0000","last_active":"2024-04-25T19:15:20.804304+0000","last_peered":"2024-04-25T19:15:20.804304+0000","last_clean":"2024-04-25T19:15:20.804304+0000","last_became_active":"2024-04-25T19:15:19.424512+0000","last_became_peered":"2024-04-25T19:15:19.424512+0000","last_unstale":"2024-04-25T19:15:20.804304+0000","last_undegraded":"2024-04-25T19:15:20.804304+0000","last_fullsized":"2024-04-25T19:15:20.804304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:50.423186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848213+0000","last_change":"2024-04-25T19:15:18.426805+0000","last_active":"2024-04-25T19:15:20.848213+0000","last_peered":"2024-04-25T19:15:20.848213+0000","last_clean":"2024-04-25T19:15:20.848213+0000","last_became_active":"2024-04-25T19:15:18.426472+0000","last_became_peered":"2024-04-25T19:15:18.426472+0000","last_unstale":"2024-04-25T19:15:20.848213+0000","last_undegraded":"2024-04-25T19:15:20.848213+0000","last_fullsized":"2024-04-25T19:15:20.848213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:29.373363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425459+0000","last_change":"2024-04-25T19:15:19.428780+0000","last_active":"2024-04-25T19:15:20.425459+0000","last_peered":"2024-04-25T19:15:20.425459+0000","last_clean":"2024-04-25T19:15:20.425459+0000","last_became_active":"2024-04-25T19:15:19.428450+0000","last_became_peered":"2024-04-25T19:15:19.428450+0000","last_unstale":"2024-04-25T19:15:20.425459+0000","last_undegraded":"2024-04-25T19:15:20.425459+0000","last_fullsized":"2024-04-25T19:15:20.425459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:01.203880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405399+0000","last_change":"2024-04-25T19:15:18.417740+0000","last_active":"2024-04-25T19:15:20.405399+0000","last_peered":"2024-04-25T19:15:20.405399+0000","last_clean":"2024-04-25T19:15:20.405399+0000","last_became_active":"2024-04-25T19:15:18.417404+0000","last_became_peered":"2024-04-25T19:15:18.417404+0000","last_unstale":"2024-04-25T19:15:20.405399+0000","last_undegraded":"2024-04-25T19:15:20.405399+0000","last_fullsized":"2024-04-25T19:15:20.405399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:21.737283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405391+0000","last_change":"2024-04-25T19:15:19.411214+0000","last_active":"2024-04-25T19:15:20.405391+0000","last_peered":"2024-04-25T19:15:20.405391+0000","last_clean":"2024-04-25T19:15:20.405391+0000","last_became_active":"2024-04-25T19:15:19.410814+0000","last_became_peered":"2024-04-25T19:15:19.410814+0000","last_unstale":"2024-04-25T19:15:20.405391+0000","last_undegraded":"2024-04-25T19:15:20.405391+0000","last_fullsized":"2024-04-25T19:15:20.405391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:42.430277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:24.417443+0000","last_change":"2024-04-25T19:15:18.422226+0000","last_active":"2024-04-25T19:15:24.417443+0000","last_peered":"2024-04-25T19:15:24.417443+0000","last_clean":"2024-04-25T19:15:24.417443+0000","last_became_active":"2024-04-25T19:15:18.421900+0000","last_became_peered":"2024-04-25T19:15:18.421900+0000","last_unstale":"2024-04-25T19:15:24.417443+0000","last_undegraded":"2024-04-25T19:15:24.417443+0000","last_fullsized":"2024-04-25T19:15:24.417443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:18.040114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425347+0000","last_change":"2024-04-25T19:15:19.428885+0000","last_active":"2024-04-25T19:15:20.425347+0000","last_peered":"2024-04-25T19:15:20.425347+0000","last_clean":"2024-04-25T19:15:20.425347+0000","last_became_active":"2024-04-25T19:15:19.428563+0000","last_became_peered":"2024-04-25T19:15:19.428563+0000","last_unstale":"2024-04-25T19:15:20.425347+0000","last_undegraded":"2024-04-25T19:15:20.425347+0000","last_fullsized":"2024-04-25T19:15:20.425347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:43.469189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822831+0000","last_change":"2024-04-25T19:15:18.422125+0000","last_active":"2024-04-25T19:15:20.822831+0000","last_peered":"2024-04-25T19:15:20.822831+0000","last_clean":"2024-04-25T19:15:20.822831+0000","last_became_active":"2024-04-25T19:15:18.421777+0000","last_became_peered":"2024-04-25T19:15:18.421777+0000","last_unstale":"2024-04-25T19:15:20.822831+0000","last_undegraded":"2024-04-25T19:15:20.822831+0000","last_fullsized":"2024-04-25T19:15:20.822831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:24.289010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425489+0000","last_change":"2024-04-25T19:15:19.429675+0000","last_active":"2024-04-25T19:15:20.425489+0000","last_peered":"2024-04-25T19:15:20.425489+0000","last_clean":"2024-04-25T19:15:20.425489+0000","last_became_active":"2024-04-25T19:15:19.429316+0000","last_became_peered":"2024-04-25T19:15:19.429316+0000","last_unstale":"2024-04-25T19:15:20.425489+0000","last_undegraded":"2024-04-25T19:15:20.425489+0000","last_fullsized":"2024-04-25T19:15:20.425489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:09:55.521944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:24.410093+0000","last_change":"2024-04-25T19:15:18.418661+0000","last_active":"2024-04-25T19:15:24.410093+0000","last_peered":"2024-04-25T19:15:24.410093+0000","last_clean":"2024-04-25T19:15:24.410093+0000","last_became_active":"2024-04-25T19:15:18.418359+0000","last_became_peered":"2024-04-25T19:15:18.418359+0000","last_unstale":"2024-04-25T19:15:24.410093+0000","last_undegraded":"2024-04-25T19:15:24.410093+0000","last_fullsized":"2024-04-25T19:15:24.410093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:49:22.023652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425687+0000","last_change":"2024-04-25T19:15:19.434853+0000","last_active":"2024-04-25T19:15:20.425687+0000","last_peered":"2024-04-25T19:15:20.425687+0000","last_clean":"2024-04-25T19:15:20.425687+0000","last_became_active":"2024-04-25T19:15:19.434508+0000","last_became_peered":"2024-04-25T19:15:19.434508+0000","last_unstale":"2024-04-25T19:15:20.425687+0000","last_undegraded":"2024-04-25T19:15:20.425687+0000","last_fullsized":"2024-04-25T19:15:20.425687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:52.965864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404840+0000","last_change":"2024-04-25T19:15:18.428626+0000","last_active":"2024-04-25T19:15:20.404840+0000","last_peered":"2024-04-25T19:15:20.404840+0000","last_clean":"2024-04-25T19:15:20.404840+0000","last_became_active":"2024-04-25T19:15:18.428332+0000","last_became_peered":"2024-04-25T19:15:18.428332+0000","last_unstale":"2024-04-25T19:15:20.404840+0000","last_undegraded":"2024-04-25T19:15:20.404840+0000","last_fullsized":"2024-04-25T19:15:20.404840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:15.595305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848535+0000","last_change":"2024-04-25T19:15:19.428754+0000","last_active":"2024-04-25T19:15:20.848535+0000","last_peered":"2024-04-25T19:15:20.848535+0000","last_clean":"2024-04-25T19:15:20.848535+0000","last_became_active":"2024-04-25T19:15:19.427992+0000","last_became_peered":"2024-04-25T19:15:19.427992+0000","last_unstale":"2024-04-25T19:15:20.848535+0000","last_undegraded":"2024-04-25T19:15:20.848535+0000","last_fullsized":"2024-04-25T19:15:20.848535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:43:48.443923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847234+0000","last_change":"2024-04-25T19:15:18.419645+0000","last_active":"2024-04-25T19:15:20.847234+0000","last_peered":"2024-04-25T19:15:20.847234+0000","last_clean":"2024-04-25T19:15:20.847234+0000","last_became_active":"2024-04-25T19:15:18.419286+0000","last_became_peered":"2024-04-25T19:15:18.419286+0000","last_unstale":"2024-04-25T19:15:20.847234+0000","last_undegraded":"2024-04-25T19:15:20.847234+0000","last_fullsized":"2024-04-25T19:15:20.847234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:08.295208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425261+0000","last_change":"2024-04-25T19:15:19.432395+0000","last_active":"2024-04-25T19:15:20.425261+0000","last_peered":"2024-04-25T19:15:20.425261+0000","last_clean":"2024-04-25T19:15:20.425261+0000","last_became_active":"2024-04-25T19:15:19.431739+0000","last_became_peered":"2024-04-25T19:15:19.431739+0000","last_unstale":"2024-04-25T19:15:20.425261+0000","last_undegraded":"2024-04-25T19:15:20.425261+0000","last_fullsized":"2024-04-25T19:15:20.425261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:50:03.964030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":110,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425087+0000","last_change":"2024-04-25T19:15:15.604825+0000","last_active":"2024-04-25T19:15:20.425087+0000","last_peered":"2024-04-25T19:15:20.425087+0000","last_clean":"2024-04-25T19:15:20.425087+0000","last_became_active":"2024-04-25T19:15:15.603730+0000","last_became_peered":"2024-04-25T19:15:15.603730+0000","last_unstale":"2024-04-25T19:15:20.425087+0000","last_undegraded":"2024-04-25T19:15:20.425087+0000","last_fullsized":"2024-04-25T19:15:20.425087+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:14.367890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:14.367890+0000","last_clean_scrub_stamp":"2024-04-25T19:15:14.367890+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:02.411899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847915+0000","last_change":"2024-04-25T19:15:18.426331+0000","last_active":"2024-04-25T19:15:20.847915+0000","last_peered":"2024-04-25T19:15:20.847915+0000","last_clean":"2024-04-25T19:15:20.847915+0000","last_became_active":"2024-04-25T19:15:18.425993+0000","last_became_peered":"2024-04-25T19:15:18.425993+0000","last_unstale":"2024-04-25T19:15:20.847915+0000","last_undegraded":"2024-04-25T19:15:20.847915+0000","last_fullsized":"2024-04-25T19:15:20.847915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:17.440484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405021+0000","last_change":"2024-04-25T19:15:19.417337+0000","last_active":"2024-04-25T19:15:20.405021+0000","last_peered":"2024-04-25T19:15:20.405021+0000","last_clean":"2024-04-25T19:15:20.405021+0000","last_became_active":"2024-04-25T19:15:19.416808+0000","last_became_peered":"2024-04-25T19:15:19.416808+0000","last_unstale":"2024-04-25T19:15:20.405021+0000","last_undegraded":"2024-04-25T19:15:20.405021+0000","last_fullsized":"2024-04-25T19:15:20.405021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:57.797904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.424926+0000","last_change":"2024-04-25T19:15:18.426358+0000","last_active":"2024-04-25T19:15:20.424926+0000","last_peered":"2024-04-25T19:15:20.424926+0000","last_clean":"2024-04-25T19:15:20.424926+0000","last_became_active":"2024-04-25T19:15:18.426037+0000","last_became_peered":"2024-04-25T19:15:18.426037+0000","last_unstale":"2024-04-25T19:15:20.424926+0000","last_undegraded":"2024-04-25T19:15:20.424926+0000","last_fullsized":"2024-04-25T19:15:20.424926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:26:28.945667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804206+0000","last_change":"2024-04-25T19:15:19.426219+0000","last_active":"2024-04-25T19:15:20.804206+0000","last_peered":"2024-04-25T19:15:20.804206+0000","last_clean":"2024-04-25T19:15:20.804206+0000","last_became_active":"2024-04-25T19:15:19.425626+0000","last_became_peered":"2024-04-25T19:15:19.425626+0000","last_unstale":"2024-04-25T19:15:20.804206+0000","last_undegraded":"2024-04-25T19:15:20.804206+0000","last_fullsized":"2024-04-25T19:15:20.804206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:58.372454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847232+0000","last_change":"2024-04-25T19:15:18.424560+0000","last_active":"2024-04-25T19:15:20.847232+0000","last_peered":"2024-04-25T19:15:20.847232+0000","last_clean":"2024-04-25T19:15:20.847232+0000","last_became_active":"2024-04-25T19:15:18.424179+0000","last_became_peered":"2024-04-25T19:15:18.424179+0000","last_unstale":"2024-04-25T19:15:20.847232+0000","last_undegraded":"2024-04-25T19:15:20.847232+0000","last_fullsized":"2024-04-25T19:15:20.847232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:06.184399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404401+0000","last_change":"2024-04-25T19:15:19.415219+0000","last_active":"2024-04-25T19:15:20.404401+0000","last_peered":"2024-04-25T19:15:20.404401+0000","last_clean":"2024-04-25T19:15:20.404401+0000","last_became_active":"2024-04-25T19:15:19.414877+0000","last_became_peered":"2024-04-25T19:15:19.414877+0000","last_unstale":"2024-04-25T19:15:20.404401+0000","last_undegraded":"2024-04-25T19:15:20.404401+0000","last_fullsized":"2024-04-25T19:15:20.404401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:15.063419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848341+0000","last_change":"2024-04-25T19:15:18.423402+0000","last_active":"2024-04-25T19:15:20.848341+0000","last_peered":"2024-04-25T19:15:20.848341+0000","last_clean":"2024-04-25T19:15:20.848341+0000","last_became_active":"2024-04-25T19:15:18.422330+0000","last_became_peered":"2024-04-25T19:15:18.422330+0000","last_unstale":"2024-04-25T19:15:20.848341+0000","last_undegraded":"2024-04-25T19:15:20.848341+0000","last_fullsized":"2024-04-25T19:15:20.848341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:39:17.512502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822708+0000","last_change":"2024-04-25T19:15:19.421096+0000","last_active":"2024-04-25T19:15:20.822708+0000","last_peered":"2024-04-25T19:15:20.822708+0000","last_clean":"2024-04-25T19:15:20.822708+0000","last_became_active":"2024-04-25T19:15:19.420598+0000","last_became_peered":"2024-04-25T19:15:19.420598+0000","last_unstale":"2024-04-25T19:15:20.822708+0000","last_undegraded":"2024-04-25T19:15:20.822708+0000","last_fullsized":"2024-04-25T19:15:20.822708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:20.733888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404629+0000","last_change":"2024-04-25T19:15:18.422979+0000","last_active":"2024-04-25T19:15:20.404629+0000","last_peered":"2024-04-25T19:15:20.404629+0000","last_clean":"2024-04-25T19:15:20.404629+0000","last_became_active":"2024-04-25T19:15:18.422621+0000","last_became_peered":"2024-04-25T19:15:18.422621+0000","last_unstale":"2024-04-25T19:15:20.404629+0000","last_undegraded":"2024-04-25T19:15:20.404629+0000","last_fullsized":"2024-04-25T19:15:20.404629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:47:56.160615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848468+0000","last_change":"2024-04-25T19:15:19.426379+0000","last_active":"2024-04-25T19:15:20.848468+0000","last_peered":"2024-04-25T19:15:20.848468+0000","last_clean":"2024-04-25T19:15:20.848468+0000","last_became_active":"2024-04-25T19:15:19.425997+0000","last_became_peered":"2024-04-25T19:15:19.425997+0000","last_unstale":"2024-04-25T19:15:20.848468+0000","last_undegraded":"2024-04-25T19:15:20.848468+0000","last_fullsized":"2024-04-25T19:15:20.848468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:32.972274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425571+0000","last_change":"2024-04-25T19:15:18.427748+0000","last_active":"2024-04-25T19:15:20.425571+0000","last_peered":"2024-04-25T19:15:20.425571+0000","last_clean":"2024-04-25T19:15:20.425571+0000","last_became_active":"2024-04-25T19:15:18.427374+0000","last_became_peered":"2024-04-25T19:15:18.427374+0000","last_unstale":"2024-04-25T19:15:20.425571+0000","last_undegraded":"2024-04-25T19:15:20.425571+0000","last_fullsized":"2024-04-25T19:15:20.425571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:47:41.469558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404710+0000","last_change":"2024-04-25T19:15:19.422196+0000","last_active":"2024-04-25T19:15:20.404710+0000","last_peered":"2024-04-25T19:15:20.404710+0000","last_clean":"2024-04-25T19:15:20.404710+0000","last_became_active":"2024-04-25T19:15:19.421835+0000","last_became_peered":"2024-04-25T19:15:19.421835+0000","last_unstale":"2024-04-25T19:15:20.404710+0000","last_undegraded":"2024-04-25T19:15:20.404710+0000","last_fullsized":"2024-04-25T19:15:20.404710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:38.500369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404858+0000","last_change":"2024-04-25T19:15:18.419086+0000","last_active":"2024-04-25T19:15:20.404858+0000","last_peered":"2024-04-25T19:15:20.404858+0000","last_clean":"2024-04-25T19:15:20.404858+0000","last_became_active":"2024-04-25T19:15:18.418775+0000","last_became_peered":"2024-04-25T19:15:18.418775+0000","last_unstale":"2024-04-25T19:15:20.404858+0000","last_undegraded":"2024-04-25T19:15:20.404858+0000","last_fullsized":"2024-04-25T19:15:20.404858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:20.074753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405244+0000","last_change":"2024-04-25T19:15:19.432155+0000","last_active":"2024-04-25T19:15:20.405244+0000","last_peered":"2024-04-25T19:15:20.405244+0000","last_clean":"2024-04-25T19:15:20.405244+0000","last_became_active":"2024-04-25T19:15:19.431905+0000","last_became_peered":"2024-04-25T19:15:19.431905+0000","last_unstale":"2024-04-25T19:15:20.405244+0000","last_undegraded":"2024-04-25T19:15:20.405244+0000","last_fullsized":"2024-04-25T19:15:20.405244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:52.752469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:24.409991+0000","last_change":"2024-04-25T19:15:18.423144+0000","last_active":"2024-04-25T19:15:24.409991+0000","last_peered":"2024-04-25T19:15:24.409991+0000","last_clean":"2024-04-25T19:15:24.409991+0000","last_became_active":"2024-04-25T19:15:18.422838+0000","last_became_peered":"2024-04-25T19:15:18.422838+0000","last_unstale":"2024-04-25T19:15:24.409991+0000","last_undegraded":"2024-04-25T19:15:24.409991+0000","last_fullsized":"2024-04-25T19:15:24.409991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:28.026912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804520+0000","last_change":"2024-04-25T19:15:19.420582+0000","last_active":"2024-04-25T19:15:20.804520+0000","last_peered":"2024-04-25T19:15:20.804520+0000","last_clean":"2024-04-25T19:15:20.804520+0000","last_became_active":"2024-04-25T19:15:19.420270+0000","last_became_peered":"2024-04-25T19:15:19.420270+0000","last_unstale":"2024-04-25T19:15:20.804520+0000","last_undegraded":"2024-04-25T19:15:20.804520+0000","last_fullsized":"2024-04-25T19:15:20.804520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:06.827819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.424926+0000","last_change":"2024-04-25T19:15:18.425469+0000","last_active":"2024-04-25T19:15:20.424926+0000","last_peered":"2024-04-25T19:15:20.424926+0000","last_clean":"2024-04-25T19:15:20.424926+0000","last_became_active":"2024-04-25T19:15:18.425084+0000","last_became_peered":"2024-04-25T19:15:18.425084+0000","last_unstale":"2024-04-25T19:15:20.424926+0000","last_undegraded":"2024-04-25T19:15:20.424926+0000","last_fullsized":"2024-04-25T19:15:20.424926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:14.363158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822624+0000","last_change":"2024-04-25T19:15:19.425159+0000","last_active":"2024-04-25T19:15:20.822624+0000","last_peered":"2024-04-25T19:15:20.822624+0000","last_clean":"2024-04-25T19:15:20.822624+0000","last_became_active":"2024-04-25T19:15:19.424847+0000","last_became_peered":"2024-04-25T19:15:19.424847+0000","last_unstale":"2024-04-25T19:15:20.822624+0000","last_undegraded":"2024-04-25T19:15:20.822624+0000","last_fullsized":"2024-04-25T19:15:20.822624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:28.715286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404461+0000","last_change":"2024-04-25T19:15:18.424645+0000","last_active":"2024-04-25T19:15:20.404461+0000","last_peered":"2024-04-25T19:15:20.404461+0000","last_clean":"2024-04-25T19:15:20.404461+0000","last_became_active":"2024-04-25T19:15:18.424241+0000","last_became_peered":"2024-04-25T19:15:18.424241+0000","last_unstale":"2024-04-25T19:15:20.404461+0000","last_undegraded":"2024-04-25T19:15:20.404461+0000","last_fullsized":"2024-04-25T19:15:20.404461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:29.373363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405640+0000","last_change":"2024-04-25T19:15:19.421095+0000","last_active":"2024-04-25T19:15:20.405640+0000","last_peered":"2024-04-25T19:15:20.405640+0000","last_clean":"2024-04-25T19:15:20.405640+0000","last_became_active":"2024-04-25T19:15:19.420795+0000","last_became_peered":"2024-04-25T19:15:19.420795+0000","last_unstale":"2024-04-25T19:15:20.405640+0000","last_undegraded":"2024-04-25T19:15:20.405640+0000","last_fullsized":"2024-04-25T19:15:20.405640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:13.061506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404579+0000","last_change":"2024-04-25T19:15:18.423841+0000","last_active":"2024-04-25T19:15:20.404579+0000","last_peered":"2024-04-25T19:15:20.404579+0000","last_clean":"2024-04-25T19:15:20.404579+0000","last_became_active":"2024-04-25T19:15:18.423500+0000","last_became_peered":"2024-04-25T19:15:18.423500+0000","last_unstale":"2024-04-25T19:15:20.404579+0000","last_undegraded":"2024-04-25T19:15:20.404579+0000","last_fullsized":"2024-04-25T19:15:20.404579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:58.933085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405564+0000","last_change":"2024-04-25T19:15:19.416232+0000","last_active":"2024-04-25T19:15:20.405564+0000","last_peered":"2024-04-25T19:15:20.405564+0000","last_clean":"2024-04-25T19:15:20.405564+0000","last_became_active":"2024-04-25T19:15:19.415828+0000","last_became_peered":"2024-04-25T19:15:19.415828+0000","last_unstale":"2024-04-25T19:15:20.405564+0000","last_undegraded":"2024-04-25T19:15:20.405564+0000","last_fullsized":"2024-04-25T19:15:20.405564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:59.751840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:24.411180+0000","last_change":"2024-04-25T19:15:18.423918+0000","last_active":"2024-04-25T19:15:24.411180+0000","last_peered":"2024-04-25T19:15:24.411180+0000","last_clean":"2024-04-25T19:15:24.411180+0000","last_became_active":"2024-04-25T19:15:18.423588+0000","last_became_peered":"2024-04-25T19:15:18.423588+0000","last_unstale":"2024-04-25T19:15:24.411180+0000","last_undegraded":"2024-04-25T19:15:24.411180+0000","last_fullsized":"2024-04-25T19:15:24.411180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:16.378285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405402+0000","last_change":"2024-04-25T19:15:19.422984+0000","last_active":"2024-04-25T19:15:20.405402+0000","last_peered":"2024-04-25T19:15:20.405402+0000","last_clean":"2024-04-25T19:15:20.405402+0000","last_became_active":"2024-04-25T19:15:19.422651+0000","last_became_peered":"2024-04-25T19:15:19.422651+0000","last_unstale":"2024-04-25T19:15:20.405402+0000","last_undegraded":"2024-04-25T19:15:20.405402+0000","last_fullsized":"2024-04-25T19:15:20.405402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:54.984914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848417+0000","last_change":"2024-04-25T19:15:19.423982+0000","last_active":"2024-04-25T19:15:20.848417+0000","last_peered":"2024-04-25T19:15:20.848417+0000","last_clean":"2024-04-25T19:15:20.848417+0000","last_became_active":"2024-04-25T19:15:19.423341+0000","last_became_peered":"2024-04-25T19:15:19.423341+0000","last_unstale":"2024-04-25T19:15:20.848417+0000","last_undegraded":"2024-04-25T19:15:20.848417+0000","last_fullsized":"2024-04-25T19:15:20.848417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:51.808352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:24.410089+0000","last_change":"2024-04-25T19:15:18.424455+0000","last_active":"2024-04-25T19:15:24.410089+0000","last_peered":"2024-04-25T19:15:24.410089+0000","last_clean":"2024-04-25T19:15:24.410089+0000","last_became_active":"2024-04-25T19:15:18.424119+0000","last_became_peered":"2024-04-25T19:15:18.424119+0000","last_unstale":"2024-04-25T19:15:24.410089+0000","last_undegraded":"2024-04-25T19:15:24.410089+0000","last_fullsized":"2024-04-25T19:15:24.410089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:43.135567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848336+0000","last_change":"2024-04-25T19:15:19.424064+0000","last_active":"2024-04-25T19:15:20.848336+0000","last_peered":"2024-04-25T19:15:20.848336+0000","last_clean":"2024-04-25T19:15:20.848336+0000","last_became_active":"2024-04-25T19:15:19.423187+0000","last_became_peered":"2024-04-25T19:15:19.423187+0000","last_unstale":"2024-04-25T19:15:20.848336+0000","last_undegraded":"2024-04-25T19:15:20.848336+0000","last_fullsized":"2024-04-25T19:15:20.848336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:43:31.590700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:24.411456+0000","last_change":"2024-04-25T19:15:18.423151+0000","last_active":"2024-04-25T19:15:24.411456+0000","last_peered":"2024-04-25T19:15:24.411456+0000","last_clean":"2024-04-25T19:15:24.411456+0000","last_became_active":"2024-04-25T19:15:18.422831+0000","last_became_peered":"2024-04-25T19:15:18.422831+0000","last_unstale":"2024-04-25T19:15:24.411456+0000","last_undegraded":"2024-04-25T19:15:24.411456+0000","last_fullsized":"2024-04-25T19:15:24.411456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:43.244963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804534+0000","last_change":"2024-04-25T19:15:19.416717+0000","last_active":"2024-04-25T19:15:20.804534+0000","last_peered":"2024-04-25T19:15:20.804534+0000","last_clean":"2024-04-25T19:15:20.804534+0000","last_became_active":"2024-04-25T19:15:19.416163+0000","last_became_peered":"2024-04-25T19:15:19.416163+0000","last_unstale":"2024-04-25T19:15:20.804534+0000","last_undegraded":"2024-04-25T19:15:20.804534+0000","last_fullsized":"2024-04-25T19:15:20.804534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:46.955159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405410+0000","last_change":"2024-04-25T19:15:18.422875+0000","last_active":"2024-04-25T19:15:20.405410+0000","last_peered":"2024-04-25T19:15:20.405410+0000","last_clean":"2024-04-25T19:15:20.405410+0000","last_became_active":"2024-04-25T19:15:18.422489+0000","last_became_peered":"2024-04-25T19:15:18.422489+0000","last_unstale":"2024-04-25T19:15:20.405410+0000","last_undegraded":"2024-04-25T19:15:20.405410+0000","last_fullsized":"2024-04-25T19:15:20.405410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:30.602500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405477+0000","last_change":"2024-04-25T19:15:19.416987+0000","last_active":"2024-04-25T19:15:20.405477+0000","last_peered":"2024-04-25T19:15:20.405477+0000","last_clean":"2024-04-25T19:15:20.405477+0000","last_became_active":"2024-04-25T19:15:19.414173+0000","last_became_peered":"2024-04-25T19:15:19.414173+0000","last_unstale":"2024-04-25T19:15:20.405477+0000","last_undegraded":"2024-04-25T19:15:20.405477+0000","last_fullsized":"2024-04-25T19:15:20.405477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:32:43.188653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:24.414611+0000","last_change":"2024-04-25T19:15:18.423867+0000","last_active":"2024-04-25T19:15:24.414611+0000","last_peered":"2024-04-25T19:15:24.414611+0000","last_clean":"2024-04-25T19:15:24.414611+0000","last_became_active":"2024-04-25T19:15:18.423532+0000","last_became_peered":"2024-04-25T19:15:18.423532+0000","last_unstale":"2024-04-25T19:15:24.414611+0000","last_undegraded":"2024-04-25T19:15:24.414611+0000","last_fullsized":"2024-04-25T19:15:24.414611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:04.531338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64838,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":286720,"data_stored":666555,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":666611,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T19:15:29.956 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T19:15:30.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.098+0000 7f4d38f6c640 1 Processor -- start 2024-04-25T19:15:30.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.098+0000 7f4d38f6c640 1 -- start start 2024-04-25T19:15:30.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.100+0000 7f4d38f6c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4d3406de20 0x7f4d3406d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:30.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.100+0000 7f4d38f6c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d340cc9e0 0x7f4d34156f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:30.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.100+0000 7f4d38f6c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d3406e310 con 0x7f4d3415d780 2024-04-25T19:15:30.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.100+0000 7f4d38f6c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f4d34157450 con 0x7f4d3406de20 2024-04-25T19:15:30.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.100+0000 7f4d38f6c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f4d3415d2f0 con 0x7f4d340cc9e0 2024-04-25T19:15:30.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.100+0000 7f4d32d9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f4d3415d780 0x7f4d340cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36322/0 (socket says 172.21.15.76:36322) 2024-04-25T19:15:30.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.100+0000 7f4d32d9d640 1 -- 172.21.15.76:0/2113003994 learned_addr learned my addr 172.21.15.76:0/2113003994 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:30.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.100+0000 7f4d3259c640 1 --2- 172.21.15.76:0/2113003994 >> v2:172.21.15.105:3300/0 conn(0x7f4d3406de20 0x7f4d3406d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:30.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.100+0000 7f4d31d9b640 1 --2- 172.21.15.76:0/2113003994 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d340cc9e0 0x7f4d34156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:30.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.100+0000 7f4d3159a640 1 -- 172.21.15.76:0/2113003994 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1803949740 0 0) 0x7f4d3406e310 con 0x7f4d3415d780 2024-04-25T19:15:30.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.100+0000 7f4d3159a640 1 -- 172.21.15.76:0/2113003994 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d1c003520 con 0x7f4d3415d780 2024-04-25T19:15:30.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.101+0000 7f4d3159a640 1 -- 172.21.15.76:0/2113003994 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1359319226 0 0) 0x7f4d1c003520 con 0x7f4d3415d780 2024-04-25T19:15:30.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.101+0000 7f4d3159a640 1 -- 172.21.15.76:0/2113003994 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d340cc9e0 msgr2=0x7f4d34156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:30.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.101+0000 7f4d3159a640 1 --2- 172.21.15.76:0/2113003994 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d340cc9e0 0x7f4d34156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:30.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.101+0000 7f4d3159a640 1 -- 172.21.15.76:0/2113003994 >> v2:172.21.15.105:3300/0 conn(0x7f4d3406de20 msgr2=0x7f4d3406d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:30.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.101+0000 7f4d3159a640 1 --2- 172.21.15.76:0/2113003994 >> v2:172.21.15.105:3300/0 conn(0x7f4d3406de20 0x7f4d3406d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:30.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.101+0000 7f4d3159a640 1 -- 172.21.15.76:0/2113003994 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d3415ccc0 con 0x7f4d3415d780 2024-04-25T19:15:30.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.101+0000 7f4d3159a640 1 -- 172.21.15.76:0/2113003994 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f4d2c001670 con 0x7f4d3415d780 2024-04-25T19:15:30.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.101+0000 7f4d3159a640 1 -- 172.21.15.76:0/2113003994 >> v1:172.21.15.76:6789/0 conn(0x7f4d3415d780 legacy=0x7f4d340cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:30.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.101+0000 7f4d3159a640 1 --2- 172.21.15.76:0/2113003994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d1c0039b0 0x7f4d1c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:30.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.101+0000 7f4d3159a640 1 --2- 172.21.15.76:0/2113003994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d1c004800 0x7f4d1c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:30.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.101+0000 7f4d3159a640 1 --2- 172.21.15.76:0/2113003994 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d340cc9e0 0x7f4d1c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:30.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.101+0000 7f4d3159a640 1 -- 172.21.15.76:0/2113003994 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4d34157450 con 0x7f4d1c004800 2024-04-25T19:15:30.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.101+0000 7f4d3159a640 1 -- 172.21.15.76:0/2113003994 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3415d2f0 con 0x7f4d1c0039b0 2024-04-25T19:15:30.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.101+0000 7f4d3159a640 1 -- 172.21.15.76:0/2113003994 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f4d1c00ba70 con 0x7f4d340cc9e0 2024-04-25T19:15:30.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.101+0000 7f4d31d9b640 1 --2- 172.21.15.76:0/2113003994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d1c004800 0x7f4d1c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:30.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.101+0000 7f4d3259c640 1 --2- 172.21.15.76:0/2113003994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d1c0039b0 0x7f4d1c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:30.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.101+0000 7f4d32d9d640 1 --2- 172.21.15.76:0/2113003994 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d340cc9e0 0x7f4d1c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:30.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.101+0000 7f4d31d9b640 1 -- 172.21.15.76:0/2113003994 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d340cc9e0 msgr2=0x7f4d1c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:30.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.101+0000 7f4d31d9b640 1 --2- 172.21.15.76:0/2113003994 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d340cc9e0 0x7f4d1c005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:30.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.101+0000 7f4d31d9b640 1 -- 172.21.15.76:0/2113003994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d1c0039b0 msgr2=0x7f4d1c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:30.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.101+0000 7f4d31d9b640 1 --2- 172.21.15.76:0/2113003994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d1c0039b0 0x7f4d1c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:30.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.101+0000 7f4d31d9b640 1 -- 172.21.15.76:0/2113003994 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1c003520 con 0x7f4d1c004800 2024-04-25T19:15:30.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.101+0000 7f4d3259c640 1 --2- 172.21.15.76:0/2113003994 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d1c0039b0 0x7f4d1c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:30.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.102+0000 7f4d31d9b640 1 --2- 172.21.15.76:0/2113003994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d1c004800 0x7f4d1c004c10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009ac0 tx=0x7f4d1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec938401df34f362 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:30.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.102+0000 7f4d3159a640 1 -- 172.21.15.76:0/2113003994 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28036070 con 0x7f4d1c004800 2024-04-25T19:15:30.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.102+0000 7f4d3159a640 1 -- 172.21.15.76:0/2113003994 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d28031400 con 0x7f4d1c004800 2024-04-25T19:15:30.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.102+0000 7f4d3159a640 1 -- 172.21.15.76:0/2113003994 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28009690 con 0x7f4d1c004800 2024-04-25T19:15:30.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.103+0000 7f4d38f6c640 1 -- 172.21.15.76:0/2113003994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d1c004800 msgr2=0x7f4d1c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:30.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.103+0000 7f4d38f6c640 1 --2- 172.21.15.76:0/2113003994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d1c004800 0x7f4d1c004c10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009ac0 tx=0x7f4d1c005830 comp rx=0 tx=0).stop 2024-04-25T19:15:30.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.103+0000 7f4d32d9d640 1 -- 172.21.15.76:0/2113003994 reap_dead start 2024-04-25T19:15:30.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.103+0000 7f4d38f6c640 1 -- 172.21.15.76:0/2113003994 shutdown_connections 2024-04-25T19:15:30.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.103+0000 7f4d38f6c640 1 -- 172.21.15.76:0/2113003994 >> 172.21.15.76:0/2113003994 conn(0x7f4d340684e0 msgr2=0x7f4d340688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:30.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.103+0000 7f4d38f6c640 1 -- 172.21.15.76:0/2113003994 shutdown_connections 2024-04-25T19:15:30.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.103+0000 7f4d38f6c640 1 -- 172.21.15.76:0/2113003994 wait complete. 2024-04-25T19:15:30.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.105+0000 7f4d38f6c640 1 Processor -- start 2024-04-25T19:15:30.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.105+0000 7f4d38f6c640 1 -- start start 2024-04-25T19:15:30.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.106+0000 7f4d38f6c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d1c004800 0x7f4d3415f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:30.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.106+0000 7f4d38f6c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d3415a140 0x7f4d3415aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:30.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.106+0000 7f4d38f6c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d341588d0 0x7f4d34158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:30.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.106+0000 7f4d38f6c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3415cf80 con 0x7f4d1c004800 2024-04-25T19:15:30.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.106+0000 7f4d38f6c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4d340cb980 con 0x7f4d341588d0 2024-04-25T19:15:30.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.106+0000 7f4d38f6c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f4d340cbda0 con 0x7f4d3415a140 2024-04-25T19:15:30.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.106+0000 7f4d3259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d1c004800 0x7f4d3415f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:30.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.106+0000 7f4d3259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d1c004800 0x7f4d3415f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58890/0 (socket says 172.21.15.76:58890) 2024-04-25T19:15:30.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.106+0000 7f4d3259c640 1 -- 172.21.15.76:0/803381181 learned_addr learned my addr 172.21.15.76:0/803381181 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:30.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.106+0000 7f4d32d9d640 1 --2- 172.21.15.76:0/803381181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d341588d0 0x7f4d34158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:30.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.106+0000 7f4d31d9b640 1 --2- 172.21.15.76:0/803381181 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d3415a140 0x7f4d3415aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:30.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.106+0000 7f4d3259c640 1 -- 172.21.15.76:0/803381181 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d3415a140 msgr2=0x7f4d3415aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:30.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.106+0000 7f4d3259c640 1 --2- 172.21.15.76:0/803381181 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d3415a140 0x7f4d3415aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:30.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.106+0000 7f4d3259c640 1 -- 172.21.15.76:0/803381181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d341588d0 msgr2=0x7f4d34158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:30.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.106+0000 7f4d3259c640 1 --2- 172.21.15.76:0/803381181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d341588d0 0x7f4d34158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:30.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.106+0000 7f4d3259c640 1 -- 172.21.15.76:0/803381181 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d3415f700 con 0x7f4d1c004800 2024-04-25T19:15:30.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.106+0000 7f4d3259c640 1 --2- 172.21.15.76:0/803381181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d1c004800 0x7f4d3415f1c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400f460 tx=0x7f4d2400f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:30.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.106+0000 7f4d22ffd640 1 -- 172.21.15.76:0/803381181 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d24003b90 con 0x7f4d1c004800 2024-04-25T19:15:30.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.106+0000 7f4d22ffd640 1 -- 172.21.15.76:0/803381181 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d24003d30 con 0x7f4d1c004800 2024-04-25T19:15:30.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.106+0000 7f4d38f6c640 1 -- 172.21.15.76:0/803381181 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d341f5ea0 con 0x7f4d1c004800 2024-04-25T19:15:30.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.106+0000 7f4d22ffd640 1 -- 172.21.15.76:0/803381181 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d24004240 con 0x7f4d1c004800 2024-04-25T19:15:30.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.107+0000 7f4d38f6c640 1 -- 172.21.15.76:0/803381181 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d341f6460 con 0x7f4d1c004800 2024-04-25T19:15:30.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.108+0000 7f4d38f6c640 1 -- 172.21.15.76:0/803381181 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d3406a390 con 0x7f4d1c004800 2024-04-25T19:15:30.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.108+0000 7f4d22ffd640 1 -- 172.21.15.76:0/803381181 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4d240043e0 con 0x7f4d1c004800 2024-04-25T19:15:30.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.109+0000 7f4d22ffd640 1 --2- 172.21.15.76:0/803381181 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4d140744e0 0x7f4d140769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:30.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.109+0000 7f4d22ffd640 1 -- 172.21.15.76:0/803381181 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4d24096310 con 0x7f4d1c004800 2024-04-25T19:15:30.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.109+0000 7f4d31d9b640 1 --2- 172.21.15.76:0/803381181 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4d140744e0 0x7f4d140769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:30.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.109+0000 7f4d31d9b640 1 --2- 172.21.15.76:0/803381181 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4d140744e0 0x7f4d140769f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009ac0 tx=0x7f4d28008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:30.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.113+0000 7f4d22ffd640 1 -- 172.21.15.76:0/803381181 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4d24019030 con 0x7f4d1c004800 2024-04-25T19:15:30.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.287+0000 7f4d38f6c640 1 -- 172.21.15.76:0/803381181 --> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4d340cc0b0 con 0x7f4d140744e0 2024-04-25T19:15:30.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.293+0000 7f4d22ffd640 1 -- 172.21.15.76:0/803381181 <== mgr.4109 v2:172.21.15.105:6832/2585775531 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251902 (secure 0 0 0) 0x7f4d340cc0b0 con 0x7f4d140744e0 2024-04-25T19:15:30.297 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:30.298 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-04-25T19:15:30.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.296+0000 7f4d38f6c640 1 -- 172.21.15.76:0/803381181 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4d140744e0 msgr2=0x7f4d140769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:30.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.296+0000 7f4d38f6c640 1 --2- 172.21.15.76:0/803381181 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4d140744e0 0x7f4d140769f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009ac0 tx=0x7f4d28008030 comp rx=0 tx=0).stop 2024-04-25T19:15:30.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.296+0000 7f4d38f6c640 1 -- 172.21.15.76:0/803381181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d1c004800 msgr2=0x7f4d3415f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:30.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.296+0000 7f4d38f6c640 1 --2- 172.21.15.76:0/803381181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d1c004800 0x7f4d3415f1c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400f460 tx=0x7f4d2400f920 comp rx=0 tx=0).stop 2024-04-25T19:15:30.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.296+0000 7f4d38f6c640 1 -- 172.21.15.76:0/803381181 shutdown_connections 2024-04-25T19:15:30.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.296+0000 7f4d38f6c640 1 --2- 172.21.15.76:0/803381181 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4d140744e0 0x7f4d140769f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:30.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.296+0000 7f4d38f6c640 1 --2- 172.21.15.76:0/803381181 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d341588d0 0x7f4d34158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:30.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.296+0000 7f4d38f6c640 1 --2- 172.21.15.76:0/803381181 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4d3415a140 0x7f4d3415aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:30.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.296+0000 7f4d38f6c640 1 --2- 172.21.15.76:0/803381181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d1c004800 0x7f4d3415f1c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:30.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.296+0000 7f4d38f6c640 1 -- 172.21.15.76:0/803381181 >> 172.21.15.76:0/803381181 conn(0x7f4d340684e0 msgr2=0x7f4d341557b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:30.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.296+0000 7f4d38f6c640 1 -- 172.21.15.76:0/803381181 shutdown_connections 2024-04-25T19:15:30.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.296+0000 7f4d38f6c640 1 -- 172.21.15.76:0/803381181 wait complete. 2024-04-25T19:15:30.309 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-25T19:15:29.831096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218016,"kb_used_data":1888,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756704,"statfs":{"total":773094113280,"available":772870864896,"internally_reserved":0,"allocated":1933312,"data_stored":1790396,"data_compressed":64838,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002587"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425589+0000","last_change":"2024-04-25T19:15:19.433569+0000","last_active":"2024-04-25T19:15:20.425589+0000","last_peered":"2024-04-25T19:15:20.425589+0000","last_clean":"2024-04-25T19:15:20.425589+0000","last_became_active":"2024-04-25T19:15:19.431332+0000","last_became_peered":"2024-04-25T19:15:19.431332+0000","last_unstale":"2024-04-25T19:15:20.425589+0000","last_undegraded":"2024-04-25T19:15:20.425589+0000","last_fullsized":"2024-04-25T19:15:20.425589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:57:01.213834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404926+0000","last_change":"2024-04-25T19:15:19.418695+0000","last_active":"2024-04-25T19:15:20.404926+0000","last_peered":"2024-04-25T19:15:20.404926+0000","last_clean":"2024-04-25T19:15:20.404926+0000","last_became_active":"2024-04-25T19:15:19.418235+0000","last_became_peered":"2024-04-25T19:15:19.418235+0000","last_unstale":"2024-04-25T19:15:20.404926+0000","last_undegraded":"2024-04-25T19:15:20.404926+0000","last_fullsized":"2024-04-25T19:15:20.404926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:30.516472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804491+0000","last_change":"2024-04-25T19:15:19.416983+0000","last_active":"2024-04-25T19:15:20.804491+0000","last_peered":"2024-04-25T19:15:20.804491+0000","last_clean":"2024-04-25T19:15:20.804491+0000","last_became_active":"2024-04-25T19:15:19.416443+0000","last_became_peered":"2024-04-25T19:15:19.416443+0000","last_unstale":"2024-04-25T19:15:20.804491+0000","last_undegraded":"2024-04-25T19:15:20.804491+0000","last_fullsized":"2024-04-25T19:15:20.804491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:02.794164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848308+0000","last_change":"2024-04-25T19:15:19.429058+0000","last_active":"2024-04-25T19:15:20.848308+0000","last_peered":"2024-04-25T19:15:20.848308+0000","last_clean":"2024-04-25T19:15:20.848308+0000","last_became_active":"2024-04-25T19:15:19.428419+0000","last_became_peered":"2024-04-25T19:15:19.428419+0000","last_unstale":"2024-04-25T19:15:20.848308+0000","last_undegraded":"2024-04-25T19:15:20.848308+0000","last_fullsized":"2024-04-25T19:15:20.848308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:23:56.817616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425402+0000","last_change":"2024-04-25T19:15:19.427615+0000","last_active":"2024-04-25T19:15:20.425402+0000","last_peered":"2024-04-25T19:15:20.425402+0000","last_clean":"2024-04-25T19:15:20.425402+0000","last_became_active":"2024-04-25T19:15:19.426968+0000","last_became_peered":"2024-04-25T19:15:19.426968+0000","last_unstale":"2024-04-25T19:15:20.425402+0000","last_undegraded":"2024-04-25T19:15:20.425402+0000","last_fullsized":"2024-04-25T19:15:20.425402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:42:00.686712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847989+0000","last_change":"2024-04-25T19:15:19.417602+0000","last_active":"2024-04-25T19:15:20.847989+0000","last_peered":"2024-04-25T19:15:20.847989+0000","last_clean":"2024-04-25T19:15:20.847989+0000","last_became_active":"2024-04-25T19:15:19.417292+0000","last_became_peered":"2024-04-25T19:15:19.417292+0000","last_unstale":"2024-04-25T19:15:20.847989+0000","last_undegraded":"2024-04-25T19:15:20.847989+0000","last_fullsized":"2024-04-25T19:15:20.847989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:37:43.583809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425114+0000","last_change":"2024-04-25T19:15:19.428180+0000","last_active":"2024-04-25T19:15:20.425114+0000","last_peered":"2024-04-25T19:15:20.425114+0000","last_clean":"2024-04-25T19:15:20.425114+0000","last_became_active":"2024-04-25T19:15:19.427852+0000","last_became_peered":"2024-04-25T19:15:19.427852+0000","last_unstale":"2024-04-25T19:15:20.425114+0000","last_undegraded":"2024-04-25T19:15:20.425114+0000","last_fullsized":"2024-04-25T19:15:20.425114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:51.300880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405446+0000","last_change":"2024-04-25T19:15:19.418253+0000","last_active":"2024-04-25T19:15:20.405446+0000","last_peered":"2024-04-25T19:15:20.405446+0000","last_clean":"2024-04-25T19:15:20.405446+0000","last_became_active":"2024-04-25T19:15:19.417183+0000","last_became_peered":"2024-04-25T19:15:19.417183+0000","last_unstale":"2024-04-25T19:15:20.405446+0000","last_undegraded":"2024-04-25T19:15:20.405446+0000","last_fullsized":"2024-04-25T19:15:20.405446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:05:13.181646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425578+0000","last_change":"2024-04-25T19:15:19.431671+0000","last_active":"2024-04-25T19:15:20.425578+0000","last_peered":"2024-04-25T19:15:20.425578+0000","last_clean":"2024-04-25T19:15:20.425578+0000","last_became_active":"2024-04-25T19:15:19.431240+0000","last_became_peered":"2024-04-25T19:15:19.431240+0000","last_unstale":"2024-04-25T19:15:20.425578+0000","last_undegraded":"2024-04-25T19:15:20.425578+0000","last_fullsized":"2024-04-25T19:15:20.425578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:22.759462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.406080+0000","last_change":"2024-04-25T19:15:19.416304+0000","last_active":"2024-04-25T19:15:20.406080+0000","last_peered":"2024-04-25T19:15:20.406080+0000","last_clean":"2024-04-25T19:15:20.406080+0000","last_became_active":"2024-04-25T19:15:19.415967+0000","last_became_peered":"2024-04-25T19:15:19.415967+0000","last_unstale":"2024-04-25T19:15:20.406080+0000","last_undegraded":"2024-04-25T19:15:20.406080+0000","last_fullsized":"2024-04-25T19:15:20.406080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:30:52.958731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405773+0000","last_change":"2024-04-25T19:15:19.417596+0000","last_active":"2024-04-25T19:15:20.405773+0000","last_peered":"2024-04-25T19:15:20.405773+0000","last_clean":"2024-04-25T19:15:20.405773+0000","last_became_active":"2024-04-25T19:15:19.417230+0000","last_became_peered":"2024-04-25T19:15:19.417230+0000","last_unstale":"2024-04-25T19:15:20.405773+0000","last_undegraded":"2024-04-25T19:15:20.405773+0000","last_fullsized":"2024-04-25T19:15:20.405773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:28:12.387651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822843+0000","last_change":"2024-04-25T19:15:19.420564+0000","last_active":"2024-04-25T19:15:20.822843+0000","last_peered":"2024-04-25T19:15:20.822843+0000","last_clean":"2024-04-25T19:15:20.822843+0000","last_became_active":"2024-04-25T19:15:19.420044+0000","last_became_peered":"2024-04-25T19:15:19.420044+0000","last_unstale":"2024-04-25T19:15:20.822843+0000","last_undegraded":"2024-04-25T19:15:20.822843+0000","last_fullsized":"2024-04-25T19:15:20.822843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:04.066030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405269+0000","last_change":"2024-04-25T19:15:19.415500+0000","last_active":"2024-04-25T19:15:20.405269+0000","last_peered":"2024-04-25T19:15:20.405269+0000","last_clean":"2024-04-25T19:15:20.405269+0000","last_became_active":"2024-04-25T19:15:19.414007+0000","last_became_peered":"2024-04-25T19:15:19.414007+0000","last_unstale":"2024-04-25T19:15:20.405269+0000","last_undegraded":"2024-04-25T19:15:20.405269+0000","last_fullsized":"2024-04-25T19:15:20.405269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:56:12.320555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804235+0000","last_change":"2024-04-25T19:15:19.423884+0000","last_active":"2024-04-25T19:15:20.804235+0000","last_peered":"2024-04-25T19:15:20.804235+0000","last_clean":"2024-04-25T19:15:20.804235+0000","last_became_active":"2024-04-25T19:15:19.422575+0000","last_became_peered":"2024-04-25T19:15:19.422575+0000","last_unstale":"2024-04-25T19:15:20.804235+0000","last_undegraded":"2024-04-25T19:15:20.804235+0000","last_fullsized":"2024-04-25T19:15:20.804235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:30.988950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404396+0000","last_change":"2024-04-25T19:15:19.424267+0000","last_active":"2024-04-25T19:15:20.404396+0000","last_peered":"2024-04-25T19:15:20.404396+0000","last_clean":"2024-04-25T19:15:20.404396+0000","last_became_active":"2024-04-25T19:15:19.422953+0000","last_became_peered":"2024-04-25T19:15:19.422953+0000","last_unstale":"2024-04-25T19:15:20.404396+0000","last_undegraded":"2024-04-25T19:15:20.404396+0000","last_fullsized":"2024-04-25T19:15:20.404396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:42.823233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847983+0000","last_change":"2024-04-25T19:15:19.420374+0000","last_active":"2024-04-25T19:15:20.847983+0000","last_peered":"2024-04-25T19:15:20.847983+0000","last_clean":"2024-04-25T19:15:20.847983+0000","last_became_active":"2024-04-25T19:15:19.420015+0000","last_became_peered":"2024-04-25T19:15:19.420015+0000","last_unstale":"2024-04-25T19:15:20.847983+0000","last_undegraded":"2024-04-25T19:15:20.847983+0000","last_fullsized":"2024-04-25T19:15:20.847983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:04:26.719359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848705+0000","last_change":"2024-04-25T19:15:19.434476+0000","last_active":"2024-04-25T19:15:20.848705+0000","last_peered":"2024-04-25T19:15:20.848705+0000","last_clean":"2024-04-25T19:15:20.848705+0000","last_became_active":"2024-04-25T19:15:19.433836+0000","last_became_peered":"2024-04-25T19:15:19.433836+0000","last_unstale":"2024-04-25T19:15:20.848705+0000","last_undegraded":"2024-04-25T19:15:20.848705+0000","last_fullsized":"2024-04-25T19:15:20.848705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:13.234829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847359+0000","last_change":"2024-04-25T19:15:19.415521+0000","last_active":"2024-04-25T19:15:20.847359+0000","last_peered":"2024-04-25T19:15:20.847359+0000","last_clean":"2024-04-25T19:15:20.847359+0000","last_became_active":"2024-04-25T19:15:19.415177+0000","last_became_peered":"2024-04-25T19:15:19.415177+0000","last_unstale":"2024-04-25T19:15:20.847359+0000","last_undegraded":"2024-04-25T19:15:20.847359+0000","last_fullsized":"2024-04-25T19:15:20.847359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:24.355619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425590+0000","last_change":"2024-04-25T19:15:19.430291+0000","last_active":"2024-04-25T19:15:20.425590+0000","last_peered":"2024-04-25T19:15:20.425590+0000","last_clean":"2024-04-25T19:15:20.425590+0000","last_became_active":"2024-04-25T19:15:19.427226+0000","last_became_peered":"2024-04-25T19:15:19.427226+0000","last_unstale":"2024-04-25T19:15:20.425590+0000","last_undegraded":"2024-04-25T19:15:20.425590+0000","last_fullsized":"2024-04-25T19:15:20.425590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:38.464211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405750+0000","last_change":"2024-04-25T19:15:19.421696+0000","last_active":"2024-04-25T19:15:20.405750+0000","last_peered":"2024-04-25T19:15:20.405750+0000","last_clean":"2024-04-25T19:15:20.405750+0000","last_became_active":"2024-04-25T19:15:19.421300+0000","last_became_peered":"2024-04-25T19:15:19.421300+0000","last_unstale":"2024-04-25T19:15:20.405750+0000","last_undegraded":"2024-04-25T19:15:20.405750+0000","last_fullsized":"2024-04-25T19:15:20.405750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:15.226398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804410+0000","last_change":"2024-04-25T19:15:19.427880+0000","last_active":"2024-04-25T19:15:20.804410+0000","last_peered":"2024-04-25T19:15:20.804410+0000","last_clean":"2024-04-25T19:15:20.804410+0000","last_became_active":"2024-04-25T19:15:19.427579+0000","last_became_peered":"2024-04-25T19:15:19.427579+0000","last_unstale":"2024-04-25T19:15:20.804410+0000","last_undegraded":"2024-04-25T19:15:20.804410+0000","last_fullsized":"2024-04-25T19:15:20.804410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:47:10.657715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848145+0000","last_change":"2024-04-25T19:15:19.430409+0000","last_active":"2024-04-25T19:15:20.848145+0000","last_peered":"2024-04-25T19:15:20.848145+0000","last_clean":"2024-04-25T19:15:20.848145+0000","last_became_active":"2024-04-25T19:15:19.428305+0000","last_became_peered":"2024-04-25T19:15:19.428305+0000","last_unstale":"2024-04-25T19:15:20.848145+0000","last_undegraded":"2024-04-25T19:15:20.848145+0000","last_fullsized":"2024-04-25T19:15:20.848145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:10:01.268963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405396+0000","last_change":"2024-04-25T19:15:19.424435+0000","last_active":"2024-04-25T19:15:20.405396+0000","last_peered":"2024-04-25T19:15:20.405396+0000","last_clean":"2024-04-25T19:15:20.405396+0000","last_became_active":"2024-04-25T19:15:19.422763+0000","last_became_peered":"2024-04-25T19:15:19.422763+0000","last_unstale":"2024-04-25T19:15:20.405396+0000","last_undegraded":"2024-04-25T19:15:20.405396+0000","last_fullsized":"2024-04-25T19:15:20.405396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:58.092412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848072+0000","last_change":"2024-04-25T19:15:19.420275+0000","last_active":"2024-04-25T19:15:20.848072+0000","last_peered":"2024-04-25T19:15:20.848072+0000","last_clean":"2024-04-25T19:15:20.848072+0000","last_became_active":"2024-04-25T19:15:19.419936+0000","last_became_peered":"2024-04-25T19:15:19.419936+0000","last_unstale":"2024-04-25T19:15:20.848072+0000","last_undegraded":"2024-04-25T19:15:20.848072+0000","last_fullsized":"2024-04-25T19:15:20.848072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:55.710620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405087+0000","last_change":"2024-04-25T19:15:19.423408+0000","last_active":"2024-04-25T19:15:20.405087+0000","last_peered":"2024-04-25T19:15:20.405087+0000","last_clean":"2024-04-25T19:15:20.405087+0000","last_became_active":"2024-04-25T19:15:19.423116+0000","last_became_peered":"2024-04-25T19:15:19.423116+0000","last_unstale":"2024-04-25T19:15:20.405087+0000","last_undegraded":"2024-04-25T19:15:20.405087+0000","last_fullsized":"2024-04-25T19:15:20.405087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:18:57.787013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.406015+0000","last_change":"2024-04-25T19:15:19.428826+0000","last_active":"2024-04-25T19:15:20.406015+0000","last_peered":"2024-04-25T19:15:20.406015+0000","last_clean":"2024-04-25T19:15:20.406015+0000","last_became_active":"2024-04-25T19:15:19.428474+0000","last_became_peered":"2024-04-25T19:15:19.428474+0000","last_unstale":"2024-04-25T19:15:20.406015+0000","last_undegraded":"2024-04-25T19:15:20.406015+0000","last_fullsized":"2024-04-25T19:15:20.406015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:19.974851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404739+0000","last_change":"2024-04-25T19:15:19.423353+0000","last_active":"2024-04-25T19:15:20.404739+0000","last_peered":"2024-04-25T19:15:20.404739+0000","last_clean":"2024-04-25T19:15:20.404739+0000","last_became_active":"2024-04-25T19:15:19.422795+0000","last_became_peered":"2024-04-25T19:15:19.422795+0000","last_unstale":"2024-04-25T19:15:20.404739+0000","last_undegraded":"2024-04-25T19:15:20.404739+0000","last_fullsized":"2024-04-25T19:15:20.404739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:34.119976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405076+0000","last_change":"2024-04-25T19:15:19.421809+0000","last_active":"2024-04-25T19:15:20.405076+0000","last_peered":"2024-04-25T19:15:20.405076+0000","last_clean":"2024-04-25T19:15:20.405076+0000","last_became_active":"2024-04-25T19:15:19.421515+0000","last_became_peered":"2024-04-25T19:15:19.421515+0000","last_unstale":"2024-04-25T19:15:20.405076+0000","last_undegraded":"2024-04-25T19:15:20.405076+0000","last_fullsized":"2024-04-25T19:15:20.405076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:22.603291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804477+0000","last_change":"2024-04-25T19:15:19.410183+0000","last_active":"2024-04-25T19:15:20.804477+0000","last_peered":"2024-04-25T19:15:20.804477+0000","last_clean":"2024-04-25T19:15:20.804477+0000","last_became_active":"2024-04-25T19:15:19.409857+0000","last_became_peered":"2024-04-25T19:15:19.409857+0000","last_unstale":"2024-04-25T19:15:20.804477+0000","last_undegraded":"2024-04-25T19:15:20.804477+0000","last_fullsized":"2024-04-25T19:15:20.804477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:30.180786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848239+0000","last_change":"2024-04-25T19:15:19.425524+0000","last_active":"2024-04-25T19:15:20.848239+0000","last_peered":"2024-04-25T19:15:20.848239+0000","last_clean":"2024-04-25T19:15:20.848239+0000","last_became_active":"2024-04-25T19:15:19.424578+0000","last_became_peered":"2024-04-25T19:15:19.424578+0000","last_unstale":"2024-04-25T19:15:20.848239+0000","last_undegraded":"2024-04-25T19:15:20.848239+0000","last_fullsized":"2024-04-25T19:15:20.848239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:02.103879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847243+0000","last_change":"2024-04-25T19:15:19.426720+0000","last_active":"2024-04-25T19:15:20.847243+0000","last_peered":"2024-04-25T19:15:20.847243+0000","last_clean":"2024-04-25T19:15:20.847243+0000","last_became_active":"2024-04-25T19:15:19.426139+0000","last_became_peered":"2024-04-25T19:15:19.426139+0000","last_unstale":"2024-04-25T19:15:20.847243+0000","last_undegraded":"2024-04-25T19:15:20.847243+0000","last_fullsized":"2024-04-25T19:15:20.847243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:31.608993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405390+0000","last_change":"2024-04-25T19:15:19.413805+0000","last_active":"2024-04-25T19:15:20.405390+0000","last_peered":"2024-04-25T19:15:20.405390+0000","last_clean":"2024-04-25T19:15:20.405390+0000","last_became_active":"2024-04-25T19:15:19.413455+0000","last_became_peered":"2024-04-25T19:15:19.413455+0000","last_unstale":"2024-04-25T19:15:20.405390+0000","last_undegraded":"2024-04-25T19:15:20.405390+0000","last_fullsized":"2024-04-25T19:15:20.405390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:05:45.008076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822709+0000","last_change":"2024-04-25T19:15:19.423770+0000","last_active":"2024-04-25T19:15:20.822709+0000","last_peered":"2024-04-25T19:15:20.822709+0000","last_clean":"2024-04-25T19:15:20.822709+0000","last_became_active":"2024-04-25T19:15:19.423406+0000","last_became_peered":"2024-04-25T19:15:19.423406+0000","last_unstale":"2024-04-25T19:15:20.822709+0000","last_undegraded":"2024-04-25T19:15:20.822709+0000","last_fullsized":"2024-04-25T19:15:20.822709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:41:23.669832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405392+0000","last_change":"2024-04-25T19:15:18.422035+0000","last_active":"2024-04-25T19:15:20.405392+0000","last_peered":"2024-04-25T19:15:20.405392+0000","last_clean":"2024-04-25T19:15:20.405392+0000","last_became_active":"2024-04-25T19:15:18.421658+0000","last_became_peered":"2024-04-25T19:15:18.421658+0000","last_unstale":"2024-04-25T19:15:20.405392+0000","last_undegraded":"2024-04-25T19:15:20.405392+0000","last_fullsized":"2024-04-25T19:15:20.405392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:23.349126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804571+0000","last_change":"2024-04-25T19:15:19.421805+0000","last_active":"2024-04-25T19:15:20.804571+0000","last_peered":"2024-04-25T19:15:20.804571+0000","last_clean":"2024-04-25T19:15:20.804571+0000","last_became_active":"2024-04-25T19:15:19.421497+0000","last_became_peered":"2024-04-25T19:15:19.421497+0000","last_unstale":"2024-04-25T19:15:20.804571+0000","last_undegraded":"2024-04-25T19:15:20.804571+0000","last_fullsized":"2024-04-25T19:15:20.804571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:29:29.949508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405561+0000","last_change":"2024-04-25T19:15:18.416894+0000","last_active":"2024-04-25T19:15:20.405561+0000","last_peered":"2024-04-25T19:15:20.405561+0000","last_clean":"2024-04-25T19:15:20.405561+0000","last_became_active":"2024-04-25T19:15:18.416539+0000","last_became_peered":"2024-04-25T19:15:18.416539+0000","last_unstale":"2024-04-25T19:15:20.405561+0000","last_undegraded":"2024-04-25T19:15:20.405561+0000","last_fullsized":"2024-04-25T19:15:20.405561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:30.388606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405329+0000","last_change":"2024-04-25T19:15:18.420129+0000","last_active":"2024-04-25T19:15:20.405329+0000","last_peered":"2024-04-25T19:15:20.405329+0000","last_clean":"2024-04-25T19:15:20.405329+0000","last_became_active":"2024-04-25T19:15:18.419732+0000","last_became_peered":"2024-04-25T19:15:18.419732+0000","last_unstale":"2024-04-25T19:15:20.405329+0000","last_undegraded":"2024-04-25T19:15:20.405329+0000","last_fullsized":"2024-04-25T19:15:20.405329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:41:48.673989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405615+0000","last_change":"2024-04-25T19:15:19.415655+0000","last_active":"2024-04-25T19:15:20.405615+0000","last_peered":"2024-04-25T19:15:20.405615+0000","last_clean":"2024-04-25T19:15:20.405615+0000","last_became_active":"2024-04-25T19:15:19.415287+0000","last_became_peered":"2024-04-25T19:15:19.415287+0000","last_unstale":"2024-04-25T19:15:20.405615+0000","last_undegraded":"2024-04-25T19:15:20.405615+0000","last_fullsized":"2024-04-25T19:15:20.405615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:49.155289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848076+0000","last_change":"2024-04-25T19:15:18.420711+0000","last_active":"2024-04-25T19:15:20.848076+0000","last_peered":"2024-04-25T19:15:20.848076+0000","last_clean":"2024-04-25T19:15:20.848076+0000","last_became_active":"2024-04-25T19:15:18.420362+0000","last_became_peered":"2024-04-25T19:15:18.420362+0000","last_unstale":"2024-04-25T19:15:20.848076+0000","last_undegraded":"2024-04-25T19:15:20.848076+0000","last_fullsized":"2024-04-25T19:15:20.848076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:48.245193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404375+0000","last_change":"2024-04-25T19:15:19.424356+0000","last_active":"2024-04-25T19:15:20.404375+0000","last_peered":"2024-04-25T19:15:20.404375+0000","last_clean":"2024-04-25T19:15:20.404375+0000","last_became_active":"2024-04-25T19:15:19.423556+0000","last_became_peered":"2024-04-25T19:15:19.423556+0000","last_unstale":"2024-04-25T19:15:20.404375+0000","last_undegraded":"2024-04-25T19:15:20.404375+0000","last_fullsized":"2024-04-25T19:15:20.404375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:19:35.427261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405514+0000","last_change":"2024-04-25T19:15:18.419250+0000","last_active":"2024-04-25T19:15:20.405514+0000","last_peered":"2024-04-25T19:15:20.405514+0000","last_clean":"2024-04-25T19:15:20.405514+0000","last_became_active":"2024-04-25T19:15:18.418927+0000","last_became_peered":"2024-04-25T19:15:18.418927+0000","last_unstale":"2024-04-25T19:15:20.405514+0000","last_undegraded":"2024-04-25T19:15:20.405514+0000","last_fullsized":"2024-04-25T19:15:20.405514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:26.012965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848606+0000","last_change":"2024-04-25T19:15:19.430117+0000","last_active":"2024-04-25T19:15:20.848606+0000","last_peered":"2024-04-25T19:15:20.848606+0000","last_clean":"2024-04-25T19:15:20.848606+0000","last_became_active":"2024-04-25T19:15:19.428985+0000","last_became_peered":"2024-04-25T19:15:19.428985+0000","last_unstale":"2024-04-25T19:15:20.848606+0000","last_undegraded":"2024-04-25T19:15:20.848606+0000","last_fullsized":"2024-04-25T19:15:20.848606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:36:29.069541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848490+0000","last_change":"2024-04-25T19:15:18.422931+0000","last_active":"2024-04-25T19:15:20.848490+0000","last_peered":"2024-04-25T19:15:20.848490+0000","last_clean":"2024-04-25T19:15:20.848490+0000","last_became_active":"2024-04-25T19:15:18.422610+0000","last_became_peered":"2024-04-25T19:15:18.422610+0000","last_unstale":"2024-04-25T19:15:20.848490+0000","last_undegraded":"2024-04-25T19:15:20.848490+0000","last_fullsized":"2024-04-25T19:15:20.848490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:18.489484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804598+0000","last_change":"2024-04-25T19:15:19.416109+0000","last_active":"2024-04-25T19:15:20.804598+0000","last_peered":"2024-04-25T19:15:20.804598+0000","last_clean":"2024-04-25T19:15:20.804598+0000","last_became_active":"2024-04-25T19:15:19.415734+0000","last_became_peered":"2024-04-25T19:15:19.415734+0000","last_unstale":"2024-04-25T19:15:20.804598+0000","last_undegraded":"2024-04-25T19:15:20.804598+0000","last_fullsized":"2024-04-25T19:15:20.804598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:57:31.406303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:24.411207+0000","last_change":"2024-04-25T19:15:18.428677+0000","last_active":"2024-04-25T19:15:24.411207+0000","last_peered":"2024-04-25T19:15:24.411207+0000","last_clean":"2024-04-25T19:15:24.411207+0000","last_became_active":"2024-04-25T19:15:18.428432+0000","last_became_peered":"2024-04-25T19:15:18.428432+0000","last_unstale":"2024-04-25T19:15:24.411207+0000","last_undegraded":"2024-04-25T19:15:24.411207+0000","last_fullsized":"2024-04-25T19:15:24.411207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:38.478595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822814+0000","last_change":"2024-04-25T19:15:19.428245+0000","last_active":"2024-04-25T19:15:20.822814+0000","last_peered":"2024-04-25T19:15:20.822814+0000","last_clean":"2024-04-25T19:15:20.822814+0000","last_became_active":"2024-04-25T19:15:19.427941+0000","last_became_peered":"2024-04-25T19:15:19.427941+0000","last_unstale":"2024-04-25T19:15:20.822814+0000","last_undegraded":"2024-04-25T19:15:20.822814+0000","last_fullsized":"2024-04-25T19:15:20.822814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:09.947923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:24.414669+0000","last_change":"2024-04-25T19:15:18.420881+0000","last_active":"2024-04-25T19:15:24.414669+0000","last_peered":"2024-04-25T19:15:24.414669+0000","last_clean":"2024-04-25T19:15:24.414669+0000","last_became_active":"2024-04-25T19:15:18.420303+0000","last_became_peered":"2024-04-25T19:15:18.420303+0000","last_unstale":"2024-04-25T19:15:24.414669+0000","last_undegraded":"2024-04-25T19:15:24.414669+0000","last_fullsized":"2024-04-25T19:15:24.414669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:39:24.356298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425356+0000","last_change":"2024-04-25T19:15:19.432735+0000","last_active":"2024-04-25T19:15:20.425356+0000","last_peered":"2024-04-25T19:15:20.425356+0000","last_clean":"2024-04-25T19:15:20.425356+0000","last_became_active":"2024-04-25T19:15:19.431828+0000","last_became_peered":"2024-04-25T19:15:19.431828+0000","last_unstale":"2024-04-25T19:15:20.425356+0000","last_undegraded":"2024-04-25T19:15:20.425356+0000","last_fullsized":"2024-04-25T19:15:20.425356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:20.189230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:24.414315+0000","last_change":"2024-04-25T19:15:18.421008+0000","last_active":"2024-04-25T19:15:24.414315+0000","last_peered":"2024-04-25T19:15:24.414315+0000","last_clean":"2024-04-25T19:15:24.414315+0000","last_became_active":"2024-04-25T19:15:18.420402+0000","last_became_peered":"2024-04-25T19:15:18.420402+0000","last_unstale":"2024-04-25T19:15:24.414315+0000","last_undegraded":"2024-04-25T19:15:24.414315+0000","last_fullsized":"2024-04-25T19:15:24.414315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:05:28.723958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804304+0000","last_change":"2024-04-25T19:15:19.424805+0000","last_active":"2024-04-25T19:15:20.804304+0000","last_peered":"2024-04-25T19:15:20.804304+0000","last_clean":"2024-04-25T19:15:20.804304+0000","last_became_active":"2024-04-25T19:15:19.424512+0000","last_became_peered":"2024-04-25T19:15:19.424512+0000","last_unstale":"2024-04-25T19:15:20.804304+0000","last_undegraded":"2024-04-25T19:15:20.804304+0000","last_fullsized":"2024-04-25T19:15:20.804304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:50.423186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848213+0000","last_change":"2024-04-25T19:15:18.426805+0000","last_active":"2024-04-25T19:15:20.848213+0000","last_peered":"2024-04-25T19:15:20.848213+0000","last_clean":"2024-04-25T19:15:20.848213+0000","last_became_active":"2024-04-25T19:15:18.426472+0000","last_became_peered":"2024-04-25T19:15:18.426472+0000","last_unstale":"2024-04-25T19:15:20.848213+0000","last_undegraded":"2024-04-25T19:15:20.848213+0000","last_fullsized":"2024-04-25T19:15:20.848213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:29.373363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425459+0000","last_change":"2024-04-25T19:15:19.428780+0000","last_active":"2024-04-25T19:15:20.425459+0000","last_peered":"2024-04-25T19:15:20.425459+0000","last_clean":"2024-04-25T19:15:20.425459+0000","last_became_active":"2024-04-25T19:15:19.428450+0000","last_became_peered":"2024-04-25T19:15:19.428450+0000","last_unstale":"2024-04-25T19:15:20.425459+0000","last_undegraded":"2024-04-25T19:15:20.425459+0000","last_fullsized":"2024-04-25T19:15:20.425459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:01.203880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405399+0000","last_change":"2024-04-25T19:15:18.417740+0000","last_active":"2024-04-25T19:15:20.405399+0000","last_peered":"2024-04-25T19:15:20.405399+0000","last_clean":"2024-04-25T19:15:20.405399+0000","last_became_active":"2024-04-25T19:15:18.417404+0000","last_became_peered":"2024-04-25T19:15:18.417404+0000","last_unstale":"2024-04-25T19:15:20.405399+0000","last_undegraded":"2024-04-25T19:15:20.405399+0000","last_fullsized":"2024-04-25T19:15:20.405399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:21.737283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405391+0000","last_change":"2024-04-25T19:15:19.411214+0000","last_active":"2024-04-25T19:15:20.405391+0000","last_peered":"2024-04-25T19:15:20.405391+0000","last_clean":"2024-04-25T19:15:20.405391+0000","last_became_active":"2024-04-25T19:15:19.410814+0000","last_became_peered":"2024-04-25T19:15:19.410814+0000","last_unstale":"2024-04-25T19:15:20.405391+0000","last_undegraded":"2024-04-25T19:15:20.405391+0000","last_fullsized":"2024-04-25T19:15:20.405391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:42.430277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:24.417443+0000","last_change":"2024-04-25T19:15:18.422226+0000","last_active":"2024-04-25T19:15:24.417443+0000","last_peered":"2024-04-25T19:15:24.417443+0000","last_clean":"2024-04-25T19:15:24.417443+0000","last_became_active":"2024-04-25T19:15:18.421900+0000","last_became_peered":"2024-04-25T19:15:18.421900+0000","last_unstale":"2024-04-25T19:15:24.417443+0000","last_undegraded":"2024-04-25T19:15:24.417443+0000","last_fullsized":"2024-04-25T19:15:24.417443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:18.040114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425347+0000","last_change":"2024-04-25T19:15:19.428885+0000","last_active":"2024-04-25T19:15:20.425347+0000","last_peered":"2024-04-25T19:15:20.425347+0000","last_clean":"2024-04-25T19:15:20.425347+0000","last_became_active":"2024-04-25T19:15:19.428563+0000","last_became_peered":"2024-04-25T19:15:19.428563+0000","last_unstale":"2024-04-25T19:15:20.425347+0000","last_undegraded":"2024-04-25T19:15:20.425347+0000","last_fullsized":"2024-04-25T19:15:20.425347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:43.469189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822831+0000","last_change":"2024-04-25T19:15:18.422125+0000","last_active":"2024-04-25T19:15:20.822831+0000","last_peered":"2024-04-25T19:15:20.822831+0000","last_clean":"2024-04-25T19:15:20.822831+0000","last_became_active":"2024-04-25T19:15:18.421777+0000","last_became_peered":"2024-04-25T19:15:18.421777+0000","last_unstale":"2024-04-25T19:15:20.822831+0000","last_undegraded":"2024-04-25T19:15:20.822831+0000","last_fullsized":"2024-04-25T19:15:20.822831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:24.289010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425489+0000","last_change":"2024-04-25T19:15:19.429675+0000","last_active":"2024-04-25T19:15:20.425489+0000","last_peered":"2024-04-25T19:15:20.425489+0000","last_clean":"2024-04-25T19:15:20.425489+0000","last_became_active":"2024-04-25T19:15:19.429316+0000","last_became_peered":"2024-04-25T19:15:19.429316+0000","last_unstale":"2024-04-25T19:15:20.425489+0000","last_undegraded":"2024-04-25T19:15:20.425489+0000","last_fullsized":"2024-04-25T19:15:20.425489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:09:55.521944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:24.410093+0000","last_change":"2024-04-25T19:15:18.418661+0000","last_active":"2024-04-25T19:15:24.410093+0000","last_peered":"2024-04-25T19:15:24.410093+0000","last_clean":"2024-04-25T19:15:24.410093+0000","last_became_active":"2024-04-25T19:15:18.418359+0000","last_became_peered":"2024-04-25T19:15:18.418359+0000","last_unstale":"2024-04-25T19:15:24.410093+0000","last_undegraded":"2024-04-25T19:15:24.410093+0000","last_fullsized":"2024-04-25T19:15:24.410093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:49:22.023652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425687+0000","last_change":"2024-04-25T19:15:19.434853+0000","last_active":"2024-04-25T19:15:20.425687+0000","last_peered":"2024-04-25T19:15:20.425687+0000","last_clean":"2024-04-25T19:15:20.425687+0000","last_became_active":"2024-04-25T19:15:19.434508+0000","last_became_peered":"2024-04-25T19:15:19.434508+0000","last_unstale":"2024-04-25T19:15:20.425687+0000","last_undegraded":"2024-04-25T19:15:20.425687+0000","last_fullsized":"2024-04-25T19:15:20.425687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:52.965864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404840+0000","last_change":"2024-04-25T19:15:18.428626+0000","last_active":"2024-04-25T19:15:20.404840+0000","last_peered":"2024-04-25T19:15:20.404840+0000","last_clean":"2024-04-25T19:15:20.404840+0000","last_became_active":"2024-04-25T19:15:18.428332+0000","last_became_peered":"2024-04-25T19:15:18.428332+0000","last_unstale":"2024-04-25T19:15:20.404840+0000","last_undegraded":"2024-04-25T19:15:20.404840+0000","last_fullsized":"2024-04-25T19:15:20.404840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:15.595305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848535+0000","last_change":"2024-04-25T19:15:19.428754+0000","last_active":"2024-04-25T19:15:20.848535+0000","last_peered":"2024-04-25T19:15:20.848535+0000","last_clean":"2024-04-25T19:15:20.848535+0000","last_became_active":"2024-04-25T19:15:19.427992+0000","last_became_peered":"2024-04-25T19:15:19.427992+0000","last_unstale":"2024-04-25T19:15:20.848535+0000","last_undegraded":"2024-04-25T19:15:20.848535+0000","last_fullsized":"2024-04-25T19:15:20.848535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:43:48.443923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847234+0000","last_change":"2024-04-25T19:15:18.419645+0000","last_active":"2024-04-25T19:15:20.847234+0000","last_peered":"2024-04-25T19:15:20.847234+0000","last_clean":"2024-04-25T19:15:20.847234+0000","last_became_active":"2024-04-25T19:15:18.419286+0000","last_became_peered":"2024-04-25T19:15:18.419286+0000","last_unstale":"2024-04-25T19:15:20.847234+0000","last_undegraded":"2024-04-25T19:15:20.847234+0000","last_fullsized":"2024-04-25T19:15:20.847234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:08.295208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425261+0000","last_change":"2024-04-25T19:15:19.432395+0000","last_active":"2024-04-25T19:15:20.425261+0000","last_peered":"2024-04-25T19:15:20.425261+0000","last_clean":"2024-04-25T19:15:20.425261+0000","last_became_active":"2024-04-25T19:15:19.431739+0000","last_became_peered":"2024-04-25T19:15:19.431739+0000","last_unstale":"2024-04-25T19:15:20.425261+0000","last_undegraded":"2024-04-25T19:15:20.425261+0000","last_fullsized":"2024-04-25T19:15:20.425261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:50:03.964030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":110,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425087+0000","last_change":"2024-04-25T19:15:15.604825+0000","last_active":"2024-04-25T19:15:20.425087+0000","last_peered":"2024-04-25T19:15:20.425087+0000","last_clean":"2024-04-25T19:15:20.425087+0000","last_became_active":"2024-04-25T19:15:15.603730+0000","last_became_peered":"2024-04-25T19:15:15.603730+0000","last_unstale":"2024-04-25T19:15:20.425087+0000","last_undegraded":"2024-04-25T19:15:20.425087+0000","last_fullsized":"2024-04-25T19:15:20.425087+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:14.367890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:14.367890+0000","last_clean_scrub_stamp":"2024-04-25T19:15:14.367890+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:02.411899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847915+0000","last_change":"2024-04-25T19:15:18.426331+0000","last_active":"2024-04-25T19:15:20.847915+0000","last_peered":"2024-04-25T19:15:20.847915+0000","last_clean":"2024-04-25T19:15:20.847915+0000","last_became_active":"2024-04-25T19:15:18.425993+0000","last_became_peered":"2024-04-25T19:15:18.425993+0000","last_unstale":"2024-04-25T19:15:20.847915+0000","last_undegraded":"2024-04-25T19:15:20.847915+0000","last_fullsized":"2024-04-25T19:15:20.847915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:17.440484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405021+0000","last_change":"2024-04-25T19:15:19.417337+0000","last_active":"2024-04-25T19:15:20.405021+0000","last_peered":"2024-04-25T19:15:20.405021+0000","last_clean":"2024-04-25T19:15:20.405021+0000","last_became_active":"2024-04-25T19:15:19.416808+0000","last_became_peered":"2024-04-25T19:15:19.416808+0000","last_unstale":"2024-04-25T19:15:20.405021+0000","last_undegraded":"2024-04-25T19:15:20.405021+0000","last_fullsized":"2024-04-25T19:15:20.405021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:57.797904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.424926+0000","last_change":"2024-04-25T19:15:18.426358+0000","last_active":"2024-04-25T19:15:20.424926+0000","last_peered":"2024-04-25T19:15:20.424926+0000","last_clean":"2024-04-25T19:15:20.424926+0000","last_became_active":"2024-04-25T19:15:18.426037+0000","last_became_peered":"2024-04-25T19:15:18.426037+0000","last_unstale":"2024-04-25T19:15:20.424926+0000","last_undegraded":"2024-04-25T19:15:20.424926+0000","last_fullsized":"2024-04-25T19:15:20.424926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:26:28.945667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804206+0000","last_change":"2024-04-25T19:15:19.426219+0000","last_active":"2024-04-25T19:15:20.804206+0000","last_peered":"2024-04-25T19:15:20.804206+0000","last_clean":"2024-04-25T19:15:20.804206+0000","last_became_active":"2024-04-25T19:15:19.425626+0000","last_became_peered":"2024-04-25T19:15:19.425626+0000","last_unstale":"2024-04-25T19:15:20.804206+0000","last_undegraded":"2024-04-25T19:15:20.804206+0000","last_fullsized":"2024-04-25T19:15:20.804206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:58.372454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847232+0000","last_change":"2024-04-25T19:15:18.424560+0000","last_active":"2024-04-25T19:15:20.847232+0000","last_peered":"2024-04-25T19:15:20.847232+0000","last_clean":"2024-04-25T19:15:20.847232+0000","last_became_active":"2024-04-25T19:15:18.424179+0000","last_became_peered":"2024-04-25T19:15:18.424179+0000","last_unstale":"2024-04-25T19:15:20.847232+0000","last_undegraded":"2024-04-25T19:15:20.847232+0000","last_fullsized":"2024-04-25T19:15:20.847232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:06.184399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404401+0000","last_change":"2024-04-25T19:15:19.415219+0000","last_active":"2024-04-25T19:15:20.404401+0000","last_peered":"2024-04-25T19:15:20.404401+0000","last_clean":"2024-04-25T19:15:20.404401+0000","last_became_active":"2024-04-25T19:15:19.414877+0000","last_became_peered":"2024-04-25T19:15:19.414877+0000","last_unstale":"2024-04-25T19:15:20.404401+0000","last_undegraded":"2024-04-25T19:15:20.404401+0000","last_fullsized":"2024-04-25T19:15:20.404401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:15.063419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848341+0000","last_change":"2024-04-25T19:15:18.423402+0000","last_active":"2024-04-25T19:15:20.848341+0000","last_peered":"2024-04-25T19:15:20.848341+0000","last_clean":"2024-04-25T19:15:20.848341+0000","last_became_active":"2024-04-25T19:15:18.422330+0000","last_became_peered":"2024-04-25T19:15:18.422330+0000","last_unstale":"2024-04-25T19:15:20.848341+0000","last_undegraded":"2024-04-25T19:15:20.848341+0000","last_fullsized":"2024-04-25T19:15:20.848341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:39:17.512502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822708+0000","last_change":"2024-04-25T19:15:19.421096+0000","last_active":"2024-04-25T19:15:20.822708+0000","last_peered":"2024-04-25T19:15:20.822708+0000","last_clean":"2024-04-25T19:15:20.822708+0000","last_became_active":"2024-04-25T19:15:19.420598+0000","last_became_peered":"2024-04-25T19:15:19.420598+0000","last_unstale":"2024-04-25T19:15:20.822708+0000","last_undegraded":"2024-04-25T19:15:20.822708+0000","last_fullsized":"2024-04-25T19:15:20.822708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:20.733888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404629+0000","last_change":"2024-04-25T19:15:18.422979+0000","last_active":"2024-04-25T19:15:20.404629+0000","last_peered":"2024-04-25T19:15:20.404629+0000","last_clean":"2024-04-25T19:15:20.404629+0000","last_became_active":"2024-04-25T19:15:18.422621+0000","last_became_peered":"2024-04-25T19:15:18.422621+0000","last_unstale":"2024-04-25T19:15:20.404629+0000","last_undegraded":"2024-04-25T19:15:20.404629+0000","last_fullsized":"2024-04-25T19:15:20.404629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:47:56.160615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848468+0000","last_change":"2024-04-25T19:15:19.426379+0000","last_active":"2024-04-25T19:15:20.848468+0000","last_peered":"2024-04-25T19:15:20.848468+0000","last_clean":"2024-04-25T19:15:20.848468+0000","last_became_active":"2024-04-25T19:15:19.425997+0000","last_became_peered":"2024-04-25T19:15:19.425997+0000","last_unstale":"2024-04-25T19:15:20.848468+0000","last_undegraded":"2024-04-25T19:15:20.848468+0000","last_fullsized":"2024-04-25T19:15:20.848468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:32.972274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425571+0000","last_change":"2024-04-25T19:15:18.427748+0000","last_active":"2024-04-25T19:15:20.425571+0000","last_peered":"2024-04-25T19:15:20.425571+0000","last_clean":"2024-04-25T19:15:20.425571+0000","last_became_active":"2024-04-25T19:15:18.427374+0000","last_became_peered":"2024-04-25T19:15:18.427374+0000","last_unstale":"2024-04-25T19:15:20.425571+0000","last_undegraded":"2024-04-25T19:15:20.425571+0000","last_fullsized":"2024-04-25T19:15:20.425571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:47:41.469558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404710+0000","last_change":"2024-04-25T19:15:19.422196+0000","last_active":"2024-04-25T19:15:20.404710+0000","last_peered":"2024-04-25T19:15:20.404710+0000","last_clean":"2024-04-25T19:15:20.404710+0000","last_became_active":"2024-04-25T19:15:19.421835+0000","last_became_peered":"2024-04-25T19:15:19.421835+0000","last_unstale":"2024-04-25T19:15:20.404710+0000","last_undegraded":"2024-04-25T19:15:20.404710+0000","last_fullsized":"2024-04-25T19:15:20.404710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:38.500369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404858+0000","last_change":"2024-04-25T19:15:18.419086+0000","last_active":"2024-04-25T19:15:20.404858+0000","last_peered":"2024-04-25T19:15:20.404858+0000","last_clean":"2024-04-25T19:15:20.404858+0000","last_became_active":"2024-04-25T19:15:18.418775+0000","last_became_peered":"2024-04-25T19:15:18.418775+0000","last_unstale":"2024-04-25T19:15:20.404858+0000","last_undegraded":"2024-04-25T19:15:20.404858+0000","last_fullsized":"2024-04-25T19:15:20.404858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:20.074753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405244+0000","last_change":"2024-04-25T19:15:19.432155+0000","last_active":"2024-04-25T19:15:20.405244+0000","last_peered":"2024-04-25T19:15:20.405244+0000","last_clean":"2024-04-25T19:15:20.405244+0000","last_became_active":"2024-04-25T19:15:19.431905+0000","last_became_peered":"2024-04-25T19:15:19.431905+0000","last_unstale":"2024-04-25T19:15:20.405244+0000","last_undegraded":"2024-04-25T19:15:20.405244+0000","last_fullsized":"2024-04-25T19:15:20.405244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:52.752469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:24.409991+0000","last_change":"2024-04-25T19:15:18.423144+0000","last_active":"2024-04-25T19:15:24.409991+0000","last_peered":"2024-04-25T19:15:24.409991+0000","last_clean":"2024-04-25T19:15:24.409991+0000","last_became_active":"2024-04-25T19:15:18.422838+0000","last_became_peered":"2024-04-25T19:15:18.422838+0000","last_unstale":"2024-04-25T19:15:24.409991+0000","last_undegraded":"2024-04-25T19:15:24.409991+0000","last_fullsized":"2024-04-25T19:15:24.409991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:28.026912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804520+0000","last_change":"2024-04-25T19:15:19.420582+0000","last_active":"2024-04-25T19:15:20.804520+0000","last_peered":"2024-04-25T19:15:20.804520+0000","last_clean":"2024-04-25T19:15:20.804520+0000","last_became_active":"2024-04-25T19:15:19.420270+0000","last_became_peered":"2024-04-25T19:15:19.420270+0000","last_unstale":"2024-04-25T19:15:20.804520+0000","last_undegraded":"2024-04-25T19:15:20.804520+0000","last_fullsized":"2024-04-25T19:15:20.804520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:06.827819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.424926+0000","last_change":"2024-04-25T19:15:18.425469+0000","last_active":"2024-04-25T19:15:20.424926+0000","last_peered":"2024-04-25T19:15:20.424926+0000","last_clean":"2024-04-25T19:15:20.424926+0000","last_became_active":"2024-04-25T19:15:18.425084+0000","last_became_peered":"2024-04-25T19:15:18.425084+0000","last_unstale":"2024-04-25T19:15:20.424926+0000","last_undegraded":"2024-04-25T19:15:20.424926+0000","last_fullsized":"2024-04-25T19:15:20.424926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:14.363158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822624+0000","last_change":"2024-04-25T19:15:19.425159+0000","last_active":"2024-04-25T19:15:20.822624+0000","last_peered":"2024-04-25T19:15:20.822624+0000","last_clean":"2024-04-25T19:15:20.822624+0000","last_became_active":"2024-04-25T19:15:19.424847+0000","last_became_peered":"2024-04-25T19:15:19.424847+0000","last_unstale":"2024-04-25T19:15:20.822624+0000","last_undegraded":"2024-04-25T19:15:20.822624+0000","last_fullsized":"2024-04-25T19:15:20.822624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:28.715286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404461+0000","last_change":"2024-04-25T19:15:18.424645+0000","last_active":"2024-04-25T19:15:20.404461+0000","last_peered":"2024-04-25T19:15:20.404461+0000","last_clean":"2024-04-25T19:15:20.404461+0000","last_became_active":"2024-04-25T19:15:18.424241+0000","last_became_peered":"2024-04-25T19:15:18.424241+0000","last_unstale":"2024-04-25T19:15:20.404461+0000","last_undegraded":"2024-04-25T19:15:20.404461+0000","last_fullsized":"2024-04-25T19:15:20.404461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:29.373363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405640+0000","last_change":"2024-04-25T19:15:19.421095+0000","last_active":"2024-04-25T19:15:20.405640+0000","last_peered":"2024-04-25T19:15:20.405640+0000","last_clean":"2024-04-25T19:15:20.405640+0000","last_became_active":"2024-04-25T19:15:19.420795+0000","last_became_peered":"2024-04-25T19:15:19.420795+0000","last_unstale":"2024-04-25T19:15:20.405640+0000","last_undegraded":"2024-04-25T19:15:20.405640+0000","last_fullsized":"2024-04-25T19:15:20.405640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:13.061506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404579+0000","last_change":"2024-04-25T19:15:18.423841+0000","last_active":"2024-04-25T19:15:20.404579+0000","last_peered":"2024-04-25T19:15:20.404579+0000","last_clean":"2024-04-25T19:15:20.404579+0000","last_became_active":"2024-04-25T19:15:18.423500+0000","last_became_peered":"2024-04-25T19:15:18.423500+0000","last_unstale":"2024-04-25T19:15:20.404579+0000","last_undegraded":"2024-04-25T19:15:20.404579+0000","last_fullsized":"2024-04-25T19:15:20.404579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:58.933085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405564+0000","last_change":"2024-04-25T19:15:19.416232+0000","last_active":"2024-04-25T19:15:20.405564+0000","last_peered":"2024-04-25T19:15:20.405564+0000","last_clean":"2024-04-25T19:15:20.405564+0000","last_became_active":"2024-04-25T19:15:19.415828+0000","last_became_peered":"2024-04-25T19:15:19.415828+0000","last_unstale":"2024-04-25T19:15:20.405564+0000","last_undegraded":"2024-04-25T19:15:20.405564+0000","last_fullsized":"2024-04-25T19:15:20.405564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:59.751840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:24.411180+0000","last_change":"2024-04-25T19:15:18.423918+0000","last_active":"2024-04-25T19:15:24.411180+0000","last_peered":"2024-04-25T19:15:24.411180+0000","last_clean":"2024-04-25T19:15:24.411180+0000","last_became_active":"2024-04-25T19:15:18.423588+0000","last_became_peered":"2024-04-25T19:15:18.423588+0000","last_unstale":"2024-04-25T19:15:24.411180+0000","last_undegraded":"2024-04-25T19:15:24.411180+0000","last_fullsized":"2024-04-25T19:15:24.411180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:16.378285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405402+0000","last_change":"2024-04-25T19:15:19.422984+0000","last_active":"2024-04-25T19:15:20.405402+0000","last_peered":"2024-04-25T19:15:20.405402+0000","last_clean":"2024-04-25T19:15:20.405402+0000","last_became_active":"2024-04-25T19:15:19.422651+0000","last_became_peered":"2024-04-25T19:15:19.422651+0000","last_unstale":"2024-04-25T19:15:20.405402+0000","last_undegraded":"2024-04-25T19:15:20.405402+0000","last_fullsized":"2024-04-25T19:15:20.405402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:54.984914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848417+0000","last_change":"2024-04-25T19:15:19.423982+0000","last_active":"2024-04-25T19:15:20.848417+0000","last_peered":"2024-04-25T19:15:20.848417+0000","last_clean":"2024-04-25T19:15:20.848417+0000","last_became_active":"2024-04-25T19:15:19.423341+0000","last_became_peered":"2024-04-25T19:15:19.423341+0000","last_unstale":"2024-04-25T19:15:20.848417+0000","last_undegraded":"2024-04-25T19:15:20.848417+0000","last_fullsized":"2024-04-25T19:15:20.848417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:51.808352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:24.410089+0000","last_change":"2024-04-25T19:15:18.424455+0000","last_active":"2024-04-25T19:15:24.410089+0000","last_peered":"2024-04-25T19:15:24.410089+0000","last_clean":"2024-04-25T19:15:24.410089+0000","last_became_active":"2024-04-25T19:15:18.424119+0000","last_became_peered":"2024-04-25T19:15:18.424119+0000","last_unstale":"2024-04-25T19:15:24.410089+0000","last_undegraded":"2024-04-25T19:15:24.410089+0000","last_fullsized":"2024-04-25T19:15:24.410089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:43.135567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848336+0000","last_change":"2024-04-25T19:15:19.424064+0000","last_active":"2024-04-25T19:15:20.848336+0000","last_peered":"2024-04-25T19:15:20.848336+0000","last_clean":"2024-04-25T19:15:20.848336+0000","last_became_active":"2024-04-25T19:15:19.423187+0000","last_became_peered":"2024-04-25T19:15:19.423187+0000","last_unstale":"2024-04-25T19:15:20.848336+0000","last_undegraded":"2024-04-25T19:15:20.848336+0000","last_fullsized":"2024-04-25T19:15:20.848336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:43:31.590700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:24.411456+0000","last_change":"2024-04-25T19:15:18.423151+0000","last_active":"2024-04-25T19:15:24.411456+0000","last_peered":"2024-04-25T19:15:24.411456+0000","last_clean":"2024-04-25T19:15:24.411456+0000","last_became_active":"2024-04-25T19:15:18.422831+0000","last_became_peered":"2024-04-25T19:15:18.422831+0000","last_unstale":"2024-04-25T19:15:24.411456+0000","last_undegraded":"2024-04-25T19:15:24.411456+0000","last_fullsized":"2024-04-25T19:15:24.411456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:43.244963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804534+0000","last_change":"2024-04-25T19:15:19.416717+0000","last_active":"2024-04-25T19:15:20.804534+0000","last_peered":"2024-04-25T19:15:20.804534+0000","last_clean":"2024-04-25T19:15:20.804534+0000","last_became_active":"2024-04-25T19:15:19.416163+0000","last_became_peered":"2024-04-25T19:15:19.416163+0000","last_unstale":"2024-04-25T19:15:20.804534+0000","last_undegraded":"2024-04-25T19:15:20.804534+0000","last_fullsized":"2024-04-25T19:15:20.804534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:46.955159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405410+0000","last_change":"2024-04-25T19:15:18.422875+0000","last_active":"2024-04-25T19:15:20.405410+0000","last_peered":"2024-04-25T19:15:20.405410+0000","last_clean":"2024-04-25T19:15:20.405410+0000","last_became_active":"2024-04-25T19:15:18.422489+0000","last_became_peered":"2024-04-25T19:15:18.422489+0000","last_unstale":"2024-04-25T19:15:20.405410+0000","last_undegraded":"2024-04-25T19:15:20.405410+0000","last_fullsized":"2024-04-25T19:15:20.405410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:30.602500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405477+0000","last_change":"2024-04-25T19:15:19.416987+0000","last_active":"2024-04-25T19:15:20.405477+0000","last_peered":"2024-04-25T19:15:20.405477+0000","last_clean":"2024-04-25T19:15:20.405477+0000","last_became_active":"2024-04-25T19:15:19.414173+0000","last_became_peered":"2024-04-25T19:15:19.414173+0000","last_unstale":"2024-04-25T19:15:20.405477+0000","last_undegraded":"2024-04-25T19:15:20.405477+0000","last_fullsized":"2024-04-25T19:15:20.405477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:32:43.188653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:24.414611+0000","last_change":"2024-04-25T19:15:18.423867+0000","last_active":"2024-04-25T19:15:24.414611+0000","last_peered":"2024-04-25T19:15:24.414611+0000","last_clean":"2024-04-25T19:15:24.414611+0000","last_became_active":"2024-04-25T19:15:18.423532+0000","last_became_peered":"2024-04-25T19:15:18.423532+0000","last_unstale":"2024-04-25T19:15:24.414611+0000","last_undegraded":"2024-04-25T19:15:24.414611+0000","last_fullsized":"2024-04-25T19:15:24.414611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:04.531338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64838,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":286720,"data_stored":666555,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":666611,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T19:15:30.313 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T19:15:30.313 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T19:15:30.313 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T19:15:30.313 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T19:15:30.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.454+0000 7f1afcd34640 1 Processor -- start 2024-04-25T19:15:30.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.454+0000 7f1afcd34640 1 -- start start 2024-04-25T19:15:30.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.456+0000 7f1afcd34640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1af80c60c0 0x7f1af814ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:30.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.456+0000 7f1afcd34640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1af8159760 con 0x7f1af8159f40 2024-04-25T19:15:30.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.456+0000 7f1afcd34640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1af80c6ae0 con 0x7f1af815a350 2024-04-25T19:15:30.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.456+0000 7f1afcd34640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f1af80c83d0 con 0x7f1af80c60c0 2024-04-25T19:15:30.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.456+0000 7f1aeffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f1af8159f40 0x7f1af80c87b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36326/0 (socket says 172.21.15.76:36326) 2024-04-25T19:15:30.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.456+0000 7f1aeffff640 1 -- 172.21.15.76:0/3250792393 learned_addr learned my addr 172.21.15.76:0/3250792393 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:30.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.456+0000 7f1af659c640 1 --2- 172.21.15.76:0/3250792393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1af80c60c0 0x7f1af814ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:30.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.456+0000 7f1af659c640 1 -- 172.21.15.76:0/3250792393 >> v1:172.21.15.105:6789/0 conn(0x7f1af815a350 legacy=0x7f1af81640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:30.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.456+0000 7f1af659c640 1 -- 172.21.15.76:0/3250792393 >> v1:172.21.15.76:6789/0 conn(0x7f1af8159f40 legacy=0x7f1af80c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:30.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.456+0000 7f1af659c640 1 -- 172.21.15.76:0/3250792393 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af80c6dc0 con 0x7f1af80c60c0 2024-04-25T19:15:30.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.457+0000 7f1af659c640 1 --2- 172.21.15.76:0/3250792393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1af80c60c0 0x7f1af814ffc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009800 tx=0x7f1ae802a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5152fa3cf80b6a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:30.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.458+0000 7f1af5d9b640 1 -- 172.21.15.76:0/3250792393 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8037070 con 0x7f1af80c60c0 2024-04-25T19:15:30.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.458+0000 7f1af5d9b640 1 -- 172.21.15.76:0/3250792393 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ae8032440 con 0x7f1af80c60c0 2024-04-25T19:15:30.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.458+0000 7f1af5d9b640 1 -- 172.21.15.76:0/3250792393 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8032810 con 0x7f1af80c60c0 2024-04-25T19:15:30.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.459+0000 7f1afcd34640 1 -- 172.21.15.76:0/3250792393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1af80c60c0 msgr2=0x7f1af814ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:30.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.459+0000 7f1afcd34640 1 --2- 172.21.15.76:0/3250792393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1af80c60c0 0x7f1af814ffc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009800 tx=0x7f1ae802a1a0 comp rx=0 tx=0).stop 2024-04-25T19:15:30.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.459+0000 7f1afcd34640 1 -- 172.21.15.76:0/3250792393 shutdown_connections 2024-04-25T19:15:30.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.459+0000 7f1afcd34640 1 --2- 172.21.15.76:0/3250792393 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1af80c60c0 0x7f1af814ffc0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:30.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.459+0000 7f1afcd34640 1 -- 172.21.15.76:0/3250792393 >> 172.21.15.76:0/3250792393 conn(0x7f1af8071a30 msgr2=0x7f1af8071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:30.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.459+0000 7f1afcd34640 1 -- 172.21.15.76:0/3250792393 shutdown_connections 2024-04-25T19:15:30.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.459+0000 7f1afcd34640 1 -- 172.21.15.76:0/3250792393 wait complete. 2024-04-25T19:15:30.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.461+0000 7f1afcd34640 1 Processor -- start 2024-04-25T19:15:30.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.461+0000 7f1afcd34640 1 -- start start 2024-04-25T19:15:30.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.462+0000 7f1afcd34640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1af80c60c0 0x7f1af81f12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:30.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.462+0000 7f1afcd34640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1af8159f40 0x7f1af81e6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:30.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.462+0000 7f1afcd34640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1af815a350 0x7f1af81568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:30.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.462+0000 7f1afcd34640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1af815ac50 con 0x7f1af815a350 2024-04-25T19:15:30.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.462+0000 7f1afcd34640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1af8159a20 con 0x7f1af80c60c0 2024-04-25T19:15:30.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.462+0000 7f1afcd34640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f1af80c6600 con 0x7f1af8159f40 2024-04-25T19:15:30.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.463+0000 7f1af6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1af815a350 0x7f1af81568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:30.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.463+0000 7f1af6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1af815a350 0x7f1af81568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58906/0 (socket says 172.21.15.76:58906) 2024-04-25T19:15:30.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.463+0000 7f1af6d9d640 1 -- 172.21.15.76:0/2507935017 learned_addr learned my addr 172.21.15.76:0/2507935017 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:30.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.463+0000 7f1af659c640 1 --2- 172.21.15.76:0/2507935017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1af80c60c0 0x7f1af81f12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:30.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.463+0000 7f1aeffff640 1 --2- 172.21.15.76:0/2507935017 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1af8159f40 0x7f1af81e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:30.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.463+0000 7f1af6d9d640 1 -- 172.21.15.76:0/2507935017 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1af8159f40 msgr2=0x7f1af81e6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:30.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.463+0000 7f1af6d9d640 1 --2- 172.21.15.76:0/2507935017 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1af8159f40 0x7f1af81e6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:30.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.463+0000 7f1af6d9d640 1 -- 172.21.15.76:0/2507935017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1af80c60c0 msgr2=0x7f1af81f12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:30.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.463+0000 7f1af6d9d640 1 --2- 172.21.15.76:0/2507935017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1af80c60c0 0x7f1af81f12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:30.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.463+0000 7f1af6d9d640 1 -- 172.21.15.76:0/2507935017 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af80cda50 con 0x7f1af815a350 2024-04-25T19:15:30.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.463+0000 7f1af6d9d640 1 --2- 172.21.15.76:0/2507935017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1af815a350 0x7f1af81568e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1af000c6b0 tx=0x7f1af000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:30.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.463+0000 7f1aeeffd640 1 -- 172.21.15.76:0/2507935017 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af0003b70 con 0x7f1af815a350 2024-04-25T19:15:30.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.463+0000 7f1aeeffd640 1 -- 172.21.15.76:0/2507935017 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1af0003d30 con 0x7f1af815a350 2024-04-25T19:15:30.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.463+0000 7f1afcd34640 1 -- 172.21.15.76:0/2507935017 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af81f1800 con 0x7f1af815a350 2024-04-25T19:15:30.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.463+0000 7f1aeeffd640 1 -- 172.21.15.76:0/2507935017 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af0005310 con 0x7f1af815a350 2024-04-25T19:15:30.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.463+0000 7f1afcd34640 1 -- 172.21.15.76:0/2507935017 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af80cdfd0 con 0x7f1af815a350 2024-04-25T19:15:30.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.465+0000 7f1afcd34640 1 -- 172.21.15.76:0/2507935017 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1af8069a50 con 0x7f1af815a350 2024-04-25T19:15:30.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.465+0000 7f1aeeffd640 1 -- 172.21.15.76:0/2507935017 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1af00054b0 con 0x7f1af815a350 2024-04-25T19:15:30.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.465+0000 7f1aeeffd640 1 --2- 172.21.15.76:0/2507935017 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1ad80744b0 0x7f1ad80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:30.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.465+0000 7f1aeeffd640 1 -- 172.21.15.76:0/2507935017 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1af0094f50 con 0x7f1af815a350 2024-04-25T19:15:30.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.466+0000 7f1af659c640 1 --2- 172.21.15.76:0/2507935017 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1ad80744b0 0x7f1ad80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:30.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.466+0000 7f1af659c640 1 --2- 172.21.15.76:0/2507935017 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1ad80744b0 0x7f1ad80769c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8034000 tx=0x7f1ae802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:30.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.470+0000 7f1aeeffd640 1 -- 172.21.15.76:0/2507935017 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1af8069a50 con 0x7f1af815a350 2024-04-25T19:15:30.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.708+0000 7f1afcd34640 1 -- 172.21.15.76:0/2507935017 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1af80c7450 con 0x7f1af815a350 2024-04-25T19:15:30.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.708+0000 7f1aeeffd640 1 -- 172.21.15.76:0/2507935017 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f1af80c7450 con 0x7f1af815a350 2024-04-25T19:15:30.710 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:30.710 INFO:teuthology.orchestra.run.smithi076.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-25T19:15:30.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.710+0000 7f1afcd34640 1 -- 172.21.15.76:0/2507935017 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1ad80744b0 msgr2=0x7f1ad80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:30.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.710+0000 7f1afcd34640 1 --2- 172.21.15.76:0/2507935017 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1ad80744b0 0x7f1ad80769c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8034000 tx=0x7f1ae802e000 comp rx=0 tx=0).stop 2024-04-25T19:15:30.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.710+0000 7f1afcd34640 1 -- 172.21.15.76:0/2507935017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1af815a350 msgr2=0x7f1af81568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:30.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.710+0000 7f1afcd34640 1 --2- 172.21.15.76:0/2507935017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1af815a350 0x7f1af81568e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1af000c6b0 tx=0x7f1af000cb70 comp rx=0 tx=0).stop 2024-04-25T19:15:30.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.710+0000 7f1afcd34640 1 -- 172.21.15.76:0/2507935017 shutdown_connections 2024-04-25T19:15:30.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.710+0000 7f1afcd34640 1 --2- 172.21.15.76:0/2507935017 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1ad80744b0 0x7f1ad80769c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:30.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.710+0000 7f1afcd34640 1 --2- 172.21.15.76:0/2507935017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1af815a350 0x7f1af81568e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:30.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.710+0000 7f1afcd34640 1 --2- 172.21.15.76:0/2507935017 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1af8159f40 0x7f1af81e6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:30.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.710+0000 7f1afcd34640 1 --2- 172.21.15.76:0/2507935017 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1af80c60c0 0x7f1af81f12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:30.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.710+0000 7f1afcd34640 1 -- 172.21.15.76:0/2507935017 >> 172.21.15.76:0/2507935017 conn(0x7f1af8071a30 msgr2=0x7f1af80cf2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:30.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.710+0000 7f1afcd34640 1 -- 172.21.15.76:0/2507935017 shutdown_connections 2024-04-25T19:15:30.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:30.710+0000 7f1afcd34640 1 -- 172.21.15.76:0/2507935017 wait complete. 2024-04-25T19:15:33.376 INFO:tasks.ceph.mds.e.smithi076.stderr:2024-04-25T19:15:33.374+0000 7faf3107a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T19:15:33.376 INFO:tasks.ceph.mds.e.smithi076.stderr:2024-04-25T19:15:33.374+0000 7faf3107a640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-25T19:15:33.723 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T19:15:33.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.829+0000 7f611efb7640 1 Processor -- start 2024-04-25T19:15:33.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.829+0000 7f611efb7640 1 -- start start 2024-04-25T19:15:33.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.831+0000 7f611efb7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6118074710 0x7f61180cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:33.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.831+0000 7f611efb7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f61180cb920 0x7f61180cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:33.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.831+0000 7f611efb7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f611806d420 con 0x7f6118073940 2024-04-25T19:15:33.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.831+0000 7f611efb7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f611806cf90 con 0x7f61180cb920 2024-04-25T19:15:33.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.831+0000 7f611efb7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f611815b400 con 0x7f6118074710 2024-04-25T19:15:33.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.831+0000 7f611d554640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f61180cb920 0x7f61180cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:33.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.831+0000 7f611d554640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f61180cb920 0x7f61180cefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58912/0 (socket says 172.21.15.76:58912) 2024-04-25T19:15:33.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.831+0000 7f611d554640 1 -- 172.21.15.76:0/3496818726 learned_addr learned my addr 172.21.15.76:0/3496818726 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:33.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.831+0000 7f611d554640 1 -- 172.21.15.76:0/3496818726 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6118074710 msgr2=0x7f61180cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:33.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.831+0000 7f611d554640 1 --2- 172.21.15.76:0/3496818726 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6118074710 0x7f61180cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:33.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.831+0000 7f611d554640 1 -- 172.21.15.76:0/3496818726 >> v1:172.21.15.76:6789/0 conn(0x7f6118073940 legacy=0x7f6118154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:33.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.831+0000 7f611d554640 1 -- 172.21.15.76:0/3496818726 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f611815b6e0 con 0x7f61180cb920 2024-04-25T19:15:33.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.832+0000 7f611d554640 1 --2- 172.21.15.76:0/3496818726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61180cb920 0x7f61180cefe0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6110009e50 tx=0x7f611002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de1de41380623bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:33.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.832+0000 7f61177fe640 1 -- 172.21.15.76:0/3496818726 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6110036070 con 0x7f61180cb920 2024-04-25T19:15:33.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.832+0000 7f61177fe640 1 -- 172.21.15.76:0/3496818726 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6110031600 con 0x7f61180cb920 2024-04-25T19:15:33.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.832+0000 7f61177fe640 1 -- 172.21.15.76:0/3496818726 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61100319d0 con 0x7f61180cb920 2024-04-25T19:15:33.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.833+0000 7f611efb7640 1 -- 172.21.15.76:0/3496818726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61180cb920 msgr2=0x7f61180cefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:33.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.833+0000 7f611efb7640 1 --2- 172.21.15.76:0/3496818726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61180cb920 0x7f61180cefe0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6110009e50 tx=0x7f611002a7a0 comp rx=0 tx=0).stop 2024-04-25T19:15:33.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.833+0000 7f611efb7640 1 -- 172.21.15.76:0/3496818726 shutdown_connections 2024-04-25T19:15:33.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.833+0000 7f611efb7640 1 --2- 172.21.15.76:0/3496818726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61180cb920 0x7f61180cefe0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:33.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.833+0000 7f611efb7640 1 --2- 172.21.15.76:0/3496818726 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6118074710 0x7f61180cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:33.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.833+0000 7f611efb7640 1 -- 172.21.15.76:0/3496818726 >> 172.21.15.76:0/3496818726 conn(0x7f6118071a30 msgr2=0x7f6118071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:33.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.833+0000 7f611efb7640 1 -- 172.21.15.76:0/3496818726 shutdown_connections 2024-04-25T19:15:33.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.833+0000 7f611efb7640 1 -- 172.21.15.76:0/3496818726 wait complete. 2024-04-25T19:15:33.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.835+0000 7f611efb7640 1 Processor -- start 2024-04-25T19:15:33.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.835+0000 7f611efb7640 1 -- start start 2024-04-25T19:15:33.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.836+0000 7f611efb7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6118073940 0x7f611815a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:33.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.836+0000 7f611efb7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6118074710 0x7f61180c0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:33.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.836+0000 7f611efb7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61180cb920 0x7f61180c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:33.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.836+0000 7f611efb7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f611806cc20 con 0x7f61180cb920 2024-04-25T19:15:33.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.836+0000 7f611efb7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f611806c940 con 0x7f6118073940 2024-04-25T19:15:33.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.836+0000 7f611efb7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f6118073d50 con 0x7f6118074710 2024-04-25T19:15:33.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.836+0000 7f611d554640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61180cb920 0x7f61180c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:33.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.836+0000 7f611d554640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61180cb920 0x7f61180c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58918/0 (socket says 172.21.15.76:58918) 2024-04-25T19:15:33.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.836+0000 7f611d554640 1 -- 172.21.15.76:0/3828481846 learned_addr learned my addr 172.21.15.76:0/3828481846 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:33.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.837+0000 7f6117fff640 1 --2- 172.21.15.76:0/3828481846 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6118074710 0x7f61180c0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:33.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.837+0000 7f611cd53640 1 --2- 172.21.15.76:0/3828481846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6118073940 0x7f611815a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:33.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.837+0000 7f611d554640 1 -- 172.21.15.76:0/3828481846 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6118074710 msgr2=0x7f61180c0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:33.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.837+0000 7f611d554640 1 --2- 172.21.15.76:0/3828481846 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6118074710 0x7f61180c0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:33.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.837+0000 7f611d554640 1 -- 172.21.15.76:0/3828481846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6118073940 msgr2=0x7f611815a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:33.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.837+0000 7f611d554640 1 --2- 172.21.15.76:0/3828481846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6118073940 0x7f611815a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:33.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.837+0000 7f611d554640 1 -- 172.21.15.76:0/3828481846 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61180bef70 con 0x7f61180cb920 2024-04-25T19:15:33.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.837+0000 7f611cd53640 1 --2- 172.21.15.76:0/3828481846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6118073940 0x7f611815a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:33.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.837+0000 7f611d554640 1 --2- 172.21.15.76:0/3828481846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61180cb920 0x7f61180c6dd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6110009f70 tx=0x7f6110004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:33.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.837+0000 7f61157fa640 1 -- 172.21.15.76:0/3828481846 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6110041020 con 0x7f61180cb920 2024-04-25T19:15:33.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.837+0000 7f61157fa640 1 -- 172.21.15.76:0/3828481846 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6110007780 con 0x7f61180cb920 2024-04-25T19:15:33.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.837+0000 7f611efb7640 1 -- 172.21.15.76:0/3828481846 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61180bf250 con 0x7f61180cb920 2024-04-25T19:15:33.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.837+0000 7f61157fa640 1 -- 172.21.15.76:0/3828481846 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6110036040 con 0x7f61180cb920 2024-04-25T19:15:33.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.837+0000 7f611efb7640 1 -- 172.21.15.76:0/3828481846 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61180bf810 con 0x7f61180cb920 2024-04-25T19:15:33.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.838+0000 7f611efb7640 1 -- 172.21.15.76:0/3828481846 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61181f39b0 con 0x7f61180cb920 2024-04-25T19:15:33.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.839+0000 7f61157fa640 1 -- 172.21.15.76:0/3828481846 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6110044020 con 0x7f61180cb920 2024-04-25T19:15:33.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.839+0000 7f61157fa640 1 --2- 172.21.15.76:0/3828481846 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f60fc0744b0 0x7f60fc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:33.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.839+0000 7f61157fa640 1 -- 172.21.15.76:0/3828481846 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f61100b5f40 con 0x7f61180cb920 2024-04-25T19:15:33.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.839+0000 7f611cd53640 1 --2- 172.21.15.76:0/3828481846 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f60fc0744b0 0x7f60fc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:33.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.840+0000 7f611cd53640 1 --2- 172.21.15.76:0/3828481846 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f60fc0744b0 0x7f60fc0769c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f610c001660 tx=0x7f610c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:33.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:33.849+0000 7f61157fa640 1 -- 172.21.15.76:0/3828481846 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f611004b030 con 0x7f61180cb920 2024-04-25T19:15:34.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:34.070+0000 7f611efb7640 1 -- 172.21.15.76:0/3828481846 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f611806a390 con 0x7f61180cb920 2024-04-25T19:15:34.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:34.070+0000 7f61157fa640 1 -- 172.21.15.76:0/3828481846 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f61100806d0 con 0x7f61180cb920 2024-04-25T19:15:34.072 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:34.072 INFO:teuthology.orchestra.run.smithi076.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-25T19:15:34.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:34.072+0000 7f611efb7640 1 -- 172.21.15.76:0/3828481846 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f60fc0744b0 msgr2=0x7f60fc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:34.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:34.072+0000 7f611efb7640 1 --2- 172.21.15.76:0/3828481846 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f60fc0744b0 0x7f60fc0769c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f610c001660 tx=0x7f610c009040 comp rx=0 tx=0).stop 2024-04-25T19:15:34.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:34.072+0000 7f611efb7640 1 -- 172.21.15.76:0/3828481846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61180cb920 msgr2=0x7f61180c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:34.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:34.072+0000 7f611efb7640 1 --2- 172.21.15.76:0/3828481846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61180cb920 0x7f61180c6dd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6110009f70 tx=0x7f6110004a40 comp rx=0 tx=0).stop 2024-04-25T19:15:34.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:34.072+0000 7f611efb7640 1 -- 172.21.15.76:0/3828481846 shutdown_connections 2024-04-25T19:15:34.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:34.072+0000 7f611efb7640 1 --2- 172.21.15.76:0/3828481846 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f60fc0744b0 0x7f60fc0769c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:34.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:34.072+0000 7f611efb7640 1 --2- 172.21.15.76:0/3828481846 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61180cb920 0x7f61180c6dd0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:34.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:34.072+0000 7f611efb7640 1 --2- 172.21.15.76:0/3828481846 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6118074710 0x7f61180c0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:34.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:34.072+0000 7f611efb7640 1 --2- 172.21.15.76:0/3828481846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6118073940 0x7f611815a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:34.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:34.072+0000 7f611efb7640 1 -- 172.21.15.76:0/3828481846 >> 172.21.15.76:0/3828481846 conn(0x7f6118071a30 msgr2=0x7f61180cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:34.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:34.073+0000 7f611efb7640 1 -- 172.21.15.76:0/3828481846 shutdown_connections 2024-04-25T19:15:34.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:34.073+0000 7f611efb7640 1 -- 172.21.15.76:0/3828481846 wait complete. 2024-04-25T19:15:37.086 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T19:15:37.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.192+0000 7f0099de3640 1 Processor -- start 2024-04-25T19:15:37.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.192+0000 7f0099de3640 1 -- start start 2024-04-25T19:15:37.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.193+0000 7f0099de3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f00940ca750 0x7f00940cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:37.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.193+0000 7f0099de3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f00940cc6f0 0x7f00940ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:37.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.193+0000 7f0099de3640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00940cd040 con 0x7f00941600b0 2024-04-25T19:15:37.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.194+0000 7f0099de3640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f00941604c0 con 0x7f00940cc6f0 2024-04-25T19:15:37.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.194+0000 7f0099de3640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f009415dae0 con 0x7f00940ca750 2024-04-25T19:15:37.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.194+0000 7f0093fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f00940cc6f0 0x7f00940ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:37.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.194+0000 7f0093fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f00940cc6f0 0x7f00940ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58922/0 (socket says 172.21.15.76:58922) 2024-04-25T19:15:37.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.194+0000 7f0093fff640 1 -- 172.21.15.76:0/4048687200 learned_addr learned my addr 172.21.15.76:0/4048687200 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:37.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.194+0000 7f00937fe640 1 --1- 172.21.15.76:0/4048687200 >> v1:172.21.15.76:6789/0 conn(0x7f00941600b0 0x7f009415d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36340/0 (socket says 172.21.15.76:36340) 2024-04-25T19:15:37.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.194+0000 7f008affd640 1 --2- 172.21.15.76:0/4048687200 >> v2:172.21.15.105:3300/0 conn(0x7f00940ca750 0x7f00940cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:37.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.194+0000 7f0093fff640 1 -- 172.21.15.76:0/4048687200 >> v2:172.21.15.105:3300/0 conn(0x7f00940ca750 msgr2=0x7f00940cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:37.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.194+0000 7f0093fff640 1 --2- 172.21.15.76:0/4048687200 >> v2:172.21.15.105:3300/0 conn(0x7f00940ca750 0x7f00940cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:37.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.194+0000 7f0093fff640 1 -- 172.21.15.76:0/4048687200 >> v1:172.21.15.76:6789/0 conn(0x7f00941600b0 legacy=0x7f009415d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:37.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.194+0000 7f0093fff640 1 -- 172.21.15.76:0/4048687200 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00941606f0 con 0x7f00940cc6f0 2024-04-25T19:15:37.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.195+0000 7f0093fff640 1 --2- 172.21.15.76:0/4048687200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00940cc6f0 0x7f00940ccb00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f00940d0e10 tx=0x7f008c02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa5b676a0481dfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:37.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.195+0000 7f0092ffd640 1 -- 172.21.15.76:0/4048687200 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008c036070 con 0x7f00940cc6f0 2024-04-25T19:15:37.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.195+0000 7f0092ffd640 1 -- 172.21.15.76:0/4048687200 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f008c031600 con 0x7f00940cc6f0 2024-04-25T19:15:37.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.195+0000 7f0092ffd640 1 -- 172.21.15.76:0/4048687200 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008c0319f0 con 0x7f00940cc6f0 2024-04-25T19:15:37.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.195+0000 7f0099de3640 1 -- 172.21.15.76:0/4048687200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00940cc6f0 msgr2=0x7f00940ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:37.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.195+0000 7f0099de3640 1 --2- 172.21.15.76:0/4048687200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00940cc6f0 0x7f00940ccb00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f00940d0e10 tx=0x7f008c02a7c0 comp rx=0 tx=0).stop 2024-04-25T19:15:37.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.195+0000 7f0099de3640 1 -- 172.21.15.76:0/4048687200 shutdown_connections 2024-04-25T19:15:37.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.195+0000 7f0099de3640 1 --2- 172.21.15.76:0/4048687200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00940cc6f0 0x7f00940ccb00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:37.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.195+0000 7f0099de3640 1 --2- 172.21.15.76:0/4048687200 >> v2:172.21.15.105:3300/0 conn(0x7f00940ca750 0x7f00940cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:37.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.195+0000 7f0099de3640 1 -- 172.21.15.76:0/4048687200 >> 172.21.15.76:0/4048687200 conn(0x7f0094068b10 msgr2=0x7f0094072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:37.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.196+0000 7f0099de3640 1 -- 172.21.15.76:0/4048687200 shutdown_connections 2024-04-25T19:15:37.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.196+0000 7f0099de3640 1 -- 172.21.15.76:0/4048687200 wait complete. 2024-04-25T19:15:37.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.197+0000 7f0099de3640 1 Processor -- start 2024-04-25T19:15:37.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.197+0000 7f0099de3640 1 -- start start 2024-04-25T19:15:37.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.199+0000 7f0099de3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00940ca750 0x7f00941f6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:37.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.199+0000 7f0099de3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00940cc6f0 0x7f00941509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:37.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.199+0000 7f0099de3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f00941600b0 0x7f00940ce600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:37.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.199+0000 7f0099de3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f009415cbd0 con 0x7f00940ca750 2024-04-25T19:15:37.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.199+0000 7f0099de3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0094160cb0 con 0x7f00940cc6f0 2024-04-25T19:15:37.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.199+0000 7f0099de3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f009415ceb0 con 0x7f00941600b0 2024-04-25T19:15:37.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.199+0000 7f00937fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00940ca750 0x7f00941f6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:37.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.199+0000 7f00937fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00940ca750 0x7f00941f6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58936/0 (socket says 172.21.15.76:58936) 2024-04-25T19:15:37.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.199+0000 7f00937fe640 1 -- 172.21.15.76:0/637381305 learned_addr learned my addr 172.21.15.76:0/637381305 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:37.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.199+0000 7f008affd640 1 --2- 172.21.15.76:0/637381305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00940cc6f0 0x7f00941509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:37.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.199+0000 7f0093fff640 1 --2- 172.21.15.76:0/637381305 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f00941600b0 0x7f00940ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:37.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.199+0000 7f00937fe640 1 -- 172.21.15.76:0/637381305 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f00941600b0 msgr2=0x7f00940ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:37.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.199+0000 7f00937fe640 1 --2- 172.21.15.76:0/637381305 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f00941600b0 0x7f00940ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:37.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.199+0000 7f00937fe640 1 -- 172.21.15.76:0/637381305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00940cc6f0 msgr2=0x7f00941509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:37.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.199+0000 7f00937fe640 1 --2- 172.21.15.76:0/637381305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00940cc6f0 0x7f00941509a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:37.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.199+0000 7f00937fe640 1 -- 172.21.15.76:0/637381305 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00941f6f90 con 0x7f00940ca750 2024-04-25T19:15:37.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.199+0000 7f008affd640 1 --2- 172.21.15.76:0/637381305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00940cc6f0 0x7f00941509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:37.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.199+0000 7f00937fe640 1 --2- 172.21.15.76:0/637381305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00940ca750 0x7f00941f6a50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f008400c3c0 tx=0x7f008400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:37.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.199+0000 7f0090ff9640 1 -- 172.21.15.76:0/637381305 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00840074e0 con 0x7f00940ca750 2024-04-25T19:15:37.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.199+0000 7f0090ff9640 1 -- 172.21.15.76:0/637381305 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0084007680 con 0x7f00940ca750 2024-04-25T19:15:37.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.199+0000 7f0099de3640 1 -- 172.21.15.76:0/637381305 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00941f35b0 con 0x7f00940ca750 2024-04-25T19:15:37.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.199+0000 7f0090ff9640 1 -- 172.21.15.76:0/637381305 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00840049e0 con 0x7f00940ca750 2024-04-25T19:15:37.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.199+0000 7f0099de3640 1 -- 172.21.15.76:0/637381305 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00941f3b70 con 0x7f00940ca750 2024-04-25T19:15:37.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.200+0000 7f0099de3640 1 -- 172.21.15.76:0/637381305 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f009406a390 con 0x7f00940ca750 2024-04-25T19:15:37.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.202+0000 7f0090ff9640 1 -- 172.21.15.76:0/637381305 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f008401a020 con 0x7f00940ca750 2024-04-25T19:15:37.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.202+0000 7f0090ff9640 1 --2- 172.21.15.76:0/637381305 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f00740744b0 0x7f00740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:37.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.202+0000 7f0090ff9640 1 -- 172.21.15.76:0/637381305 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f00840948e0 con 0x7f00940ca750 2024-04-25T19:15:37.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.203+0000 7f008affd640 1 --2- 172.21.15.76:0/637381305 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f00740744b0 0x7f00740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:37.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.203+0000 7f008affd640 1 --2- 172.21.15.76:0/637381305 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f00740744b0 0x7f00740769c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f009415a380 tx=0x7f007c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:37.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.207+0000 7f0090ff9640 1 -- 172.21.15.76:0/637381305 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f008405f100 con 0x7f00940ca750 2024-04-25T19:15:37.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.430+0000 7f0099de3640 1 -- 172.21.15.76:0/637381305 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f009406e740 con 0x7f00940ca750 2024-04-25T19:15:37.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.431+0000 7f0090ff9640 1 -- 172.21.15.76:0/637381305 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f008405ea00 con 0x7f00940ca750 2024-04-25T19:15:37.432 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:37.432 INFO:teuthology.orchestra.run.smithi076.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-25T19:15:37.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.433+0000 7f0099de3640 1 -- 172.21.15.76:0/637381305 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f00740744b0 msgr2=0x7f00740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:37.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.433+0000 7f0099de3640 1 --2- 172.21.15.76:0/637381305 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f00740744b0 0x7f00740769c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f009415a380 tx=0x7f007c008040 comp rx=0 tx=0).stop 2024-04-25T19:15:37.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.433+0000 7f0099de3640 1 -- 172.21.15.76:0/637381305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00940ca750 msgr2=0x7f00941f6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:37.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.433+0000 7f0099de3640 1 --2- 172.21.15.76:0/637381305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00940ca750 0x7f00941f6a50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f008400c3c0 tx=0x7f008400c880 comp rx=0 tx=0).stop 2024-04-25T19:15:37.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.433+0000 7f0099de3640 1 -- 172.21.15.76:0/637381305 shutdown_connections 2024-04-25T19:15:37.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.433+0000 7f0099de3640 1 --2- 172.21.15.76:0/637381305 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f00740744b0 0x7f00740769c0 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f009415a380 tx=0x7f007c008040 comp rx=0 tx=0).stop 2024-04-25T19:15:37.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.433+0000 7f0099de3640 1 --2- 172.21.15.76:0/637381305 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f00941600b0 0x7f00940ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:37.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.433+0000 7f0099de3640 1 --2- 172.21.15.76:0/637381305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00940cc6f0 0x7f00941509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:37.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.433+0000 7f0099de3640 1 --2- 172.21.15.76:0/637381305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00940ca750 0x7f00941f6a50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:37.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.433+0000 7f0099de3640 1 -- 172.21.15.76:0/637381305 >> 172.21.15.76:0/637381305 conn(0x7f0094068b10 msgr2=0x7f009406d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:37.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.433+0000 7f0099de3640 1 -- 172.21.15.76:0/637381305 shutdown_connections 2024-04-25T19:15:37.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:37.433+0000 7f0099de3640 1 -- 172.21.15.76:0/637381305 wait complete. 2024-04-25T19:15:38.376 INFO:tasks.ceph.mds.e.smithi076.stderr:2024-04-25T19:15:38.375+0000 7faf3107a640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-25T19:15:40.447 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T19:15:40.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.552+0000 7f3e0858f640 1 Processor -- start 2024-04-25T19:15:40.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.552+0000 7f3e0858f640 1 -- start start 2024-04-25T19:15:40.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.554+0000 7f3e0858f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e000cea70 0x7f3e000cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:40.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.554+0000 7f3e0858f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3e0006e160 0x7f3e0006e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:40.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.554+0000 7f3e0858f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e0015ce80 con 0x7f3e000d0810 2024-04-25T19:15:40.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.554+0000 7f3e0858f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f3e0015d160 con 0x7f3e0006e160 2024-04-25T19:15:40.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.554+0000 7f3e0858f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f3e0015d440 con 0x7f3e000cea70 2024-04-25T19:15:40.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.554+0000 7f3e0632b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f3e000d0810 0x7f3e0006f000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53546/0 (socket says 172.21.15.76:53546) 2024-04-25T19:15:40.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.554+0000 7f3e0632b640 1 -- 172.21.15.76:0/3841874385 learned_addr learned my addr 172.21.15.76:0/3841874385 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:40.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.554+0000 7f3e06b2c640 1 --2- 172.21.15.76:0/3841874385 >> v2:172.21.15.105:3300/0 conn(0x7f3e0006e160 0x7f3e0006e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:40.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.554+0000 7f3e05b2a640 1 --2- 172.21.15.76:0/3841874385 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e000cea70 0x7f3e000cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:40.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.554+0000 7f3e05329640 1 -- 172.21.15.76:0/3841874385 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1609216339 0 0) 0x7f3e0015ce80 con 0x7f3e000d0810 2024-04-25T19:15:40.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.554+0000 7f3e05329640 1 -- 172.21.15.76:0/3841874385 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3dec003520 con 0x7f3e000d0810 2024-04-25T19:15:40.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.554+0000 7f3e05329640 1 -- 172.21.15.76:0/3841874385 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2870099913 0 0) 0x7f3dec003520 con 0x7f3e000d0810 2024-04-25T19:15:40.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.554+0000 7f3e05329640 1 -- 172.21.15.76:0/3841874385 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e000cea70 msgr2=0x7f3e000cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:40.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.554+0000 7f3e05329640 1 --2- 172.21.15.76:0/3841874385 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e000cea70 0x7f3e000cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:40.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.554+0000 7f3e05329640 1 -- 172.21.15.76:0/3841874385 >> v2:172.21.15.105:3300/0 conn(0x7f3e0006e160 msgr2=0x7f3e0006e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:40.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.554+0000 7f3e05329640 1 --2- 172.21.15.76:0/3841874385 >> v2:172.21.15.105:3300/0 conn(0x7f3e0006e160 0x7f3e0006e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:40.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.554+0000 7f3e05329640 1 -- 172.21.15.76:0/3841874385 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0015d720 con 0x7f3e000d0810 2024-04-25T19:15:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.555+0000 7f3e05329640 1 -- 172.21.15.76:0/3841874385 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f3df4001690 con 0x7f3e000d0810 2024-04-25T19:15:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.555+0000 7f3e06b2c640 1 --2- 172.21.15.76:0/3841874385 >> v2:172.21.15.105:3300/0 conn(0x7f3e0006e160 0x7f3e0006e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.555+0000 7f3e05329640 1 -- 172.21.15.76:0/3841874385 >> v1:172.21.15.76:6789/0 conn(0x7f3e000d0810 legacy=0x7f3e0006f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.555+0000 7f3e05329640 1 --2- 172.21.15.76:0/3841874385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3dec0039b0 0x7f3dec003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.555+0000 7f3e05329640 1 --2- 172.21.15.76:0/3841874385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dec004800 0x7f3dec004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.555+0000 7f3e05329640 1 --2- 172.21.15.76:0/3841874385 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e000cea70 0x7f3dec005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.555+0000 7f3e05329640 1 -- 172.21.15.76:0/3841874385 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0015d160 con 0x7f3dec0039b0 2024-04-25T19:15:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.555+0000 7f3e05329640 1 -- 172.21.15.76:0/3841874385 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0015d440 con 0x7f3dec004800 2024-04-25T19:15:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.555+0000 7f3e05329640 1 -- 172.21.15.76:0/3841874385 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f3dec00b9f0 con 0x7f3e000cea70 2024-04-25T19:15:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.555+0000 7f3e0632b640 1 --2- 172.21.15.76:0/3841874385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3dec0039b0 0x7f3dec003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.555+0000 7f3e06b2c640 1 --2- 172.21.15.76:0/3841874385 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e000cea70 0x7f3dec005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.555+0000 7f3e05b2a640 1 --2- 172.21.15.76:0/3841874385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dec004800 0x7f3dec004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.555+0000 7f3e0632b640 1 -- 172.21.15.76:0/3841874385 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e000cea70 msgr2=0x7f3dec005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.555+0000 7f3e0632b640 1 --2- 172.21.15.76:0/3841874385 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e000cea70 0x7f3dec005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.555+0000 7f3e0632b640 1 -- 172.21.15.76:0/3841874385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dec004800 msgr2=0x7f3dec004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:40.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.555+0000 7f3e0632b640 1 --2- 172.21.15.76:0/3841874385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3dec004800 0x7f3dec004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:40.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.555+0000 7f3e0632b640 1 -- 172.21.15.76:0/3841874385 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dec003520 con 0x7f3dec0039b0 2024-04-25T19:15:40.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.555+0000 7f3e06b2c640 1 --2- 172.21.15.76:0/3841874385 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e000cea70 0x7f3dec005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:40.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.556+0000 7f3e0632b640 1 --2- 172.21.15.76:0/3841874385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3dec0039b0 0x7f3dec003e20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3df4007fd0 tx=0x7f3dec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696e0194c49d26d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:40.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.556+0000 7f3e05329640 1 -- 172.21.15.76:0/3841874385 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df4007d50 con 0x7f3dec0039b0 2024-04-25T19:15:40.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.556+0000 7f3e05329640 1 -- 172.21.15.76:0/3841874385 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3df4005af0 con 0x7f3dec0039b0 2024-04-25T19:15:40.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.556+0000 7f3e05329640 1 -- 172.21.15.76:0/3841874385 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df4007070 con 0x7f3dec0039b0 2024-04-25T19:15:40.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.556+0000 7f3e0858f640 1 -- 172.21.15.76:0/3841874385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3dec0039b0 msgr2=0x7f3dec003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:40.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.556+0000 7f3e0858f640 1 --2- 172.21.15.76:0/3841874385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3dec0039b0 0x7f3dec003e20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3df4007fd0 tx=0x7f3dec005830 comp rx=0 tx=0).stop 2024-04-25T19:15:40.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.556+0000 7f3e06b2c640 1 -- 172.21.15.76:0/3841874385 reap_dead start 2024-04-25T19:15:40.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.557+0000 7f3e0858f640 1 -- 172.21.15.76:0/3841874385 shutdown_connections 2024-04-25T19:15:40.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.557+0000 7f3e0858f640 1 -- 172.21.15.76:0/3841874385 >> 172.21.15.76:0/3841874385 conn(0x7f3e000b3c00 msgr2=0x7f3e000b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:40.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.557+0000 7f3e0858f640 1 -- 172.21.15.76:0/3841874385 shutdown_connections 2024-04-25T19:15:40.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.557+0000 7f3e0858f640 1 -- 172.21.15.76:0/3841874385 wait complete. 2024-04-25T19:15:40.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.558+0000 7f3e0858f640 1 Processor -- start 2024-04-25T19:15:40.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.558+0000 7f3e0858f640 1 -- start start 2024-04-25T19:15:40.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.559+0000 7f3e0858f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3dec0039b0 0x7f3e000c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:40.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.559+0000 7f3e0858f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e00159140 0x7f3e001f2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:40.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.559+0000 7f3e0858f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e001562d0 0x7f3e001566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:40.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.559+0000 7f3e0858f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0006eab0 con 0x7f3e00159140 2024-04-25T19:15:40.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.559+0000 7f3e0858f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0006dc40 con 0x7f3e001562d0 2024-04-25T19:15:40.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.559+0000 7f3e0858f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f3e0015cb70 con 0x7f3dec0039b0 2024-04-25T19:15:40.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.560+0000 7f3e05b2a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e00159140 0x7f3e001f2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:40.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.560+0000 7f3e05b2a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e00159140 0x7f3e001f2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39850/0 (socket says 172.21.15.76:39850) 2024-04-25T19:15:40.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.560+0000 7f3e05b2a640 1 -- 172.21.15.76:0/904273103 learned_addr learned my addr 172.21.15.76:0/904273103 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:40.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.560+0000 7f3e0632b640 1 --2- 172.21.15.76:0/904273103 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3dec0039b0 0x7f3e000c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:40.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.560+0000 7f3e06b2c640 1 --2- 172.21.15.76:0/904273103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e001562d0 0x7f3e001566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:40.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.560+0000 7f3e05b2a640 1 -- 172.21.15.76:0/904273103 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3dec0039b0 msgr2=0x7f3e000c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:40.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.560+0000 7f3e05b2a640 1 --2- 172.21.15.76:0/904273103 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3dec0039b0 0x7f3e000c6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:40.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.560+0000 7f3e05b2a640 1 -- 172.21.15.76:0/904273103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e001562d0 msgr2=0x7f3e001566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:40.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.560+0000 7f3e05b2a640 1 --2- 172.21.15.76:0/904273103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e001562d0 0x7f3e001566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:40.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.560+0000 7f3e05b2a640 1 -- 172.21.15.76:0/904273103 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e001f3240 con 0x7f3e00159140 2024-04-25T19:15:40.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.560+0000 7f3e06b2c640 1 --2- 172.21.15.76:0/904273103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e001562d0 0x7f3e001566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:40.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.560+0000 7f3e05b2a640 1 --2- 172.21.15.76:0/904273103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e00159140 0x7f3e001f2d00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc011570 tx=0x7f3dfc011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:40.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.560+0000 7f3df2ffd640 1 -- 172.21.15.76:0/904273103 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc015930 con 0x7f3e00159140 2024-04-25T19:15:40.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.560+0000 7f3df2ffd640 1 -- 172.21.15.76:0/904273103 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dfc015ad0 con 0x7f3e00159140 2024-04-25T19:15:40.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.560+0000 7f3e0858f640 1 -- 172.21.15.76:0/904273103 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e000c5da0 con 0x7f3e00159140 2024-04-25T19:15:40.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.560+0000 7f3df2ffd640 1 -- 172.21.15.76:0/904273103 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc00f460 con 0x7f3e00159140 2024-04-25T19:15:40.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.560+0000 7f3e0858f640 1 -- 172.21.15.76:0/904273103 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e00159f40 con 0x7f3e00159140 2024-04-25T19:15:40.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.561+0000 7f3e0858f640 1 -- 172.21.15.76:0/904273103 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e00159d60 con 0x7f3e00159140 2024-04-25T19:15:40.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.562+0000 7f3df2ffd640 1 -- 172.21.15.76:0/904273103 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3dfc00f600 con 0x7f3e00159140 2024-04-25T19:15:40.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.562+0000 7f3df2ffd640 1 --2- 172.21.15.76:0/904273103 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f3de40744b0 0x7f3de40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:40.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.563+0000 7f3df2ffd640 1 -- 172.21.15.76:0/904273103 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3dfc095d70 con 0x7f3e00159140 2024-04-25T19:15:40.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.563+0000 7f3e0632b640 1 --2- 172.21.15.76:0/904273103 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f3de40744b0 0x7f3de40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:40.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.563+0000 7f3e0632b640 1 --2- 172.21.15.76:0/904273103 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f3de40744b0 0x7f3de40769c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3df4033000 tx=0x7f3df4009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:40.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.566+0000 7f3df2ffd640 1 -- 172.21.15.76:0/904273103 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3e00159d60 con 0x7f3e00159140 2024-04-25T19:15:40.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.780+0000 7f3e0858f640 1 -- 172.21.15.76:0/904273103 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3e0006a390 con 0x7f3e00159140 2024-04-25T19:15:40.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.780+0000 7f3df2ffd640 1 -- 172.21.15.76:0/904273103 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f3dfc019030 con 0x7f3e00159140 2024-04-25T19:15:40.782 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:40.782 INFO:teuthology.orchestra.run.smithi076.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-25T19:15:40.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.782+0000 7f3e0858f640 1 -- 172.21.15.76:0/904273103 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f3de40744b0 msgr2=0x7f3de40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:40.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.782+0000 7f3e0858f640 1 --2- 172.21.15.76:0/904273103 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f3de40744b0 0x7f3de40769c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3df4033000 tx=0x7f3df4009030 comp rx=0 tx=0).stop 2024-04-25T19:15:40.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.782+0000 7f3e0858f640 1 -- 172.21.15.76:0/904273103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e00159140 msgr2=0x7f3e001f2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:40.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.782+0000 7f3e0858f640 1 --2- 172.21.15.76:0/904273103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e00159140 0x7f3e001f2d00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc011570 tx=0x7f3dfc011a30 comp rx=0 tx=0).stop 2024-04-25T19:15:40.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.782+0000 7f3e0858f640 1 -- 172.21.15.76:0/904273103 shutdown_connections 2024-04-25T19:15:40.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.782+0000 7f3e0858f640 1 --2- 172.21.15.76:0/904273103 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f3de40744b0 0x7f3de40769c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:40.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.782+0000 7f3e0858f640 1 --2- 172.21.15.76:0/904273103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e001562d0 0x7f3e001566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:40.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.782+0000 7f3e0858f640 1 --2- 172.21.15.76:0/904273103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e00159140 0x7f3e001f2d00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:40.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.782+0000 7f3e0858f640 1 --2- 172.21.15.76:0/904273103 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3dec0039b0 0x7f3e000c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:40.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.782+0000 7f3e0858f640 1 -- 172.21.15.76:0/904273103 >> 172.21.15.76:0/904273103 conn(0x7f3e000b3c00 msgr2=0x7f3e000d0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:40.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.782+0000 7f3e0858f640 1 -- 172.21.15.76:0/904273103 shutdown_connections 2024-04-25T19:15:40.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:40.782+0000 7f3e0858f640 1 -- 172.21.15.76:0/904273103 wait complete. 2024-04-25T19:15:43.376 INFO:tasks.ceph.mds.e.smithi076.stderr:2024-04-25T19:15:43.375+0000 7faf3107a640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-25T19:15:43.796 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T19:15:43.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.903+0000 7fd679caa640 1 Processor -- start 2024-04-25T19:15:43.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.903+0000 7fd679caa640 1 -- start start 2024-04-25T19:15:43.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.905+0000 7fd679caa640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd674073880 0x7fd6740c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:43.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.905+0000 7fd679caa640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd6740c8910 0x7fd67415ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:43.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.905+0000 7fd679caa640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd674071ad0 con 0x7fd674071520 2024-04-25T19:15:43.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.905+0000 7fd679caa640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd674071d70 con 0x7fd674073880 2024-04-25T19:15:43.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.905+0000 7fd679caa640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fd674074150 con 0x7fd6740c8910 2024-04-25T19:15:43.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.905+0000 7fd66affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd674073880 0x7fd6740c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:43.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.905+0000 7fd66affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd674073880 0x7fd6740c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39866/0 (socket says 172.21.15.76:39866) 2024-04-25T19:15:43.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.905+0000 7fd66affd640 1 -- 172.21.15.76:0/520085803 learned_addr learned my addr 172.21.15.76:0/520085803 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:43.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.905+0000 7fd673fff640 1 --2- 172.21.15.76:0/520085803 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd6740c8910 0x7fd67415ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:43.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.905+0000 7fd66affd640 1 -- 172.21.15.76:0/520085803 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd6740c8910 msgr2=0x7fd67415ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:43.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.905+0000 7fd66affd640 1 --2- 172.21.15.76:0/520085803 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd6740c8910 0x7fd67415ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:43.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.905+0000 7fd66affd640 1 -- 172.21.15.76:0/520085803 >> v1:172.21.15.105:6789/0 conn(0x7fd674071520 legacy=0x7fd674151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:43.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.905+0000 7fd66affd640 1 -- 172.21.15.76:0/520085803 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd674074430 con 0x7fd674073880 2024-04-25T19:15:43.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.906+0000 7fd66affd640 1 --2- 172.21.15.76:0/520085803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd674073880 0x7fd6740c83d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009870 tx=0x7fd65c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f93641a2cee133 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:43.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.906+0000 7fd672ffd640 1 -- 172.21.15.76:0/520085803 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c037070 con 0x7fd674073880 2024-04-25T19:15:43.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.906+0000 7fd672ffd640 1 -- 172.21.15.76:0/520085803 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd65c032440 con 0x7fd674073880 2024-04-25T19:15:43.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.906+0000 7fd672ffd640 1 -- 172.21.15.76:0/520085803 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c032810 con 0x7fd674073880 2024-04-25T19:15:43.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.906+0000 7fd679caa640 1 -- 172.21.15.76:0/520085803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd674073880 msgr2=0x7fd6740c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:43.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.906+0000 7fd679caa640 1 --2- 172.21.15.76:0/520085803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd674073880 0x7fd6740c83d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009870 tx=0x7fd65c02a2a0 comp rx=0 tx=0).stop 2024-04-25T19:15:43.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.907+0000 7fd679caa640 1 -- 172.21.15.76:0/520085803 shutdown_connections 2024-04-25T19:15:43.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.907+0000 7fd679caa640 1 --2- 172.21.15.76:0/520085803 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd6740c8910 0x7fd67415ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:43.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.907+0000 7fd679caa640 1 --2- 172.21.15.76:0/520085803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd674073880 0x7fd6740c83d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:43.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.907+0000 7fd679caa640 1 -- 172.21.15.76:0/520085803 >> 172.21.15.76:0/520085803 conn(0x7fd6740cecc0 msgr2=0x7fd6740cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:43.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.907+0000 7fd679caa640 1 -- 172.21.15.76:0/520085803 shutdown_connections 2024-04-25T19:15:43.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.907+0000 7fd679caa640 1 -- 172.21.15.76:0/520085803 wait complete. 2024-04-25T19:15:43.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.908+0000 7fd679caa640 1 Processor -- start 2024-04-25T19:15:43.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.908+0000 7fd679caa640 1 -- start start 2024-04-25T19:15:43.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.910+0000 7fd679caa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd674071520 0x7fd6741529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:43.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.910+0000 7fd679caa640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd674073880 0x7fd67414cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:43.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.910+0000 7fd679caa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6740c8910 0x7fd6741e6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:43.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.910+0000 7fd679caa640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd674154fc0 con 0x7fd674071520 2024-04-25T19:15:43.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.910+0000 7fd679caa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd6740749f0 con 0x7fd6740c8910 2024-04-25T19:15:43.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.910+0000 7fd679caa640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fd67414d800 con 0x7fd674073880 2024-04-25T19:15:43.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.910+0000 7fd6737fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd674071520 0x7fd6741529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:43.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.910+0000 7fd6737fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd674071520 0x7fd6741529b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39876/0 (socket says 172.21.15.76:39876) 2024-04-25T19:15:43.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.910+0000 7fd6737fe640 1 -- 172.21.15.76:0/2649279578 learned_addr learned my addr 172.21.15.76:0/2649279578 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:43.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.910+0000 7fd66affd640 1 --2- 172.21.15.76:0/2649279578 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd674073880 0x7fd67414cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:43.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.910+0000 7fd673fff640 1 --2- 172.21.15.76:0/2649279578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6740c8910 0x7fd6741e6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:43.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.910+0000 7fd6737fe640 1 -- 172.21.15.76:0/2649279578 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd674073880 msgr2=0x7fd67414cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:43.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.910+0000 7fd6737fe640 1 --2- 172.21.15.76:0/2649279578 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd674073880 0x7fd67414cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:43.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.910+0000 7fd6737fe640 1 -- 172.21.15.76:0/2649279578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6740c8910 msgr2=0x7fd6741e6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:43.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.910+0000 7fd6737fe640 1 --2- 172.21.15.76:0/2649279578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6740c8910 0x7fd6741e6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:43.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.910+0000 7fd6737fe640 1 -- 172.21.15.76:0/2649279578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67414d330 con 0x7fd674071520 2024-04-25T19:15:43.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.910+0000 7fd6737fe640 1 --2- 172.21.15.76:0/2649279578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd674071520 0x7fd6741529b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd66400b420 tx=0x7fd66400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:43.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.910+0000 7fd670ff9640 1 -- 172.21.15.76:0/2649279578 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664011830 con 0x7fd674071520 2024-04-25T19:15:43.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.910+0000 7fd670ff9640 1 -- 172.21.15.76:0/2649279578 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd664016070 con 0x7fd674071520 2024-04-25T19:15:43.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.910+0000 7fd670ff9640 1 -- 172.21.15.76:0/2649279578 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664011c10 con 0x7fd674071520 2024-04-25T19:15:43.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.910+0000 7fd679caa640 1 -- 172.21.15.76:0/2649279578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd674153880 con 0x7fd674071520 2024-04-25T19:15:43.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.911+0000 7fd679caa640 1 -- 172.21.15.76:0/2649279578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd674153e40 con 0x7fd674071520 2024-04-25T19:15:43.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.911+0000 7fd679caa640 1 -- 172.21.15.76:0/2649279578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd67406a390 con 0x7fd674071520 2024-04-25T19:15:43.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.912+0000 7fd670ff9640 1 -- 172.21.15.76:0/2649279578 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd66401b460 con 0x7fd674071520 2024-04-25T19:15:43.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.913+0000 7fd670ff9640 1 --2- 172.21.15.76:0/2649279578 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fd6540744b0 0x7fd6540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:43.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.913+0000 7fd670ff9640 1 -- 172.21.15.76:0/2649279578 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd664094c90 con 0x7fd674071520 2024-04-25T19:15:43.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.913+0000 7fd66affd640 1 --2- 172.21.15.76:0/2649279578 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fd6540744b0 0x7fd6540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:43.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.913+0000 7fd66affd640 1 --2- 172.21.15.76:0/2649279578 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fd6540744b0 0x7fd6540769c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd65c034000 tx=0x7fd65c02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:43.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:43.922+0000 7fd670ff9640 1 -- 172.21.15.76:0/2649279578 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd664020020 con 0x7fd674071520 2024-04-25T19:15:44.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.144+0000 7fd679caa640 1 -- 172.21.15.76:0/2649279578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd674072040 con 0x7fd674071520 2024-04-25T19:15:44.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.144+0000 7fd670ff9640 1 -- 172.21.15.76:0/2649279578 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fd66405f4b0 con 0x7fd674071520 2024-04-25T19:15:44.146 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:44.147 INFO:teuthology.orchestra.run.smithi076.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T19:15:44.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.146+0000 7fd679caa640 1 -- 172.21.15.76:0/2649279578 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fd6540744b0 msgr2=0x7fd6540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:44.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.146+0000 7fd679caa640 1 --2- 172.21.15.76:0/2649279578 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fd6540744b0 0x7fd6540769c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd65c034000 tx=0x7fd65c02e000 comp rx=0 tx=0).stop 2024-04-25T19:15:44.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.146+0000 7fd679caa640 1 -- 172.21.15.76:0/2649279578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd674071520 msgr2=0x7fd6741529b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:44.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.146+0000 7fd679caa640 1 --2- 172.21.15.76:0/2649279578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd674071520 0x7fd6741529b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd66400b420 tx=0x7fd66400b8e0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.146+0000 7fd679caa640 1 -- 172.21.15.76:0/2649279578 shutdown_connections 2024-04-25T19:15:44.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.146+0000 7fd679caa640 1 --2- 172.21.15.76:0/2649279578 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fd6540744b0 0x7fd6540769c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.146+0000 7fd679caa640 1 --2- 172.21.15.76:0/2649279578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6740c8910 0x7fd6741e6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.146+0000 7fd679caa640 1 --2- 172.21.15.76:0/2649279578 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd674073880 0x7fd67414cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.146+0000 7fd679caa640 1 --2- 172.21.15.76:0/2649279578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd674071520 0x7fd6741529b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.146+0000 7fd679caa640 1 -- 172.21.15.76:0/2649279578 >> 172.21.15.76:0/2649279578 conn(0x7fd6740cecc0 msgr2=0x7fd674158850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:44.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.146+0000 7fd679caa640 1 -- 172.21.15.76:0/2649279578 shutdown_connections 2024-04-25T19:15:44.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.147+0000 7fd679caa640 1 -- 172.21.15.76:0/2649279578 wait complete. 2024-04-25T19:15:44.159 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T19:15:44.160 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:15:44.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.267+0000 7ff11d284640 1 Processor -- start 2024-04-25T19:15:44.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.267+0000 7ff11d284640 1 -- start start 2024-04-25T19:15:44.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.268+0000 7ff11d284640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff118074650 0x7ff11815cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:44.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.268+0000 7ff11d284640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff11806c5c0 0x7ff11806c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:44.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.268+0000 7ff11d284640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff11815d1c0 con 0x7ff1180c96c0 2024-04-25T19:15:44.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.268+0000 7ff11d284640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff118161120 con 0x7ff11806c5c0 2024-04-25T19:15:44.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.268+0000 7ff11d284640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff11815d4a0 con 0x7ff118074650 2024-04-25T19:15:44.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.268+0000 7ff1177fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff11806c5c0 0x7ff11806c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:44.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.268+0000 7ff1177fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff11806c5c0 0x7ff11806c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39888/0 (socket says 172.21.15.76:39888) 2024-04-25T19:15:44.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.268+0000 7ff1177fe640 1 -- 172.21.15.76:0/2765095337 learned_addr learned my addr 172.21.15.76:0/2765095337 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:44.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.268+0000 7ff10ffff640 1 --2- 172.21.15.76:0/2765095337 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff118074650 0x7ff11815cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:44.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.268+0000 7ff1177fe640 1 -- 172.21.15.76:0/2765095337 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff118074650 msgr2=0x7ff11815cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:44.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.268+0000 7ff1177fe640 1 --2- 172.21.15.76:0/2765095337 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff118074650 0x7ff11815cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.268+0000 7ff1177fe640 1 -- 172.21.15.76:0/2765095337 >> v1:172.21.15.76:6789/0 conn(0x7ff1180c96c0 legacy=0x7ff118160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:44.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.268+0000 7ff1177fe640 1 -- 172.21.15.76:0/2765095337 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11815d780 con 0x7ff11806c5c0 2024-04-25T19:15:44.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.269+0000 7ff1177fe640 1 --2- 172.21.15.76:0/2765095337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff11806c5c0 0x7ff11806c9d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff110009e50 tx=0x7ff11002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b6c2bfcc7ff735 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:44.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.269+0000 7ff1167fc640 1 -- 172.21.15.76:0/2765095337 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff110036070 con 0x7ff11806c5c0 2024-04-25T19:15:44.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.269+0000 7ff1167fc640 1 -- 172.21.15.76:0/2765095337 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff110031600 con 0x7ff11806c5c0 2024-04-25T19:15:44.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.269+0000 7ff1167fc640 1 -- 172.21.15.76:0/2765095337 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1100319d0 con 0x7ff11806c5c0 2024-04-25T19:15:44.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.270+0000 7ff11d284640 1 -- 172.21.15.76:0/2765095337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff11806c5c0 msgr2=0x7ff11806c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:44.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.270+0000 7ff11d284640 1 --2- 172.21.15.76:0/2765095337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff11806c5c0 0x7ff11806c9d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff110009e50 tx=0x7ff11002a7a0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.270+0000 7ff11d284640 1 -- 172.21.15.76:0/2765095337 shutdown_connections 2024-04-25T19:15:44.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.270+0000 7ff11d284640 1 --2- 172.21.15.76:0/2765095337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff11806c5c0 0x7ff11806c9d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.270+0000 7ff11d284640 1 --2- 172.21.15.76:0/2765095337 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff118074650 0x7ff11815cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.270+0000 7ff11d284640 1 -- 172.21.15.76:0/2765095337 >> 172.21.15.76:0/2765095337 conn(0x7ff1180cecc0 msgr2=0x7ff1180cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:44.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.270+0000 7ff11d284640 1 -- 172.21.15.76:0/2765095337 shutdown_connections 2024-04-25T19:15:44.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.270+0000 7ff11d284640 1 -- 172.21.15.76:0/2765095337 wait complete. 2024-04-25T19:15:44.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.272+0000 7ff11d284640 1 Processor -- start 2024-04-25T19:15:44.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.272+0000 7ff11d284640 1 -- start start 2024-04-25T19:15:44.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.273+0000 7ff11d284640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff11806c5c0 0x7ff1180cad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:44.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.273+0000 7ff11d284640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff118074650 0x7ff1180ce200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:44.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.273+0000 7ff11d284640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1180c96c0 0x7ff118071240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:44.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.273+0000 7ff11d284640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff1180c7330 con 0x7ff1180c96c0 2024-04-25T19:15:44.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.273+0000 7ff11d284640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff1181604f0 con 0x7ff11806c5c0 2024-04-25T19:15:44.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.273+0000 7ff11d284640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff1180ca060 con 0x7ff118074650 2024-04-25T19:15:44.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.273+0000 7ff1177fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1180c96c0 0x7ff118071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:44.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.273+0000 7ff1177fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1180c96c0 0x7ff118071240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39898/0 (socket says 172.21.15.76:39898) 2024-04-25T19:15:44.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.273+0000 7ff1177fe640 1 -- 172.21.15.76:0/3226934208 learned_addr learned my addr 172.21.15.76:0/3226934208 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:44.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.273+0000 7ff10ffff640 1 --2- 172.21.15.76:0/3226934208 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff118074650 0x7ff1180ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:44.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.273+0000 7ff116ffd640 1 --2- 172.21.15.76:0/3226934208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff11806c5c0 0x7ff1180cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:44.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.273+0000 7ff1177fe640 1 -- 172.21.15.76:0/3226934208 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff118074650 msgr2=0x7ff1180ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:44.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.273+0000 7ff1177fe640 1 --2- 172.21.15.76:0/3226934208 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff118074650 0x7ff1180ce200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.273+0000 7ff1177fe640 1 -- 172.21.15.76:0/3226934208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff11806c5c0 msgr2=0x7ff1180cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:44.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.273+0000 7ff1177fe640 1 --2- 172.21.15.76:0/3226934208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff11806c5c0 0x7ff1180cad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.274+0000 7ff1177fe640 1 -- 172.21.15.76:0/3226934208 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1180ce740 con 0x7ff1180c96c0 2024-04-25T19:15:44.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.274+0000 7ff116ffd640 1 --2- 172.21.15.76:0/3226934208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff11806c5c0 0x7ff1180cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:44.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.274+0000 7ff1177fe640 1 --2- 172.21.15.76:0/3226934208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1180c96c0 0x7ff118071240 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff110034040 tx=0x7ff110004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:44.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.274+0000 7ff10f7fe640 1 -- 172.21.15.76:0/3226934208 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff110036050 con 0x7ff1180c96c0 2024-04-25T19:15:44.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.274+0000 7ff10f7fe640 1 -- 172.21.15.76:0/3226934208 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff110042070 con 0x7ff1180c96c0 2024-04-25T19:15:44.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.274+0000 7ff11d284640 1 -- 172.21.15.76:0/3226934208 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1181f2b60 con 0x7ff1180c96c0 2024-04-25T19:15:44.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.274+0000 7ff10f7fe640 1 -- 172.21.15.76:0/3226934208 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff110009510 con 0x7ff1180c96c0 2024-04-25T19:15:44.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.274+0000 7ff11d284640 1 -- 172.21.15.76:0/3226934208 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1180ccff0 con 0x7ff1180c96c0 2024-04-25T19:15:44.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.275+0000 7ff11d284640 1 -- 172.21.15.76:0/3226934208 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff11806a390 con 0x7ff1180c96c0 2024-04-25T19:15:44.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.276+0000 7ff10f7fe640 1 -- 172.21.15.76:0/3226934208 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff110007bb0 con 0x7ff1180c96c0 2024-04-25T19:15:44.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.276+0000 7ff10f7fe640 1 --2- 172.21.15.76:0/3226934208 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff0f80744e0 0x7ff0f80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:44.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.276+0000 7ff10f7fe640 1 -- 172.21.15.76:0/3226934208 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff1100b5910 con 0x7ff1180c96c0 2024-04-25T19:15:44.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.276+0000 7ff116ffd640 1 --2- 172.21.15.76:0/3226934208 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff0f80744e0 0x7ff0f80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:44.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.277+0000 7ff116ffd640 1 --2- 172.21.15.76:0/3226934208 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff0f80744e0 0x7ff0f80769f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff108001660 tx=0x7ff108009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:44.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.281+0000 7ff10f7fe640 1 -- 172.21.15.76:0/3226934208 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff11004b020 con 0x7ff1180c96c0 2024-04-25T19:15:44.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.465+0000 7ff11d284640 1 -- 172.21.15.76:0/3226934208 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff118074430 con 0x7ff1180c96c0 2024-04-25T19:15:44.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.466+0000 7ff10f7fe640 1 -- 172.21.15.76:0/3226934208 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7ff1100801c0 con 0x7ff1180c96c0 2024-04-25T19:15:44.468 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:44.468 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":21,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","created":"2024-04-25T19:15:01.693137+0000","modified":"2024-04-25T19:15:20.401211+0000","last_up_change":"2024-04-25T19:15:11.891681+0000","last_in_change":"2024-04-25T19:15:03.898065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:15:13.846514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:15:16.606533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:15:17.719197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72a16c02-48f9-45aa-9447-fcf755ebf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409574998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409574998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409574998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409574998}]},"public_addr":"172.21.15.76:6817/1409574998","cluster_addr":"172.21.15.76:6819/1409574998","heartbeat_back_addr":"172.21.15.76:6823/1409574998","heartbeat_front_addr":"172.21.15.76:6821/1409574998","state":["exists","up"]},{"osd":1,"uuid":"818d50f0-76c6-45f8-bf79-5e304f14c5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6801","nonce":1302663821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6803","nonce":1302663821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6807","nonce":1302663821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6805","nonce":1302663821}]},"public_addr":"172.21.15.76:6801/1302663821","cluster_addr":"172.21.15.76:6803/1302663821","heartbeat_back_addr":"172.21.15.76:6807/1302663821","heartbeat_front_addr":"172.21.15.76:6805/1302663821","state":["exists","up"]},{"osd":2,"uuid":"5714c8f5-ae23-473c-8cf1-db73811acded","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6809","nonce":1992938181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6811","nonce":1992938181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6815","nonce":1992938181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6813","nonce":1992938181}]},"public_addr":"172.21.15.76:6809/1992938181","cluster_addr":"172.21.15.76:6811/1992938181","heartbeat_back_addr":"172.21.15.76:6815/1992938181","heartbeat_front_addr":"172.21.15.76:6813/1992938181","state":["exists","up"]},{"osd":3,"uuid":"13204cdb-d8e6-4f97-bc6e-9561f442decf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6825","nonce":3418257163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6827","nonce":3418257163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6831","nonce":3418257163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6829","nonce":3418257163}]},"public_addr":"172.21.15.76:6825/3418257163","cluster_addr":"172.21.15.76:6827/3418257163","heartbeat_back_addr":"172.21.15.76:6831/3418257163","heartbeat_front_addr":"172.21.15.76:6829/3418257163","state":["exists","up"]},{"osd":4,"uuid":"5bea5166-1986-4d56-b5f5-ecc8cbe7f3ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6801","nonce":2125002468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6803","nonce":2125002468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6807","nonce":2125002468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6805","nonce":2125002468}]},"public_addr":"172.21.15.105:6801/2125002468","cluster_addr":"172.21.15.105:6803/2125002468","heartbeat_back_addr":"172.21.15.105:6807/2125002468","heartbeat_front_addr":"172.21.15.105:6805/2125002468","state":["exists","up"]},{"osd":5,"uuid":"96f89dcc-270d-4421-b8e8-3b92ba5f9844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6819","nonce":3413103121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6821","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6823","nonce":3413103121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6831","nonce":3413103121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6828","nonce":3413103121}]},"public_addr":"172.21.15.105:6819/3413103121","cluster_addr":"172.21.15.105:6823/3413103121","heartbeat_back_addr":"172.21.15.105:6831/3413103121","heartbeat_front_addr":"172.21.15.105:6828/3413103121","state":["exists","up"]},{"osd":6,"uuid":"93eb087a-d9ba-4096-b85d-b74d96e819a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6809","nonce":2721832954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6811","nonce":2721832954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6815","nonce":2721832954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6813","nonce":2721832954}]},"public_addr":"172.21.15.105:6809/2721832954","cluster_addr":"172.21.15.105:6811/2721832954","heartbeat_back_addr":"172.21.15.105:6815/2721832954","heartbeat_front_addr":"172.21.15.105:6813/2721832954","state":["exists","up"]},{"osd":7,"uuid":"2f80f790-ad2d-460e-a37d-47bf1771ef43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6818","nonce":548350430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6822","nonce":548350430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6829","nonce":548350430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6825","nonce":548350430}]},"public_addr":"172.21.15.105:6818/548350430","cluster_addr":"172.21.15.105:6822/548350430","heartbeat_back_addr":"172.21.15.105:6829/548350430","heartbeat_front_addr":"172.21.15.105:6825/548350430","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.106901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.072819+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.126865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.053464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:08.982614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.104825+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.112714+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.107387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:15:44.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.468+0000 7ff11d284640 1 -- 172.21.15.76:0/3226934208 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff0f80744e0 msgr2=0x7ff0f80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:44.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.468+0000 7ff11d284640 1 --2- 172.21.15.76:0/3226934208 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff0f80744e0 0x7ff0f80769f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff108001660 tx=0x7ff108009040 comp rx=0 tx=0).stop 2024-04-25T19:15:44.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.468+0000 7ff11d284640 1 -- 172.21.15.76:0/3226934208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1180c96c0 msgr2=0x7ff118071240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:44.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.468+0000 7ff11d284640 1 --2- 172.21.15.76:0/3226934208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1180c96c0 0x7ff118071240 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff110034040 tx=0x7ff110004a40 comp rx=0 tx=0).stop 2024-04-25T19:15:44.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.468+0000 7ff11d284640 1 -- 172.21.15.76:0/3226934208 shutdown_connections 2024-04-25T19:15:44.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.468+0000 7ff11d284640 1 --2- 172.21.15.76:0/3226934208 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff0f80744e0 0x7ff0f80769f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.468+0000 7ff11d284640 1 --2- 172.21.15.76:0/3226934208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1180c96c0 0x7ff118071240 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.468+0000 7ff11d284640 1 --2- 172.21.15.76:0/3226934208 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff118074650 0x7ff1180ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.468+0000 7ff11d284640 1 --2- 172.21.15.76:0/3226934208 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff11806c5c0 0x7ff1180cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.468+0000 7ff11d284640 1 -- 172.21.15.76:0/3226934208 >> 172.21.15.76:0/3226934208 conn(0x7ff1180cecc0 msgr2=0x7ff118154fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:44.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.469+0000 7ff11d284640 1 -- 172.21.15.76:0/3226934208 shutdown_connections 2024-04-25T19:15:44.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.469+0000 7ff11d284640 1 -- 172.21.15.76:0/3226934208 wait complete. 2024-04-25T19:15:44.482 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T19:15:13.846514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T19:15:16.606533+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T19:15:17.719197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T19:15:44.482 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:15:44.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.592+0000 7f6dda12d640 1 Processor -- start 2024-04-25T19:15:44.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.592+0000 7f6dda12d640 1 -- start start 2024-04-25T19:15:44.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.593+0000 7f6dda12d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6dd4153910 0x7f6dd415d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:44.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.594+0000 7f6dda12d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6dd4071130 0x7f6dd40ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:44.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.594+0000 7f6dda12d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dd4071540 con 0x7f6dd40cbb80 2024-04-25T19:15:44.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.594+0000 7f6dda12d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f6dd415d0c0 con 0x7f6dd4071130 2024-04-25T19:15:44.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.594+0000 7f6dda12d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f6dd40c6b50 con 0x7f6dd4153910 2024-04-25T19:15:44.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.594+0000 7f6dd3fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6dd4071130 0x7f6dd40ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:44.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.594+0000 7f6dd3fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6dd4071130 0x7f6dd40ca7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39908/0 (socket says 172.21.15.76:39908) 2024-04-25T19:15:44.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.594+0000 7f6dd3fff640 1 -- 172.21.15.76:0/3744510108 learned_addr learned my addr 172.21.15.76:0/3744510108 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:44.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.594+0000 7f6dd37fe640 1 --2- 172.21.15.76:0/3744510108 >> v2:172.21.15.105:3300/0 conn(0x7f6dd4153910 0x7f6dd415d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:44.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.594+0000 7f6dd3fff640 1 -- 172.21.15.76:0/3744510108 >> v2:172.21.15.105:3300/0 conn(0x7f6dd4153910 msgr2=0x7f6dd415d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:44.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.594+0000 7f6dd3fff640 1 --2- 172.21.15.76:0/3744510108 >> v2:172.21.15.105:3300/0 conn(0x7f6dd4153910 0x7f6dd415d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.594+0000 7f6dd3fff640 1 -- 172.21.15.76:0/3744510108 >> v1:172.21.15.76:6789/0 conn(0x7f6dd40cbb80 legacy=0x7f6dd4070a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:44.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.594+0000 7f6dd3fff640 1 -- 172.21.15.76:0/3744510108 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd4071d30 con 0x7f6dd4071130 2024-04-25T19:15:44.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.594+0000 7f6dd37fe640 1 --2- 172.21.15.76:0/3744510108 >> v2:172.21.15.105:3300/0 conn(0x7f6dd4153910 0x7f6dd415d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:15:44.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.595+0000 7f6dd3fff640 1 --2- 172.21.15.76:0/3744510108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dd4071130 0x7f6dd40ca7c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4074410 tx=0x7f6dcc02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab3f6c06f4b8ad7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:44.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.595+0000 7f6dd2ffd640 1 -- 172.21.15.76:0/3744510108 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc036070 con 0x7f6dd4071130 2024-04-25T19:15:44.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.595+0000 7f6dd2ffd640 1 -- 172.21.15.76:0/3744510108 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dcc031600 con 0x7f6dd4071130 2024-04-25T19:15:44.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.595+0000 7f6dd2ffd640 1 -- 172.21.15.76:0/3744510108 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc0319f0 con 0x7f6dd4071130 2024-04-25T19:15:44.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.595+0000 7f6dda12d640 1 -- 172.21.15.76:0/3744510108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dd4071130 msgr2=0x7f6dd40ca7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:44.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.595+0000 7f6dda12d640 1 --2- 172.21.15.76:0/3744510108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dd4071130 0x7f6dd40ca7c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4074410 tx=0x7f6dcc02a7c0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.596+0000 7f6dda12d640 1 -- 172.21.15.76:0/3744510108 shutdown_connections 2024-04-25T19:15:44.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.596+0000 7f6dda12d640 1 --2- 172.21.15.76:0/3744510108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dd4071130 0x7f6dd40ca7c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.596+0000 7f6dda12d640 1 --2- 172.21.15.76:0/3744510108 >> v2:172.21.15.105:3300/0 conn(0x7f6dd4153910 0x7f6dd415d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.596+0000 7f6dda12d640 1 -- 172.21.15.76:0/3744510108 >> 172.21.15.76:0/3744510108 conn(0x7f6dd40cfdc0 msgr2=0x7f6dd40d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:44.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.596+0000 7f6dda12d640 1 -- 172.21.15.76:0/3744510108 shutdown_connections 2024-04-25T19:15:44.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.596+0000 7f6dda12d640 1 -- 172.21.15.76:0/3744510108 wait complete. 2024-04-25T19:15:44.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.597+0000 7f6dda12d640 1 Processor -- start 2024-04-25T19:15:44.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.597+0000 7f6dda12d640 1 -- start start 2024-04-25T19:15:44.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.598+0000 7f6dda12d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6dd4071130 0x7f6dd415e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:44.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.598+0000 7f6dda12d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dd40cbb80 0x7f6dd4160230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:44.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.598+0000 7f6dda12d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6dd4153910 0x7f6dd414c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:44.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.598+0000 7f6dda12d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd40c6e10 con 0x7f6dd40cbb80 2024-04-25T19:15:44.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.598+0000 7f6dda12d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd415cd50 con 0x7f6dd4071130 2024-04-25T19:15:44.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.598+0000 7f6dda12d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f6dd415ca00 con 0x7f6dd4153910 2024-04-25T19:15:44.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.599+0000 7f6dcbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dd40cbb80 0x7f6dd4160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:44.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.599+0000 7f6dcbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dd40cbb80 0x7f6dd4160230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39918/0 (socket says 172.21.15.76:39918) 2024-04-25T19:15:44.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.599+0000 7f6dcbfff640 1 -- 172.21.15.76:0/1523843333 learned_addr learned my addr 172.21.15.76:0/1523843333 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:44.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.599+0000 7f6dd37fe640 1 --2- 172.21.15.76:0/1523843333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6dd4071130 0x7f6dd415e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:44.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.599+0000 7f6dd3fff640 1 --2- 172.21.15.76:0/1523843333 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6dd4153910 0x7f6dd414c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:44.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.599+0000 7f6dcbfff640 1 -- 172.21.15.76:0/1523843333 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6dd4153910 msgr2=0x7f6dd414c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:44.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.599+0000 7f6dcbfff640 1 --2- 172.21.15.76:0/1523843333 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6dd4153910 0x7f6dd414c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.599+0000 7f6dcbfff640 1 -- 172.21.15.76:0/1523843333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6dd4071130 msgr2=0x7f6dd415e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:44.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.599+0000 7f6dcbfff640 1 --2- 172.21.15.76:0/1523843333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6dd4071130 0x7f6dd415e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.599+0000 7f6dcbfff640 1 -- 172.21.15.76:0/1523843333 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd4160770 con 0x7f6dd40cbb80 2024-04-25T19:15:44.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.599+0000 7f6dcbfff640 1 --2- 172.21.15.76:0/1523843333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dd40cbb80 0x7f6dd4160230 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc00c3f0 tx=0x7f6dbc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:44.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.599+0000 7f6dd0ff9640 1 -- 172.21.15.76:0/1523843333 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc0052d0 con 0x7f6dd40cbb80 2024-04-25T19:15:44.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.599+0000 7f6dd0ff9640 1 -- 172.21.15.76:0/1523843333 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dbc005490 con 0x7f6dd40cbb80 2024-04-25T19:15:44.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.599+0000 7f6dda12d640 1 -- 172.21.15.76:0/1523843333 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dd41f3ad0 con 0x7f6dd40cbb80 2024-04-25T19:15:44.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.599+0000 7f6dd0ff9640 1 -- 172.21.15.76:0/1523843333 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc007260 con 0x7f6dd40cbb80 2024-04-25T19:15:44.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.599+0000 7f6dda12d640 1 -- 172.21.15.76:0/1523843333 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dd415fa30 con 0x7f6dd40cbb80 2024-04-25T19:15:44.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.600+0000 7f6dda12d640 1 -- 172.21.15.76:0/1523843333 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dd406a390 con 0x7f6dd40cbb80 2024-04-25T19:15:44.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.601+0000 7f6dd0ff9640 1 -- 172.21.15.76:0/1523843333 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6dbc007400 con 0x7f6dd40cbb80 2024-04-25T19:15:44.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.601+0000 7f6dd0ff9640 1 --2- 172.21.15.76:0/1523843333 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f6db40744e0 0x7f6db40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:44.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.602+0000 7f6dd0ff9640 1 -- 172.21.15.76:0/1523843333 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6dbc094660 con 0x7f6dd40cbb80 2024-04-25T19:15:44.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.602+0000 7f6dd37fe640 1 --2- 172.21.15.76:0/1523843333 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f6db40744e0 0x7f6db40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:44.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.602+0000 7f6dd37fe640 1 --2- 172.21.15.76:0/1523843333 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f6db40744e0 0x7f6db40769f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400b840 tx=0x7f6dc4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:44.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.606+0000 7f6dd0ff9640 1 -- 172.21.15.76:0/1523843333 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6dd4071540 con 0x7f6dd40cbb80 2024-04-25T19:15:44.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.788+0000 7f6dda12d640 1 -- 172.21.15.76:0/1523843333 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6dd41f3860 con 0x7f6dd40cbb80 2024-04-25T19:15:44.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.789+0000 7f6dd0ff9640 1 -- 172.21.15.76:0/1523843333 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f6dbc065e20 con 0x7f6dd40cbb80 2024-04-25T19:15:44.791 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-04-25T19:15:44.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.791+0000 7f6dda12d640 1 -- 172.21.15.76:0/1523843333 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f6db40744e0 msgr2=0x7f6db40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:44.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.791+0000 7f6dda12d640 1 --2- 172.21.15.76:0/1523843333 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f6db40744e0 0x7f6db40769f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400b840 tx=0x7f6dc4008040 comp rx=0 tx=0).stop 2024-04-25T19:15:44.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.791+0000 7f6dda12d640 1 -- 172.21.15.76:0/1523843333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dd40cbb80 msgr2=0x7f6dd4160230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:44.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.791+0000 7f6dda12d640 1 --2- 172.21.15.76:0/1523843333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dd40cbb80 0x7f6dd4160230 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc00c3f0 tx=0x7f6dbc00c8b0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.791+0000 7f6dda12d640 1 -- 172.21.15.76:0/1523843333 shutdown_connections 2024-04-25T19:15:44.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.791+0000 7f6dda12d640 1 --2- 172.21.15.76:0/1523843333 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f6db40744e0 0x7f6db40769f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.791+0000 7f6dda12d640 1 --2- 172.21.15.76:0/1523843333 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f6dd4153910 0x7f6dd414c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.791+0000 7f6dda12d640 1 --2- 172.21.15.76:0/1523843333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6dd40cbb80 0x7f6dd4160230 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.791+0000 7f6dda12d640 1 --2- 172.21.15.76:0/1523843333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6dd4071130 0x7f6dd415e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.791+0000 7f6dda12d640 1 -- 172.21.15.76:0/1523843333 >> 172.21.15.76:0/1523843333 conn(0x7f6dd40cfdc0 msgr2=0x7f6dd4151c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:44.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.791+0000 7f6dda12d640 1 -- 172.21.15.76:0/1523843333 shutdown_connections 2024-04-25T19:15:44.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.791+0000 7f6dda12d640 1 -- 172.21.15.76:0/1523843333 wait complete. 2024-04-25T19:15:44.804 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T19:15:44.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.913+0000 7f49b42e0640 1 Processor -- start 2024-04-25T19:15:44.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.913+0000 7f49b42e0640 1 -- start start 2024-04-25T19:15:44.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.915+0000 7f49b42e0640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f49ac159840 0x7f49ac153b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:44.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.915+0000 7f49b42e0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49ac0715d0 con 0x7f49ac15d880 2024-04-25T19:15:44.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.915+0000 7f49b42e0640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49ac0718b0 con 0x7f49ac0c5b20 2024-04-25T19:15:44.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.915+0000 7f49b42e0640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f49ac15ce30 con 0x7f49ac159840 2024-04-25T19:15:44.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.915+0000 7f49b287d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f49ac15d880 0x7f49ac0c88f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53572/0 (socket says 172.21.15.76:53572) 2024-04-25T19:15:44.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.915+0000 7f49b287d640 1 -- 172.21.15.76:0/182621442 learned_addr learned my addr 172.21.15.76:0/182621442 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:44.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.915+0000 7f49b207c640 1 --2- 172.21.15.76:0/182621442 >> v2:172.21.15.105:3300/0 conn(0x7f49ac159840 0x7f49ac153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:44.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.915+0000 7f49b107a640 1 -- 172.21.15.76:0/182621442 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3175287386 0 0) 0x7f49ac0715d0 con 0x7f49ac15d880 2024-04-25T19:15:44.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.915+0000 7f49b107a640 1 -- 172.21.15.76:0/182621442 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4998003520 con 0x7f49ac15d880 2024-04-25T19:15:44.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.916+0000 7f49b107a640 1 -- 172.21.15.76:0/182621442 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 621000844 0 0) 0x7f4998003520 con 0x7f49ac15d880 2024-04-25T19:15:44.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.916+0000 7f49b107a640 1 -- 172.21.15.76:0/182621442 >> v2:172.21.15.105:3300/0 conn(0x7f49ac159840 msgr2=0x7f49ac153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:44.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.916+0000 7f49b107a640 1 --2- 172.21.15.76:0/182621442 >> v2:172.21.15.105:3300/0 conn(0x7f49ac159840 0x7f49ac153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.916+0000 7f49b107a640 1 -- 172.21.15.76:0/182621442 >> v1:172.21.15.105:6789/0 conn(0x7f49ac0c5b20 legacy=0x7f49ac0c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:44.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.916+0000 7f49b107a640 1 -- 172.21.15.76:0/182621442 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49ac071b90 con 0x7f49ac15d880 2024-04-25T19:15:44.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.916+0000 7f49b107a640 1 -- 172.21.15.76:0/182621442 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f49a40019f0 con 0x7f49ac15d880 2024-04-25T19:15:44.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.916+0000 7f49b107a640 1 -- 172.21.15.76:0/182621442 >> v1:172.21.15.76:6789/0 conn(0x7f49ac15d880 legacy=0x7f49ac0c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:44.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.916+0000 7f49b107a640 1 --2- 172.21.15.76:0/182621442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49980039b0 0x7f4998003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:44.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.916+0000 7f49b107a640 1 --2- 172.21.15.76:0/182621442 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4998004800 0x7f4998004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:44.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.916+0000 7f49b107a640 1 --2- 172.21.15.76:0/182621442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4998005150 0x7f499800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:44.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.916+0000 7f49b107a640 1 -- 172.21.15.76:0/182621442 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac15ce30 con 0x7f4998005150 2024-04-25T19:15:44.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.916+0000 7f49b107a640 1 -- 172.21.15.76:0/182621442 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f499800c0c0 con 0x7f49980039b0 2024-04-25T19:15:44.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.916+0000 7f49b107a640 1 -- 172.21.15.76:0/182621442 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f499800c2c0 con 0x7f4998004800 2024-04-25T19:15:44.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.916+0000 7f49b287d640 1 --2- 172.21.15.76:0/182621442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4998005150 0x7f499800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:44.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.916+0000 7f49b207c640 1 --2- 172.21.15.76:0/182621442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49980039b0 0x7f4998003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:44.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.916+0000 7f49b187b640 1 --2- 172.21.15.76:0/182621442 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4998004800 0x7f4998004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:44.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.916+0000 7f49b207c640 1 -- 172.21.15.76:0/182621442 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4998004800 msgr2=0x7f4998004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:44.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.916+0000 7f49b207c640 1 --2- 172.21.15.76:0/182621442 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4998004800 0x7f4998004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.916+0000 7f49b207c640 1 -- 172.21.15.76:0/182621442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4998005150 msgr2=0x7f499800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:44.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.916+0000 7f49b207c640 1 --2- 172.21.15.76:0/182621442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4998005150 0x7f499800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.916+0000 7f49b207c640 1 -- 172.21.15.76:0/182621442 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4998003520 con 0x7f49980039b0 2024-04-25T19:15:44.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.916+0000 7f49b287d640 1 --2- 172.21.15.76:0/182621442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4998005150 0x7f499800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:44.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.916+0000 7f49b287d640 1 -- 172.21.15.76:0/182621442 reap_dead start 2024-04-25T19:15:44.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.917+0000 7f49b207c640 1 --2- 172.21.15.76:0/182621442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49980039b0 0x7f4998003de0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f499c009ac0 tx=0x7f4998005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91acaaf8f76937f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:44.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.917+0000 7f49b107a640 1 -- 172.21.15.76:0/182621442 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c036070 con 0x7f49980039b0 2024-04-25T19:15:44.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.917+0000 7f49b107a640 1 -- 172.21.15.76:0/182621442 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f499c031400 con 0x7f49980039b0 2024-04-25T19:15:44.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.917+0000 7f49b107a640 1 -- 172.21.15.76:0/182621442 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c009690 con 0x7f49980039b0 2024-04-25T19:15:44.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.918+0000 7f49b42e0640 1 -- 172.21.15.76:0/182621442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49980039b0 msgr2=0x7f4998003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:44.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.918+0000 7f49b42e0640 1 --2- 172.21.15.76:0/182621442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49980039b0 0x7f4998003de0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f499c009ac0 tx=0x7f4998005830 comp rx=0 tx=0).stop 2024-04-25T19:15:44.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.918+0000 7f49b42e0640 1 -- 172.21.15.76:0/182621442 shutdown_connections 2024-04-25T19:15:44.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.918+0000 7f49b42e0640 1 --2- 172.21.15.76:0/182621442 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49980039b0 0x7f4998003de0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.918+0000 7f49b42e0640 1 -- 172.21.15.76:0/182621442 >> 172.21.15.76:0/182621442 conn(0x7f49ac0cecc0 msgr2=0x7f49ac0cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:44.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.918+0000 7f49b42e0640 1 -- 172.21.15.76:0/182621442 shutdown_connections 2024-04-25T19:15:44.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.918+0000 7f49b42e0640 1 -- 172.21.15.76:0/182621442 wait complete. 2024-04-25T19:15:44.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.920+0000 7f49b42e0640 1 Processor -- start 2024-04-25T19:15:44.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.920+0000 7f49b42e0640 1 -- start start 2024-04-25T19:15:44.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.921+0000 7f49b42e0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49980039b0 0x7f49ac152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:44.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.921+0000 7f49b42e0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49ac1488a0 0x7f49ac155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:44.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.921+0000 7f49b42e0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49ac148cb0 0x7f49ac154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:44.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.921+0000 7f49b42e0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac15dc90 con 0x7f49980039b0 2024-04-25T19:15:44.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.921+0000 7f49b42e0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac071270 con 0x7f49ac1488a0 2024-04-25T19:15:44.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.921+0000 7f49b42e0640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f49ac153b70 con 0x7f49ac148cb0 2024-04-25T19:15:44.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.921+0000 7f49b207c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49980039b0 0x7f49ac152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:44.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.921+0000 7f49b207c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49980039b0 0x7f49ac152590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39928/0 (socket says 172.21.15.76:39928) 2024-04-25T19:15:44.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.921+0000 7f49b207c640 1 -- 172.21.15.76:0/3649138198 learned_addr learned my addr 172.21.15.76:0/3649138198 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:44.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.921+0000 7f49b187b640 1 --2- 172.21.15.76:0/3649138198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49ac1488a0 0x7f49ac155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:44.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.921+0000 7f49b287d640 1 --2- 172.21.15.76:0/3649138198 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49ac148cb0 0x7f49ac154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:44.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.921+0000 7f49b207c640 1 -- 172.21.15.76:0/3649138198 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49ac148cb0 msgr2=0x7f49ac154b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:44.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.921+0000 7f49b207c640 1 --2- 172.21.15.76:0/3649138198 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49ac148cb0 0x7f49ac154b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.921+0000 7f49b207c640 1 -- 172.21.15.76:0/3649138198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49ac1488a0 msgr2=0x7f49ac155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:44.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.921+0000 7f49b207c640 1 --2- 172.21.15.76:0/3649138198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49ac1488a0 0x7f49ac155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:44.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.921+0000 7f49b207c640 1 -- 172.21.15.76:0/3649138198 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49ac152ad0 con 0x7f49980039b0 2024-04-25T19:15:44.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.921+0000 7f49b207c640 1 --2- 172.21.15.76:0/3649138198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49980039b0 0x7f49ac152590 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f499c034040 tx=0x7f499c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:44.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.922+0000 7f49a2ffd640 1 -- 172.21.15.76:0/3649138198 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c03f020 con 0x7f49980039b0 2024-04-25T19:15:44.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.922+0000 7f49a2ffd640 1 -- 172.21.15.76:0/3649138198 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f499c037040 con 0x7f49980039b0 2024-04-25T19:15:44.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.922+0000 7f49b42e0640 1 -- 172.21.15.76:0/3649138198 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49ac151720 con 0x7f49980039b0 2024-04-25T19:15:44.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.922+0000 7f49a2ffd640 1 -- 172.21.15.76:0/3649138198 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c036070 con 0x7f49980039b0 2024-04-25T19:15:44.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.922+0000 7f49b42e0640 1 -- 172.21.15.76:0/3649138198 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49ac156e60 con 0x7f49980039b0 2024-04-25T19:15:44.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.923+0000 7f49b42e0640 1 -- 172.21.15.76:0/3649138198 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49ac06a390 con 0x7f49980039b0 2024-04-25T19:15:44.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.923+0000 7f49a2ffd640 1 -- 172.21.15.76:0/3649138198 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f499c042020 con 0x7f49980039b0 2024-04-25T19:15:44.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.924+0000 7f49a2ffd640 1 --2- 172.21.15.76:0/3649138198 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f49900744b0 0x7f49900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:44.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.924+0000 7f49a2ffd640 1 -- 172.21.15.76:0/3649138198 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f499c0b4f90 con 0x7f49980039b0 2024-04-25T19:15:44.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.924+0000 7f49b187b640 1 --2- 172.21.15.76:0/3649138198 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f49900744b0 0x7f49900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:44.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.924+0000 7f49b187b640 1 --2- 172.21.15.76:0/3649138198 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f49900744b0 0x7f49900769c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f49ac0c72d0 tx=0x7f49a8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:44.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:44.928+0000 7f49a2ffd640 1 -- 172.21.15.76:0/3649138198 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f499c049030 con 0x7f49980039b0 2024-04-25T19:15:45.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.110+0000 7f49b42e0640 1 -- 172.21.15.76:0/3649138198 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f49ac0c6960 con 0x7f49980039b0 2024-04-25T19:15:45.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.111+0000 7f49a2ffd640 1 -- 172.21.15.76:0/3649138198 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f499c07f160 con 0x7f49980039b0 2024-04-25T19:15:45.113 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-04-25T19:15:45.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.113+0000 7f49b42e0640 1 -- 172.21.15.76:0/3649138198 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f49900744b0 msgr2=0x7f49900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.113+0000 7f49b42e0640 1 --2- 172.21.15.76:0/3649138198 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f49900744b0 0x7f49900769c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f49ac0c72d0 tx=0x7f49a8009040 comp rx=0 tx=0).stop 2024-04-25T19:15:45.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.113+0000 7f49b42e0640 1 -- 172.21.15.76:0/3649138198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49980039b0 msgr2=0x7f49ac152590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.113+0000 7f49b42e0640 1 --2- 172.21.15.76:0/3649138198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49980039b0 0x7f49ac152590 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f499c034040 tx=0x7f499c004730 comp rx=0 tx=0).stop 2024-04-25T19:15:45.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.113+0000 7f49b42e0640 1 -- 172.21.15.76:0/3649138198 shutdown_connections 2024-04-25T19:15:45.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.113+0000 7f49b42e0640 1 --2- 172.21.15.76:0/3649138198 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f49900744b0 0x7f49900769c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.113+0000 7f49b42e0640 1 --2- 172.21.15.76:0/3649138198 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f49ac148cb0 0x7f49ac154b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.113+0000 7f49b42e0640 1 --2- 172.21.15.76:0/3649138198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f49ac1488a0 0x7f49ac155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.113+0000 7f49b42e0640 1 --2- 172.21.15.76:0/3649138198 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49980039b0 0x7f49ac152590 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.113+0000 7f49b42e0640 1 -- 172.21.15.76:0/3649138198 >> 172.21.15.76:0/3649138198 conn(0x7f49ac0cecc0 msgr2=0x7f49ac0c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:45.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.113+0000 7f49b42e0640 1 -- 172.21.15.76:0/3649138198 shutdown_connections 2024-04-25T19:15:45.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.113+0000 7f49b42e0640 1 -- 172.21.15.76:0/3649138198 wait complete. 2024-04-25T19:15:45.126 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T19:15:45.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.236+0000 7f1457a20640 1 Processor -- start 2024-04-25T19:15:45.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.236+0000 7f1457a20640 1 -- start start 2024-04-25T19:15:45.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.237+0000 7f1457a20640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1450159840 0x7f1450153b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.237+0000 7f1457a20640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14500c5b20 0x7f145015d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.237+0000 7f1457a20640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f14500c96c0 0x7f14500c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.237+0000 7f1457a20640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f145015ce30 con 0x7f14500c96c0 2024-04-25T19:15:45.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.237+0000 7f1457a20640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f145015ddc0 con 0x7f1450159840 2024-04-25T19:15:45.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.237+0000 7f1457a20640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f14500715d0 con 0x7f14500c5b20 2024-04-25T19:15:45.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.238+0000 7f1455fbd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f14500c96c0 0x7f14500c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.238+0000 7f1455fbd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f14500c96c0 0x7f14500c88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39944/0 (socket says 172.21.15.76:39944) 2024-04-25T19:15:45.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.238+0000 7f1455fbd640 1 -- 172.21.15.76:0/3264313877 learned_addr learned my addr 172.21.15.76:0/3264313877 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:45.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.238+0000 7f14557bc640 1 --2- 172.21.15.76:0/3264313877 >> v2:172.21.15.105:3300/0 conn(0x7f1450159840 0x7f1450153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.238+0000 7f1454fbb640 1 --2- 172.21.15.76:0/3264313877 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14500c5b20 0x7f145015d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.238+0000 7f1455fbd640 1 -- 172.21.15.76:0/3264313877 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14500c5b20 msgr2=0x7f145015d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:45.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.238+0000 7f1455fbd640 1 --2- 172.21.15.76:0/3264313877 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14500c5b20 0x7f145015d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.238+0000 7f1455fbd640 1 -- 172.21.15.76:0/3264313877 >> v2:172.21.15.105:3300/0 conn(0x7f1450159840 msgr2=0x7f1450153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:45.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.238+0000 7f1455fbd640 1 --2- 172.21.15.76:0/3264313877 >> v2:172.21.15.105:3300/0 conn(0x7f1450159840 0x7f1450153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.238+0000 7f1455fbd640 1 -- 172.21.15.76:0/3264313877 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14500718b0 con 0x7f14500c96c0 2024-04-25T19:15:45.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.239+0000 7f1455fbd640 1 --2- 172.21.15.76:0/3264313877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14500c96c0 0x7f14500c88f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1448009e50 tx=0x7f144802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d672d9ab42991a89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:45.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.239+0000 7f1447fff640 1 -- 172.21.15.76:0/3264313877 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1448036070 con 0x7f14500c96c0 2024-04-25T19:15:45.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.239+0000 7f1447fff640 1 -- 172.21.15.76:0/3264313877 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1448031600 con 0x7f14500c96c0 2024-04-25T19:15:45.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.239+0000 7f1447fff640 1 -- 172.21.15.76:0/3264313877 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14480319d0 con 0x7f14500c96c0 2024-04-25T19:15:45.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.240+0000 7f1457a20640 1 -- 172.21.15.76:0/3264313877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14500c96c0 msgr2=0x7f14500c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.240+0000 7f1457a20640 1 --2- 172.21.15.76:0/3264313877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14500c96c0 0x7f14500c88f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1448009e50 tx=0x7f144802a6c0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.240+0000 7f1457a20640 1 -- 172.21.15.76:0/3264313877 shutdown_connections 2024-04-25T19:15:45.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.240+0000 7f1457a20640 1 --2- 172.21.15.76:0/3264313877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14500c96c0 0x7f14500c88f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.240+0000 7f1457a20640 1 --2- 172.21.15.76:0/3264313877 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14500c5b20 0x7f145015d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.240+0000 7f1457a20640 1 --2- 172.21.15.76:0/3264313877 >> v2:172.21.15.105:3300/0 conn(0x7f1450159840 0x7f1450153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.240+0000 7f1457a20640 1 -- 172.21.15.76:0/3264313877 >> 172.21.15.76:0/3264313877 conn(0x7f14500cecc0 msgr2=0x7f14500cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:45.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.240+0000 7f1457a20640 1 -- 172.21.15.76:0/3264313877 shutdown_connections 2024-04-25T19:15:45.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.240+0000 7f1457a20640 1 -- 172.21.15.76:0/3264313877 wait complete. 2024-04-25T19:15:45.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.242+0000 7f1457a20640 1 Processor -- start 2024-04-25T19:15:45.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.242+0000 7f1457a20640 1 -- start start 2024-04-25T19:15:45.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.243+0000 7f1457a20640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14500c5b20 0x7f14500ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.243+0000 7f1457a20640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14500c96c0 0x7f14501e6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.243+0000 7f1457a20640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1450159840 0x7f1450152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.243+0000 7f1457a20640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1450071270 con 0x7f14500c96c0 2024-04-25T19:15:45.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.243+0000 7f1457a20640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f145015d080 con 0x7f1450159840 2024-04-25T19:15:45.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.243+0000 7f1457a20640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f14500c9ad0 con 0x7f14500c5b20 2024-04-25T19:15:45.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.243+0000 7f1454fbb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14500c96c0 0x7f14501e6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.243+0000 7f1454fbb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14500c96c0 0x7f14501e6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39956/0 (socket says 172.21.15.76:39956) 2024-04-25T19:15:45.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.243+0000 7f1454fbb640 1 -- 172.21.15.76:0/1474500733 learned_addr learned my addr 172.21.15.76:0/1474500733 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:45.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.243+0000 7f14557bc640 1 --2- 172.21.15.76:0/1474500733 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14500c5b20 0x7f14500ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.243+0000 7f1455fbd640 1 --2- 172.21.15.76:0/1474500733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1450159840 0x7f1450152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.243+0000 7f1454fbb640 1 -- 172.21.15.76:0/1474500733 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14500c5b20 msgr2=0x7f14500ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.243+0000 7f1454fbb640 1 --2- 172.21.15.76:0/1474500733 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14500c5b20 0x7f14500ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.243+0000 7f1454fbb640 1 -- 172.21.15.76:0/1474500733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1450159840 msgr2=0x7f1450152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.243+0000 7f1454fbb640 1 --2- 172.21.15.76:0/1474500733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1450159840 0x7f1450152c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.243+0000 7f1454fbb640 1 -- 172.21.15.76:0/1474500733 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14500cd420 con 0x7f14500c96c0 2024-04-25T19:15:45.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.243+0000 7f1454fbb640 1 --2- 172.21.15.76:0/1474500733 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14500c96c0 0x7f14501e6fa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f144c00d830 tx=0x7f144c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:45.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.244+0000 7f1445ffb640 1 -- 172.21.15.76:0/1474500733 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c018030 con 0x7f14500c96c0 2024-04-25T19:15:45.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.244+0000 7f1445ffb640 1 -- 172.21.15.76:0/1474500733 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f144c00be10 con 0x7f14500c96c0 2024-04-25T19:15:45.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.244+0000 7f1457a20640 1 -- 172.21.15.76:0/1474500733 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14501e7790 con 0x7f14500c96c0 2024-04-25T19:15:45.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.244+0000 7f1445ffb640 1 -- 172.21.15.76:0/1474500733 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c0098b0 con 0x7f14500c96c0 2024-04-25T19:15:45.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.244+0000 7f1457a20640 1 -- 172.21.15.76:0/1474500733 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f145015b4d0 con 0x7f14500c96c0 2024-04-25T19:15:45.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.245+0000 7f1457a20640 1 -- 172.21.15.76:0/1474500733 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14501e7a50 con 0x7f14500c96c0 2024-04-25T19:15:45.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.245+0000 7f1445ffb640 1 -- 172.21.15.76:0/1474500733 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f144c01b070 con 0x7f14500c96c0 2024-04-25T19:15:45.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.246+0000 7f1445ffb640 1 --2- 172.21.15.76:0/1474500733 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f14340744b0 0x7f14340769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.246+0000 7f1445ffb640 1 -- 172.21.15.76:0/1474500733 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f144c095180 con 0x7f14500c96c0 2024-04-25T19:15:45.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.246+0000 7f14557bc640 1 --2- 172.21.15.76:0/1474500733 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f14340744b0 0x7f14340769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.246+0000 7f14557bc640 1 --2- 172.21.15.76:0/1474500733 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f14340744b0 0x7f14340769c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f14400079a0 tx=0x7f1440008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:45.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.250+0000 7f1445ffb640 1 -- 172.21.15.76:0/1474500733 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f144c05f3e0 con 0x7f14500c96c0 2024-04-25T19:15:45.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.431+0000 7f1457a20640 1 -- 172.21.15.76:0/1474500733 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1450070c30 con 0x7f14500c96c0 2024-04-25T19:15:45.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.431+0000 7f1445ffb640 1 -- 172.21.15.76:0/1474500733 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f1450070c30 con 0x7f14500c96c0 2024-04-25T19:15:45.433 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-04-25T19:15:45.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.433+0000 7f1457a20640 1 -- 172.21.15.76:0/1474500733 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f14340744b0 msgr2=0x7f14340769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.433+0000 7f1457a20640 1 --2- 172.21.15.76:0/1474500733 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f14340744b0 0x7f14340769c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f14400079a0 tx=0x7f1440008040 comp rx=0 tx=0).stop 2024-04-25T19:15:45.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.433+0000 7f1457a20640 1 -- 172.21.15.76:0/1474500733 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14500c96c0 msgr2=0x7f14501e6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.433+0000 7f1457a20640 1 --2- 172.21.15.76:0/1474500733 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14500c96c0 0x7f14501e6fa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f144c00d830 tx=0x7f144c00dcf0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.433+0000 7f1457a20640 1 -- 172.21.15.76:0/1474500733 shutdown_connections 2024-04-25T19:15:45.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.433+0000 7f1457a20640 1 --2- 172.21.15.76:0/1474500733 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f14340744b0 0x7f14340769c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.433+0000 7f1457a20640 1 --2- 172.21.15.76:0/1474500733 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1450159840 0x7f1450152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.433+0000 7f1457a20640 1 --2- 172.21.15.76:0/1474500733 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14500c96c0 0x7f14501e6fa0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.433+0000 7f1457a20640 1 --2- 172.21.15.76:0/1474500733 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f14500c5b20 0x7f14500ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.433+0000 7f1457a20640 1 -- 172.21.15.76:0/1474500733 >> 172.21.15.76:0/1474500733 conn(0x7f14500cecc0 msgr2=0x7f1450159fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:45.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.433+0000 7f1457a20640 1 -- 172.21.15.76:0/1474500733 shutdown_connections 2024-04-25T19:15:45.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.433+0000 7f1457a20640 1 -- 172.21.15.76:0/1474500733 wait complete. 2024-04-25T19:15:45.446 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:15:45.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.553+0000 7faad4c92640 1 Processor -- start 2024-04-25T19:15:45.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.553+0000 7faad4c92640 1 -- start start 2024-04-25T19:15:45.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.555+0000 7faad4c92640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faad00c96c0 0x7faad015cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.555+0000 7faad4c92640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faad015d1c0 con 0x7faad0074650 2024-04-25T19:15:45.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.555+0000 7faad4c92640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faad015d4a0 con 0x7faad006c5c0 2024-04-25T19:15:45.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.555+0000 7faad4c92640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7faad0161120 con 0x7faad00c96c0 2024-04-25T19:15:45.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.555+0000 7faace59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faad00c96c0 0x7faad015cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.555+0000 7faace59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faad00c96c0 0x7faad015cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39968/0 (socket says 172.21.15.76:39968) 2024-04-25T19:15:45.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.555+0000 7faace59c640 1 -- 172.21.15.76:0/2225664282 learned_addr learned my addr 172.21.15.76:0/2225664282 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:45.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.555+0000 7faace59c640 1 -- 172.21.15.76:0/2225664282 >> v1:172.21.15.105:6789/0 conn(0x7faad006c5c0 legacy=0x7faad0167500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.555+0000 7faace59c640 1 -- 172.21.15.76:0/2225664282 >> v1:172.21.15.76:6789/0 conn(0x7faad0074650 legacy=0x7faad0160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.555+0000 7faace59c640 1 -- 172.21.15.76:0/2225664282 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad015d780 con 0x7faad00c96c0 2024-04-25T19:15:45.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.556+0000 7faace59c640 1 --2- 172.21.15.76:0/2225664282 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faad00c96c0 0x7faad015cb80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faac4009870 tx=0x7faac402a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3c34236c179a34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:45.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.556+0000 7faacd59a640 1 -- 172.21.15.76:0/2225664282 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac4037070 con 0x7faad00c96c0 2024-04-25T19:15:45.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.556+0000 7faacd59a640 1 -- 172.21.15.76:0/2225664282 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faac4032640 con 0x7faad00c96c0 2024-04-25T19:15:45.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.556+0000 7faacd59a640 1 -- 172.21.15.76:0/2225664282 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac4032a10 con 0x7faad00c96c0 2024-04-25T19:15:45.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.557+0000 7faad4c92640 1 -- 172.21.15.76:0/2225664282 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faad00c96c0 msgr2=0x7faad015cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.557+0000 7faad4c92640 1 --2- 172.21.15.76:0/2225664282 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faad00c96c0 0x7faad015cb80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faac4009870 tx=0x7faac402a3c0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.557+0000 7faad4c92640 1 -- 172.21.15.76:0/2225664282 shutdown_connections 2024-04-25T19:15:45.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.557+0000 7faad4c92640 1 --2- 172.21.15.76:0/2225664282 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faad00c96c0 0x7faad015cb80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.557+0000 7faad4c92640 1 -- 172.21.15.76:0/2225664282 >> 172.21.15.76:0/2225664282 conn(0x7faad00cecc0 msgr2=0x7faad00cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:45.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.557+0000 7faad4c92640 1 -- 172.21.15.76:0/2225664282 shutdown_connections 2024-04-25T19:15:45.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.557+0000 7faad4c92640 1 -- 172.21.15.76:0/2225664282 wait complete. 2024-04-25T19:15:45.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.559+0000 7faad4c92640 1 Processor -- start 2024-04-25T19:15:45.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.559+0000 7faad4c92640 1 -- start start 2024-04-25T19:15:45.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.560+0000 7faad4c92640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faad006c5c0 0x7faad014f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.560+0000 7faad4c92640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faad0074650 0x7faad00cde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.560+0000 7faad4c92640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faad00c96c0 0x7faad014c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.560+0000 7faad4c92640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faad00c7330 con 0x7faad0074650 2024-04-25T19:15:45.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.560+0000 7faad4c92640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faad01604f0 con 0x7faad006c5c0 2024-04-25T19:15:45.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.560+0000 7faad4c92640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7faad00ca060 con 0x7faad00c96c0 2024-04-25T19:15:45.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.560+0000 7faacdd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faad0074650 0x7faad00cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.560+0000 7faacdd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faad0074650 0x7faad00cde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39976/0 (socket says 172.21.15.76:39976) 2024-04-25T19:15:45.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.560+0000 7faacdd9b640 1 -- 172.21.15.76:0/1308569252 learned_addr learned my addr 172.21.15.76:0/1308569252 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:45.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.560+0000 7faace59c640 1 --2- 172.21.15.76:0/1308569252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faad006c5c0 0x7faad014f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.560+0000 7faaced9d640 1 --2- 172.21.15.76:0/1308569252 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faad00c96c0 0x7faad014c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.560+0000 7faacdd9b640 1 -- 172.21.15.76:0/1308569252 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faad00c96c0 msgr2=0x7faad014c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.560+0000 7faacdd9b640 1 --2- 172.21.15.76:0/1308569252 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faad00c96c0 0x7faad014c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.560+0000 7faacdd9b640 1 -- 172.21.15.76:0/1308569252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faad006c5c0 msgr2=0x7faad014f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.560+0000 7faacdd9b640 1 --2- 172.21.15.76:0/1308569252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faad006c5c0 0x7faad014f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.560+0000 7faacdd9b640 1 -- 172.21.15.76:0/1308569252 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad014f990 con 0x7faad0074650 2024-04-25T19:15:45.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.560+0000 7faaced9d640 1 --2- 172.21.15.76:0/1308569252 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faad00c96c0 0x7faad014c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:15:45.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.561+0000 7faacdd9b640 1 --2- 172.21.15.76:0/1308569252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faad0074650 0x7faad00cde20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faac000c430 tx=0x7faac000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:45.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.561+0000 7faabeffd640 1 -- 172.21.15.76:0/1308569252 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac0017070 con 0x7faad0074650 2024-04-25T19:15:45.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.561+0000 7faabeffd640 1 -- 172.21.15.76:0/1308569252 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faac00052d0 con 0x7faad0074650 2024-04-25T19:15:45.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.561+0000 7faad4c92640 1 -- 172.21.15.76:0/1308569252 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad0151030 con 0x7faad0074650 2024-04-25T19:15:45.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.561+0000 7faabeffd640 1 -- 172.21.15.76:0/1308569252 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac0003830 con 0x7faad0074650 2024-04-25T19:15:45.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.561+0000 7faad4c92640 1 -- 172.21.15.76:0/1308569252 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad01f33c0 con 0x7faad0074650 2024-04-25T19:15:45.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.562+0000 7faad4c92640 1 -- 172.21.15.76:0/1308569252 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faad006a390 con 0x7faad0074650 2024-04-25T19:15:45.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.562+0000 7faabeffd640 1 -- 172.21.15.76:0/1308569252 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faac001a070 con 0x7faad0074650 2024-04-25T19:15:45.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.563+0000 7faabeffd640 1 --2- 172.21.15.76:0/1308569252 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7faab00744b0 0x7faab00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.563+0000 7faabeffd640 1 -- 172.21.15.76:0/1308569252 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7faac0094460 con 0x7faad0074650 2024-04-25T19:15:45.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.563+0000 7faace59c640 1 --2- 172.21.15.76:0/1308569252 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7faab00744b0 0x7faab00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.564+0000 7faace59c640 1 --2- 172.21.15.76:0/1308569252 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7faab00744b0 0x7faab00769c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faac4034000 tx=0x7faac402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:45.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.567+0000 7faabeffd640 1 -- 172.21.15.76:0/1308569252 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faac005ed10 con 0x7faad0074650 2024-04-25T19:15:45.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.788+0000 7faad4c92640 1 -- 172.21.15.76:0/1308569252 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faad0151360 con 0x7faad0074650 2024-04-25T19:15:45.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.789+0000 7faabeffd640 1 -- 172.21.15.76:0/1308569252 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+6911 (secure 0 0 0) 0x7faac005e610 con 0x7faad0074650 2024-04-25T19:15:45.791 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:45.791 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/3003966182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3003966182},{"type":"v1","addr":"172.21.15.76:6836","nonce":3003966182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T19:15:18.769449+0000","modified":"2024-04-25T19:15:42.486997+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4284,"mds_1":4278,"mds_2":4241,"mds_3":4238,"mds_4":4235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4235":{"gid":4235,"name":"b","rank":4,"incarnation":15,"state":"up:creating","state_seq":1,"addr":"172.21.15.105:6836/1809877606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1809877606},{"type":"v1","addr":"172.21.15.105:6836","nonce":1809877606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4238":{"gid":4238,"name":"f","rank":3,"incarnation":13,"state":"up:active","state_seq":9,"addr":"172.21.15.105:6838/2724142282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2724142282},{"type":"v1","addr":"172.21.15.105:6838","nonce":2724142282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4241":{"gid":4241,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.105:6839/1798232582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6837","nonce":1798232582},{"type":"v1","addr":"172.21.15.105:6839","nonce":1798232582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.76:6833/2101494685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2101494685},{"type":"v1","addr":"172.21.15.76:6833","nonce":2101494685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4284":{"gid":4284,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6837/4226277412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":4226277412},{"type":"v1","addr":"172.21.15.76:6837","nonce":4226277412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284,4278,4241,4238]},"id":1}]} 2024-04-25T19:15:45.791 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 15 2024-04-25T19:15:45.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.791+0000 7faad4c92640 1 -- 172.21.15.76:0/1308569252 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7faab00744b0 msgr2=0x7faab00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.791+0000 7faad4c92640 1 --2- 172.21.15.76:0/1308569252 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7faab00744b0 0x7faab00769c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faac4034000 tx=0x7faac402e000 comp rx=0 tx=0).stop 2024-04-25T19:15:45.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.791+0000 7faad4c92640 1 -- 172.21.15.76:0/1308569252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faad0074650 msgr2=0x7faad00cde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.791+0000 7faad4c92640 1 --2- 172.21.15.76:0/1308569252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faad0074650 0x7faad00cde20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faac000c430 tx=0x7faac000c8f0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.791+0000 7faad4c92640 1 -- 172.21.15.76:0/1308569252 shutdown_connections 2024-04-25T19:15:45.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.791+0000 7faad4c92640 1 --2- 172.21.15.76:0/1308569252 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7faab00744b0 0x7faab00769c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.791+0000 7faad4c92640 1 --2- 172.21.15.76:0/1308569252 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faad00c96c0 0x7faad014c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.791+0000 7faad4c92640 1 --2- 172.21.15.76:0/1308569252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faad0074650 0x7faad00cde20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.791+0000 7faad4c92640 1 --2- 172.21.15.76:0/1308569252 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faad006c5c0 0x7faad014f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.791+0000 7faad4c92640 1 -- 172.21.15.76:0/1308569252 >> 172.21.15.76:0/1308569252 conn(0x7faad00cecc0 msgr2=0x7faad0155200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:45.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.791+0000 7faad4c92640 1 -- 172.21.15.76:0/1308569252 shutdown_connections 2024-04-25T19:15:45.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.791+0000 7faad4c92640 1 -- 172.21.15.76:0/1308569252 wait complete. 2024-04-25T19:15:45.804 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:15:45.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.911+0000 7f7512251640 1 Processor -- start 2024-04-25T19:15:45.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.911+0000 7f7512251640 1 -- start start 2024-04-25T19:15:45.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.913+0000 7f7512251640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f750c153b70 0x7f750c15d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.913+0000 7f7512251640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f750c153f80 con 0x7f750c0c5b20 2024-04-25T19:15:45.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.913+0000 7f7512251640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f750c0715d0 con 0x7f750c159840 2024-04-25T19:15:45.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.913+0000 7f7512251640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f750c15ce30 con 0x7f750c153b70 2024-04-25T19:15:45.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.913+0000 7f750affd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f750c0c5b20 0x7f750c0c88f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53596/0 (socket says 172.21.15.76:53596) 2024-04-25T19:15:45.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.913+0000 7f750affd640 1 -- 172.21.15.76:0/4146222161 learned_addr learned my addr 172.21.15.76:0/4146222161 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:45.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.913+0000 7f750bfff640 1 --2- 172.21.15.76:0/4146222161 >> v2:172.21.15.105:3300/0 conn(0x7f750c153b70 0x7f750c15d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.913+0000 7f750a7fc640 1 -- 172.21.15.76:0/4146222161 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1193748568 0 0) 0x7f750c153f80 con 0x7f750c0c5b20 2024-04-25T19:15:45.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.913+0000 7f750a7fc640 1 -- 172.21.15.76:0/4146222161 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74f4003520 con 0x7f750c0c5b20 2024-04-25T19:15:45.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.913+0000 7f750a7fc640 1 -- 172.21.15.76:0/4146222161 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2183713265 0 0) 0x7f74f4003520 con 0x7f750c0c5b20 2024-04-25T19:15:45.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.913+0000 7f750a7fc640 1 -- 172.21.15.76:0/4146222161 >> v2:172.21.15.105:3300/0 conn(0x7f750c153b70 msgr2=0x7f750c15d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:45.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.913+0000 7f750a7fc640 1 --2- 172.21.15.76:0/4146222161 >> v2:172.21.15.105:3300/0 conn(0x7f750c153b70 0x7f750c15d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.913+0000 7f750a7fc640 1 -- 172.21.15.76:0/4146222161 >> v1:172.21.15.105:6789/0 conn(0x7f750c159840 legacy=0x7f750c0c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.913+0000 7f750a7fc640 1 -- 172.21.15.76:0/4146222161 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750c0718b0 con 0x7f750c0c5b20 2024-04-25T19:15:45.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.913+0000 7f750a7fc640 1 -- 172.21.15.76:0/4146222161 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f74fc001670 con 0x7f750c0c5b20 2024-04-25T19:15:45.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.913+0000 7f750a7fc640 1 -- 172.21.15.76:0/4146222161 >> v1:172.21.15.76:6789/0 conn(0x7f750c0c5b20 legacy=0x7f750c0c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.913+0000 7f750a7fc640 1 --2- 172.21.15.76:0/4146222161 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74f40039b0 0x7f74f4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.913+0000 7f750a7fc640 1 --2- 172.21.15.76:0/4146222161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74f4004800 0x7f74f4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.914+0000 7f750a7fc640 1 --2- 172.21.15.76:0/4146222161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74f40051c0 0x7f74f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.914+0000 7f750a7fc640 1 -- 172.21.15.76:0/4146222161 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f750c15ce30 con 0x7f74f40051c0 2024-04-25T19:15:45.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.914+0000 7f750a7fc640 1 -- 172.21.15.76:0/4146222161 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f74f400c160 con 0x7f74f4004800 2024-04-25T19:15:45.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.914+0000 7f750a7fc640 1 -- 172.21.15.76:0/4146222161 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f74f400c360 con 0x7f74f40039b0 2024-04-25T19:15:45.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.914+0000 7f750bfff640 1 --2- 172.21.15.76:0/4146222161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74f40051c0 0x7f74f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.914+0000 7f750b7fe640 1 --2- 172.21.15.76:0/4146222161 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74f40039b0 0x7f74f4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.914+0000 7f750affd640 1 --2- 172.21.15.76:0/4146222161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74f4004800 0x7f74f4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.914+0000 7f750bfff640 1 -- 172.21.15.76:0/4146222161 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74f40039b0 msgr2=0x7f74f4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:45.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.914+0000 7f750bfff640 1 --2- 172.21.15.76:0/4146222161 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f74f40039b0 0x7f74f4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.914+0000 7f750bfff640 1 -- 172.21.15.76:0/4146222161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74f4004800 msgr2=0x7f74f4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:45.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.914+0000 7f750bfff640 1 --2- 172.21.15.76:0/4146222161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74f4004800 0x7f74f4004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.914+0000 7f750bfff640 1 -- 172.21.15.76:0/4146222161 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f4003520 con 0x7f74f40051c0 2024-04-25T19:15:45.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.914+0000 7f750bfff640 1 -- 172.21.15.76:0/4146222161 reap_dead start 2024-04-25T19:15:45.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.915+0000 7f750bfff640 1 --2- 172.21.15.76:0/4146222161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74f40051c0 0x7f74f400b9c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7504009f90 tx=0x7f74f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3620ed4afef2b380 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:45.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.915+0000 7f750a7fc640 1 -- 172.21.15.76:0/4146222161 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7504009c10 con 0x7f74f40051c0 2024-04-25T19:15:45.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.915+0000 7f750a7fc640 1 -- 172.21.15.76:0/4146222161 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7504003b90 con 0x7f74f40051c0 2024-04-25T19:15:45.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.915+0000 7f750a7fc640 1 -- 172.21.15.76:0/4146222161 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7504004020 con 0x7f74f40051c0 2024-04-25T19:15:45.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.915+0000 7f7512251640 1 -- 172.21.15.76:0/4146222161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74f40051c0 msgr2=0x7f74f400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.915+0000 7f7512251640 1 --2- 172.21.15.76:0/4146222161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74f40051c0 0x7f74f400b9c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7504009f90 tx=0x7f74f4005830 comp rx=0 tx=0).stop 2024-04-25T19:15:45.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.915+0000 7f7512251640 1 -- 172.21.15.76:0/4146222161 shutdown_connections 2024-04-25T19:15:45.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.915+0000 7f7512251640 1 --2- 172.21.15.76:0/4146222161 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74f40051c0 0x7f74f400b9c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.915+0000 7f7512251640 1 -- 172.21.15.76:0/4146222161 >> 172.21.15.76:0/4146222161 conn(0x7f750c0cecc0 msgr2=0x7f750c0cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:45.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.916+0000 7f7512251640 1 -- 172.21.15.76:0/4146222161 shutdown_connections 2024-04-25T19:15:45.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.916+0000 7f7512251640 1 -- 172.21.15.76:0/4146222161 wait complete. 2024-04-25T19:15:45.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.917+0000 7f7512251640 1 Processor -- start 2024-04-25T19:15:45.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.917+0000 7f7512251640 1 -- start start 2024-04-25T19:15:45.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.919+0000 7f7512251640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74f40051c0 0x7f750c1e2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.919+0000 7f7512251640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f750c151f80 0x7f750c1eefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.919+0000 7f7512251640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f750c150f00 0x7f750c1584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.919+0000 7f7512251640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f750c071270 con 0x7f750c150f00 2024-04-25T19:15:45.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.919+0000 7f7512251640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f750c15d080 con 0x7f74f40051c0 2024-04-25T19:15:45.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.919+0000 7f7512251640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f750c15d880 con 0x7f750c151f80 2024-04-25T19:15:45.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.919+0000 7f750bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f750c150f00 0x7f750c1584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.919+0000 7f750bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f750c150f00 0x7f750c1584e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39984/0 (socket says 172.21.15.76:39984) 2024-04-25T19:15:45.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.919+0000 7f750bfff640 1 -- 172.21.15.76:0/1214089930 learned_addr learned my addr 172.21.15.76:0/1214089930 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:45.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.919+0000 7f750b7fe640 1 --2- 172.21.15.76:0/1214089930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74f40051c0 0x7f750c1e2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.919+0000 7f750affd640 1 --2- 172.21.15.76:0/1214089930 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f750c151f80 0x7f750c1eefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.919+0000 7f750bfff640 1 -- 172.21.15.76:0/1214089930 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f750c151f80 msgr2=0x7f750c1eefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.919+0000 7f750bfff640 1 --2- 172.21.15.76:0/1214089930 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f750c151f80 0x7f750c1eefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.919+0000 7f750bfff640 1 -- 172.21.15.76:0/1214089930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74f40051c0 msgr2=0x7f750c1e2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:45.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.919+0000 7f750bfff640 1 --2- 172.21.15.76:0/1214089930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74f40051c0 0x7f750c1e2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:45.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.919+0000 7f750bfff640 1 -- 172.21.15.76:0/1214089930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750c15aa00 con 0x7f750c150f00 2024-04-25T19:15:45.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.919+0000 7f750bfff640 1 --2- 172.21.15.76:0/1214089930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f750c150f00 0x7f750c1584e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f750402ae60 tx=0x7f7504009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:45.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.919+0000 7f74f3fff640 1 -- 172.21.15.76:0/1214089930 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7504009c10 con 0x7f750c150f00 2024-04-25T19:15:45.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.919+0000 7f74f3fff640 1 -- 172.21.15.76:0/1214089930 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7504043070 con 0x7f750c150f00 2024-04-25T19:15:45.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.919+0000 7f7512251640 1 -- 172.21.15.76:0/1214089930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f750c1ef500 con 0x7f750c150f00 2024-04-25T19:15:45.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.919+0000 7f74f3fff640 1 -- 172.21.15.76:0/1214089930 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750403d470 con 0x7f750c150f00 2024-04-25T19:15:45.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.919+0000 7f7512251640 1 -- 172.21.15.76:0/1214089930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f750c152a10 con 0x7f750c150f00 2024-04-25T19:15:45.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.921+0000 7f7512251640 1 -- 172.21.15.76:0/1214089930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f750c15abd0 con 0x7f750c150f00 2024-04-25T19:15:45.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.921+0000 7f74f3fff640 1 -- 172.21.15.76:0/1214089930 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7504005d10 con 0x7f750c150f00 2024-04-25T19:15:45.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.922+0000 7f74f3fff640 1 --2- 172.21.15.76:0/1214089930 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f74f80744b0 0x7f74f80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:45.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.922+0000 7f74f3fff640 1 -- 172.21.15.76:0/1214089930 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f75040b5e80 con 0x7f750c150f00 2024-04-25T19:15:45.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.922+0000 7f750b7fe640 1 --2- 172.21.15.76:0/1214089930 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f74f80744b0 0x7f74f80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:45.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.922+0000 7f750b7fe640 1 --2- 172.21.15.76:0/1214089930 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f74f80744b0 0x7f74f80769c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7500000c00 tx=0x7f7500009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:45.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:45.927+0000 7f74f3fff640 1 -- 172.21.15.76:0/1214089930 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f75040806a0 con 0x7f750c150f00 2024-04-25T19:15:46.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.109+0000 7f7512251640 1 -- 172.21.15.76:0/1214089930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f750c06a390 con 0x7f750c150f00 2024-04-25T19:15:46.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.110+0000 7f74f3fff640 1 -- 172.21.15.76:0/1214089930 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7f750407ffa0 con 0x7f750c150f00 2024-04-25T19:15:46.113 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:46.114 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":21,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","created":"2024-04-25T19:15:01.693137+0000","modified":"2024-04-25T19:15:20.401211+0000","last_up_change":"2024-04-25T19:15:11.891681+0000","last_in_change":"2024-04-25T19:15:03.898065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:15:13.846514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:15:16.606533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:15:17.719197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72a16c02-48f9-45aa-9447-fcf755ebf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409574998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409574998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409574998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409574998}]},"public_addr":"172.21.15.76:6817/1409574998","cluster_addr":"172.21.15.76:6819/1409574998","heartbeat_back_addr":"172.21.15.76:6823/1409574998","heartbeat_front_addr":"172.21.15.76:6821/1409574998","state":["exists","up"]},{"osd":1,"uuid":"818d50f0-76c6-45f8-bf79-5e304f14c5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6801","nonce":1302663821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6803","nonce":1302663821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6807","nonce":1302663821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6805","nonce":1302663821}]},"public_addr":"172.21.15.76:6801/1302663821","cluster_addr":"172.21.15.76:6803/1302663821","heartbeat_back_addr":"172.21.15.76:6807/1302663821","heartbeat_front_addr":"172.21.15.76:6805/1302663821","state":["exists","up"]},{"osd":2,"uuid":"5714c8f5-ae23-473c-8cf1-db73811acded","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6809","nonce":1992938181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6811","nonce":1992938181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6815","nonce":1992938181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6813","nonce":1992938181}]},"public_addr":"172.21.15.76:6809/1992938181","cluster_addr":"172.21.15.76:6811/1992938181","heartbeat_back_addr":"172.21.15.76:6815/1992938181","heartbeat_front_addr":"172.21.15.76:6813/1992938181","state":["exists","up"]},{"osd":3,"uuid":"13204cdb-d8e6-4f97-bc6e-9561f442decf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6825","nonce":3418257163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6827","nonce":3418257163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6831","nonce":3418257163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6829","nonce":3418257163}]},"public_addr":"172.21.15.76:6825/3418257163","cluster_addr":"172.21.15.76:6827/3418257163","heartbeat_back_addr":"172.21.15.76:6831/3418257163","heartbeat_front_addr":"172.21.15.76:6829/3418257163","state":["exists","up"]},{"osd":4,"uuid":"5bea5166-1986-4d56-b5f5-ecc8cbe7f3ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6801","nonce":2125002468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6803","nonce":2125002468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6807","nonce":2125002468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6805","nonce":2125002468}]},"public_addr":"172.21.15.105:6801/2125002468","cluster_addr":"172.21.15.105:6803/2125002468","heartbeat_back_addr":"172.21.15.105:6807/2125002468","heartbeat_front_addr":"172.21.15.105:6805/2125002468","state":["exists","up"]},{"osd":5,"uuid":"96f89dcc-270d-4421-b8e8-3b92ba5f9844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6819","nonce":3413103121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6821","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6823","nonce":3413103121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6831","nonce":3413103121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6828","nonce":3413103121}]},"public_addr":"172.21.15.105:6819/3413103121","cluster_addr":"172.21.15.105:6823/3413103121","heartbeat_back_addr":"172.21.15.105:6831/3413103121","heartbeat_front_addr":"172.21.15.105:6828/3413103121","state":["exists","up"]},{"osd":6,"uuid":"93eb087a-d9ba-4096-b85d-b74d96e819a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6809","nonce":2721832954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6811","nonce":2721832954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6815","nonce":2721832954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6813","nonce":2721832954}]},"public_addr":"172.21.15.105:6809/2721832954","cluster_addr":"172.21.15.105:6811/2721832954","heartbeat_back_addr":"172.21.15.105:6815/2721832954","heartbeat_front_addr":"172.21.15.105:6813/2721832954","state":["exists","up"]},{"osd":7,"uuid":"2f80f790-ad2d-460e-a37d-47bf1771ef43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6818","nonce":548350430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6822","nonce":548350430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6829","nonce":548350430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6825","nonce":548350430}]},"public_addr":"172.21.15.105:6818/548350430","cluster_addr":"172.21.15.105:6822/548350430","heartbeat_back_addr":"172.21.15.105:6829/548350430","heartbeat_front_addr":"172.21.15.105:6825/548350430","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.106901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.072819+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.126865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.053464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:08.982614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.104825+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.112714+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.107387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:15:46.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.112+0000 7f7512251640 1 -- 172.21.15.76:0/1214089930 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f74f80744b0 msgr2=0x7f74f80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:46.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.112+0000 7f7512251640 1 --2- 172.21.15.76:0/1214089930 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f74f80744b0 0x7f74f80769c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7500000c00 tx=0x7f7500009040 comp rx=0 tx=0).stop 2024-04-25T19:15:46.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.112+0000 7f7512251640 1 -- 172.21.15.76:0/1214089930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f750c150f00 msgr2=0x7f750c1584e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:46.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.112+0000 7f7512251640 1 --2- 172.21.15.76:0/1214089930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f750c150f00 0x7f750c1584e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f750402ae60 tx=0x7f7504009750 comp rx=0 tx=0).stop 2024-04-25T19:15:46.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.112+0000 7f7512251640 1 -- 172.21.15.76:0/1214089930 shutdown_connections 2024-04-25T19:15:46.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.112+0000 7f7512251640 1 --2- 172.21.15.76:0/1214089930 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f74f80744b0 0x7f74f80769c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.112+0000 7f7512251640 1 --2- 172.21.15.76:0/1214089930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f750c150f00 0x7f750c1584e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.112+0000 7f7512251640 1 --2- 172.21.15.76:0/1214089930 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f750c151f80 0x7f750c1eefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.112+0000 7f7512251640 1 --2- 172.21.15.76:0/1214089930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f74f40051c0 0x7f750c1e2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.112+0000 7f7512251640 1 -- 172.21.15.76:0/1214089930 >> 172.21.15.76:0/1214089930 conn(0x7f750c0cecc0 msgr2=0x7f750c0c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:46.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.112+0000 7f7512251640 1 -- 172.21.15.76:0/1214089930 shutdown_connections 2024-04-25T19:15:46.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.113+0000 7f7512251640 1 -- 172.21.15.76:0/1214089930 wait complete. 2024-04-25T19:15:46.126 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T19:15:13.846514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T19:15:16.606533+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T19:15:17.719197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T19:15:46.127 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:15:46.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.234+0000 7ffbc8d00640 1 Processor -- start 2024-04-25T19:15:46.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.234+0000 7ffbc8d00640 1 -- start start 2024-04-25T19:15:46.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.235+0000 7ffbc8d00640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffbc4159840 0x7ffbc4153b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:46.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.235+0000 7ffbc8d00640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffbc415d880 0x7ffbc40c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:46.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.235+0000 7ffbc8d00640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbc40715d0 con 0x7ffbc40c5b20 2024-04-25T19:15:46.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.235+0000 7ffbc8d00640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ffbc415ce30 con 0x7ffbc415d880 2024-04-25T19:15:46.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.235+0000 7ffbc8d00640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ffbc40718b0 con 0x7ffbc4159840 2024-04-25T19:15:46.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.236+0000 7ffbc2d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffbc415d880 0x7ffbc40c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:46.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.236+0000 7ffbc2d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffbc415d880 0x7ffbc40c88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39990/0 (socket says 172.21.15.76:39990) 2024-04-25T19:15:46.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.236+0000 7ffbc2d9d640 1 -- 172.21.15.76:0/754298245 learned_addr learned my addr 172.21.15.76:0/754298245 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:46.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.236+0000 7ffbc259c640 1 --2- 172.21.15.76:0/754298245 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffbc4159840 0x7ffbc4153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:46.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.236+0000 7ffbc2d9d640 1 -- 172.21.15.76:0/754298245 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffbc4159840 msgr2=0x7ffbc4153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:46.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.236+0000 7ffbc2d9d640 1 --2- 172.21.15.76:0/754298245 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffbc4159840 0x7ffbc4153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.236+0000 7ffbc2d9d640 1 -- 172.21.15.76:0/754298245 >> v1:172.21.15.105:6789/0 conn(0x7ffbc40c5b20 legacy=0x7ffbc40c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:46.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.236+0000 7ffbc2d9d640 1 -- 172.21.15.76:0/754298245 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc4071b90 con 0x7ffbc415d880 2024-04-25T19:15:46.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.237+0000 7ffbc2d9d640 1 --2- 172.21.15.76:0/754298245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbc415d880 0x7ffbc40c88f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc00d150 tx=0x7ffbbc02c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4989501685def6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:46.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.237+0000 7ffbc159a640 1 -- 172.21.15.76:0/754298245 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc009600 con 0x7ffbc415d880 2024-04-25T19:15:46.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.237+0000 7ffbc159a640 1 -- 172.21.15.76:0/754298245 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbbc037070 con 0x7ffbc415d880 2024-04-25T19:15:46.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.237+0000 7ffbc159a640 1 -- 172.21.15.76:0/754298245 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0099c0 con 0x7ffbc415d880 2024-04-25T19:15:46.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.238+0000 7ffbc8d00640 1 -- 172.21.15.76:0/754298245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbc415d880 msgr2=0x7ffbc40c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:46.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.238+0000 7ffbc8d00640 1 --2- 172.21.15.76:0/754298245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbc415d880 0x7ffbc40c88f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc00d150 tx=0x7ffbbc02c930 comp rx=0 tx=0).stop 2024-04-25T19:15:46.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.238+0000 7ffbc8d00640 1 -- 172.21.15.76:0/754298245 shutdown_connections 2024-04-25T19:15:46.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.238+0000 7ffbc8d00640 1 --2- 172.21.15.76:0/754298245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbc415d880 0x7ffbc40c88f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.238+0000 7ffbc8d00640 1 --2- 172.21.15.76:0/754298245 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffbc4159840 0x7ffbc4153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.238+0000 7ffbc8d00640 1 -- 172.21.15.76:0/754298245 >> 172.21.15.76:0/754298245 conn(0x7ffbc40cecc0 msgr2=0x7ffbc40cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:46.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.238+0000 7ffbc8d00640 1 -- 172.21.15.76:0/754298245 shutdown_connections 2024-04-25T19:15:46.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.238+0000 7ffbc8d00640 1 -- 172.21.15.76:0/754298245 wait complete. 2024-04-25T19:15:46.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.240+0000 7ffbc8d00640 1 Processor -- start 2024-04-25T19:15:46.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.240+0000 7ffbc8d00640 1 -- start start 2024-04-25T19:15:46.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.241+0000 7ffbc8d00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbc40c5b20 0x7ffbc415b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:46.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.241+0000 7ffbc8d00640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbc4159840 0x7ffbc414b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:46.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.241+0000 7ffbc8d00640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffbc415d880 0x7ffbc414bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:46.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.241+0000 7ffbc8d00640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc415dc90 con 0x7ffbc40c5b20 2024-04-25T19:15:46.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.241+0000 7ffbc8d00640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc4071270 con 0x7ffbc4159840 2024-04-25T19:15:46.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.241+0000 7ffbc8d00640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ffbc40c8e30 con 0x7ffbc415d880 2024-04-25T19:15:46.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.241+0000 7ffbc259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbc40c5b20 0x7ffbc415b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:46.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.241+0000 7ffbc259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbc40c5b20 0x7ffbc415b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40004/0 (socket says 172.21.15.76:40004) 2024-04-25T19:15:46.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.241+0000 7ffbc259c640 1 -- 172.21.15.76:0/3626601214 learned_addr learned my addr 172.21.15.76:0/3626601214 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:46.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.241+0000 7ffbc2d9d640 1 --2- 172.21.15.76:0/3626601214 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffbc415d880 0x7ffbc414bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:46.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.241+0000 7ffbc259c640 1 -- 172.21.15.76:0/3626601214 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffbc415d880 msgr2=0x7ffbc414bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:46.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.241+0000 7ffbc259c640 1 --2- 172.21.15.76:0/3626601214 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffbc415d880 0x7ffbc414bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.241+0000 7ffbc1d9b640 1 --2- 172.21.15.76:0/3626601214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbc4159840 0x7ffbc414b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:46.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.241+0000 7ffbc259c640 1 -- 172.21.15.76:0/3626601214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbc4159840 msgr2=0x7ffbc414b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:46.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.241+0000 7ffbc259c640 1 --2- 172.21.15.76:0/3626601214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbc4159840 0x7ffbc414b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.241+0000 7ffbc259c640 1 -- 172.21.15.76:0/3626601214 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc415b7e0 con 0x7ffbc40c5b20 2024-04-25T19:15:46.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.242+0000 7ffbc259c640 1 --2- 172.21.15.76:0/3626601214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbc40c5b20 0x7ffbc415b2a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400e870 tx=0x7ffbb400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:46.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.242+0000 7ffbb2ffd640 1 -- 172.21.15.76:0/3626601214 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4017070 con 0x7ffbc40c5b20 2024-04-25T19:15:46.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.242+0000 7ffbb2ffd640 1 -- 172.21.15.76:0/3626601214 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbb400c940 con 0x7ffbc40c5b20 2024-04-25T19:15:46.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.242+0000 7ffbc8d00640 1 -- 172.21.15.76:0/3626601214 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc41e3530 con 0x7ffbc40c5b20 2024-04-25T19:15:46.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.242+0000 7ffbb2ffd640 1 -- 172.21.15.76:0/3626601214 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb400cd30 con 0x7ffbc40c5b20 2024-04-25T19:15:46.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.242+0000 7ffbc8d00640 1 -- 172.21.15.76:0/3626601214 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc41e3db0 con 0x7ffbc40c5b20 2024-04-25T19:15:46.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.243+0000 7ffbc8d00640 1 -- 172.21.15.76:0/3626601214 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbc4072000 con 0x7ffbc40c5b20 2024-04-25T19:15:46.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.243+0000 7ffbb2ffd640 1 -- 172.21.15.76:0/3626601214 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ffbb401a070 con 0x7ffbc40c5b20 2024-04-25T19:15:46.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.244+0000 7ffbb2ffd640 1 --2- 172.21.15.76:0/3626601214 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ffba40744e0 0x7ffba40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:46.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.244+0000 7ffbb2ffd640 1 -- 172.21.15.76:0/3626601214 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ffbb4094fb0 con 0x7ffbc40c5b20 2024-04-25T19:15:46.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.244+0000 7ffbc1d9b640 1 --2- 172.21.15.76:0/3626601214 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ffba40744e0 0x7ffba40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:46.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.244+0000 7ffbc1d9b640 1 --2- 172.21.15.76:0/3626601214 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ffba40744e0 0x7ffba40769f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4159e70 tx=0x7ffbb8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:46.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.253+0000 7ffbb2ffd640 1 -- 172.21.15.76:0/3626601214 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffbc4072000 con 0x7ffbc40c5b20 2024-04-25T19:15:46.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.433+0000 7ffbc8d00640 1 -- 172.21.15.76:0/3626601214 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffbc406a390 con 0x7ffbc40c5b20 2024-04-25T19:15:46.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.434+0000 7ffbb2ffd640 1 -- 172.21.15.76:0/3626601214 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7ffbb405f860 con 0x7ffbc40c5b20 2024-04-25T19:15:46.436 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-04-25T19:15:46.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.435+0000 7ffbc8d00640 1 -- 172.21.15.76:0/3626601214 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ffba40744e0 msgr2=0x7ffba40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:46.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.435+0000 7ffbc8d00640 1 --2- 172.21.15.76:0/3626601214 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ffba40744e0 0x7ffba40769f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4159e70 tx=0x7ffbb8009040 comp rx=0 tx=0).stop 2024-04-25T19:15:46.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.435+0000 7ffbc8d00640 1 -- 172.21.15.76:0/3626601214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbc40c5b20 msgr2=0x7ffbc415b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:46.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.435+0000 7ffbc8d00640 1 --2- 172.21.15.76:0/3626601214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbc40c5b20 0x7ffbc415b2a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400e870 tx=0x7ffbb400ed30 comp rx=0 tx=0).stop 2024-04-25T19:15:46.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.436+0000 7ffbc8d00640 1 -- 172.21.15.76:0/3626601214 shutdown_connections 2024-04-25T19:15:46.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.436+0000 7ffbc8d00640 1 --2- 172.21.15.76:0/3626601214 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ffba40744e0 0x7ffba40769f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.436+0000 7ffbc8d00640 1 --2- 172.21.15.76:0/3626601214 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffbc415d880 0x7ffbc414bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.436+0000 7ffbc8d00640 1 --2- 172.21.15.76:0/3626601214 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffbc4159840 0x7ffbc414b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.436+0000 7ffbc8d00640 1 --2- 172.21.15.76:0/3626601214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbc40c5b20 0x7ffbc415b2a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.436+0000 7ffbc8d00640 1 -- 172.21.15.76:0/3626601214 >> 172.21.15.76:0/3626601214 conn(0x7ffbc40cecc0 msgr2=0x7ffbc415a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:46.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.436+0000 7ffbc8d00640 1 -- 172.21.15.76:0/3626601214 shutdown_connections 2024-04-25T19:15:46.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.436+0000 7ffbc8d00640 1 -- 172.21.15.76:0/3626601214 wait complete. 2024-04-25T19:15:46.449 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T19:15:46.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.563+0000 7fb8d186e640 1 Processor -- start 2024-04-25T19:15:46.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.563+0000 7fb8d186e640 1 -- start start 2024-04-25T19:15:46.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.564+0000 7fb8d186e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb8cc0c96c0 0x7fb8cc15cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:46.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.564+0000 7fb8d186e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8cc074650 0x7fb8cc160a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:46.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.564+0000 7fb8d186e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb8cc06c5c0 0x7fb8cc06c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:46.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.564+0000 7fb8d186e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb8cc15d1c0 con 0x7fb8cc06c5c0 2024-04-25T19:15:46.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.564+0000 7fb8d186e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fb8cc15d4a0 con 0x7fb8cc0c96c0 2024-04-25T19:15:46.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.564+0000 7fb8d186e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb8cc15d780 con 0x7fb8cc074650 2024-04-25T19:15:46.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.564+0000 7fb8cb7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb8cc06c5c0 0x7fb8cc06c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:46.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.564+0000 7fb8cb7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb8cc06c5c0 0x7fb8cc06c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40018/0 (socket says 172.21.15.76:40018) 2024-04-25T19:15:46.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.564+0000 7fb8cb7fe640 1 -- 172.21.15.76:0/1619641851 learned_addr learned my addr 172.21.15.76:0/1619641851 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:46.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.564+0000 7fb8c3fff640 1 --2- 172.21.15.76:0/1619641851 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8cc074650 0x7fb8cc160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:46.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.564+0000 7fb8caffd640 1 --2- 172.21.15.76:0/1619641851 >> v2:172.21.15.105:3300/0 conn(0x7fb8cc0c96c0 0x7fb8cc15cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:46.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.565+0000 7fb8cb7fe640 1 -- 172.21.15.76:0/1619641851 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8cc074650 msgr2=0x7fb8cc160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:46.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.565+0000 7fb8cb7fe640 1 --2- 172.21.15.76:0/1619641851 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8cc074650 0x7fb8cc160a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.565+0000 7fb8cb7fe640 1 -- 172.21.15.76:0/1619641851 >> v2:172.21.15.105:3300/0 conn(0x7fb8cc0c96c0 msgr2=0x7fb8cc15cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:46.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.565+0000 7fb8cb7fe640 1 --2- 172.21.15.76:0/1619641851 >> v2:172.21.15.105:3300/0 conn(0x7fb8cc0c96c0 0x7fb8cc15cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.565+0000 7fb8cb7fe640 1 -- 172.21.15.76:0/1619641851 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8cc160210 con 0x7fb8cc06c5c0 2024-04-25T19:15:46.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.565+0000 7fb8cb7fe640 1 --2- 172.21.15.76:0/1619641851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8cc06c5c0 0x7fb8cc06c9d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc06d5e0 tx=0x7fb8c402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b3a11dd3614eb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:46.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.565+0000 7fb8ca7fc640 1 -- 172.21.15.76:0/1619641851 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4036070 con 0x7fb8cc06c5c0 2024-04-25T19:15:46.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.565+0000 7fb8ca7fc640 1 -- 172.21.15.76:0/1619641851 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8c4031600 con 0x7fb8cc06c5c0 2024-04-25T19:15:46.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.565+0000 7fb8ca7fc640 1 -- 172.21.15.76:0/1619641851 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c40319d0 con 0x7fb8cc06c5c0 2024-04-25T19:15:46.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.566+0000 7fb8d186e640 1 -- 172.21.15.76:0/1619641851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8cc06c5c0 msgr2=0x7fb8cc06c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:46.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.566+0000 7fb8d186e640 1 --2- 172.21.15.76:0/1619641851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8cc06c5c0 0x7fb8cc06c9d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc06d5e0 tx=0x7fb8c402a730 comp rx=0 tx=0).stop 2024-04-25T19:15:46.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.566+0000 7fb8d186e640 1 -- 172.21.15.76:0/1619641851 shutdown_connections 2024-04-25T19:15:46.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.566+0000 7fb8d186e640 1 --2- 172.21.15.76:0/1619641851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8cc06c5c0 0x7fb8cc06c9d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.566+0000 7fb8d186e640 1 --2- 172.21.15.76:0/1619641851 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8cc074650 0x7fb8cc160a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.566+0000 7fb8d186e640 1 --2- 172.21.15.76:0/1619641851 >> v2:172.21.15.105:3300/0 conn(0x7fb8cc0c96c0 0x7fb8cc15cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.566+0000 7fb8d186e640 1 -- 172.21.15.76:0/1619641851 >> 172.21.15.76:0/1619641851 conn(0x7fb8cc0cecc0 msgr2=0x7fb8cc0cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:46.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.566+0000 7fb8d186e640 1 -- 172.21.15.76:0/1619641851 shutdown_connections 2024-04-25T19:15:46.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.566+0000 7fb8d186e640 1 -- 172.21.15.76:0/1619641851 wait complete. 2024-04-25T19:15:46.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.568+0000 7fb8d186e640 1 Processor -- start 2024-04-25T19:15:46.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.568+0000 7fb8d186e640 1 -- start start 2024-04-25T19:15:46.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.569+0000 7fb8d186e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8cc06c5c0 0x7fb8cc1f3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:46.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.569+0000 7fb8d186e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8cc074650 0x7fb8cc0cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:46.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.569+0000 7fb8d186e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8cc0c96c0 0x7fb8cc1e5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:46.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.569+0000 7fb8d186e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc0ca060 con 0x7fb8cc074650 2024-04-25T19:15:46.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.569+0000 7fb8d186e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc0c7330 con 0x7fb8cc0c96c0 2024-04-25T19:15:46.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.569+0000 7fb8d186e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb8cc15da60 con 0x7fb8cc06c5c0 2024-04-25T19:15:46.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.569+0000 7fb8c3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8cc074650 0x7fb8cc0cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:46.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.569+0000 7fb8c3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8cc074650 0x7fb8cc0cbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40032/0 (socket says 172.21.15.76:40032) 2024-04-25T19:15:46.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.569+0000 7fb8c3fff640 1 -- 172.21.15.76:0/855984906 learned_addr learned my addr 172.21.15.76:0/855984906 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:46.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.569+0000 7fb8caffd640 1 --2- 172.21.15.76:0/855984906 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8cc06c5c0 0x7fb8cc1f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:46.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.569+0000 7fb8cb7fe640 1 --2- 172.21.15.76:0/855984906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8cc0c96c0 0x7fb8cc1e5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:46.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.569+0000 7fb8c3fff640 1 -- 172.21.15.76:0/855984906 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8cc06c5c0 msgr2=0x7fb8cc1f3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:46.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.569+0000 7fb8c3fff640 1 --2- 172.21.15.76:0/855984906 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8cc06c5c0 0x7fb8cc1f3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.570+0000 7fb8c3fff640 1 -- 172.21.15.76:0/855984906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8cc0c96c0 msgr2=0x7fb8cc1e5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:46.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.570+0000 7fb8c3fff640 1 --2- 172.21.15.76:0/855984906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8cc0c96c0 0x7fb8cc1e5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.570+0000 7fb8c3fff640 1 -- 172.21.15.76:0/855984906 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8cc1f3580 con 0x7fb8cc074650 2024-04-25T19:15:46.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.570+0000 7fb8cb7fe640 1 --2- 172.21.15.76:0/855984906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8cc0c96c0 0x7fb8cc1e5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:15:46.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.570+0000 7fb8c3fff640 1 --2- 172.21.15.76:0/855984906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8cc074650 0x7fb8cc0cbfd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb8b400e8a0 tx=0x7fb8b400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:46.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.570+0000 7fb8c8ff9640 1 -- 172.21.15.76:0/855984906 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4017070 con 0x7fb8cc074650 2024-04-25T19:15:46.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.570+0000 7fb8c8ff9640 1 -- 172.21.15.76:0/855984906 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b400ee90 con 0x7fb8cc074650 2024-04-25T19:15:46.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.570+0000 7fb8d186e640 1 -- 172.21.15.76:0/855984906 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8cc0ca610 con 0x7fb8cc074650 2024-04-25T19:15:46.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.570+0000 7fb8c8ff9640 1 -- 172.21.15.76:0/855984906 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b400cb80 con 0x7fb8cc074650 2024-04-25T19:15:46.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.570+0000 7fb8d186e640 1 -- 172.21.15.76:0/855984906 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8cc0cb440 con 0x7fb8cc074650 2024-04-25T19:15:46.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.571+0000 7fb8d186e640 1 -- 172.21.15.76:0/855984906 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8cc06a390 con 0x7fb8cc074650 2024-04-25T19:15:46.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.572+0000 7fb8c8ff9640 1 -- 172.21.15.76:0/855984906 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb8b401a070 con 0x7fb8cc074650 2024-04-25T19:15:46.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.572+0000 7fb8c8ff9640 1 --2- 172.21.15.76:0/855984906 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb8ac0744b0 0x7fb8ac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:46.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.572+0000 7fb8c8ff9640 1 -- 172.21.15.76:0/855984906 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb8b40956a0 con 0x7fb8cc074650 2024-04-25T19:15:46.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.572+0000 7fb8caffd640 1 --2- 172.21.15.76:0/855984906 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb8ac0744b0 0x7fb8ac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:46.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.573+0000 7fb8caffd640 1 --2- 172.21.15.76:0/855984906 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb8ac0744b0 0x7fb8ac0769c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc009a10 tx=0x7fb8bc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:46.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.576+0000 7fb8c8ff9640 1 -- 172.21.15.76:0/855984906 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb8b4060030 con 0x7fb8cc074650 2024-04-25T19:15:46.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.760+0000 7fb8d186e640 1 -- 172.21.15.76:0/855984906 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb8cc074430 con 0x7fb8cc074650 2024-04-25T19:15:46.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.761+0000 7fb8c8ff9640 1 -- 172.21.15.76:0/855984906 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fb8b405f930 con 0x7fb8cc074650 2024-04-25T19:15:46.764 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-04-25T19:15:46.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.763+0000 7fb8d186e640 1 -- 172.21.15.76:0/855984906 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb8ac0744b0 msgr2=0x7fb8ac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:46.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.763+0000 7fb8d186e640 1 --2- 172.21.15.76:0/855984906 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb8ac0744b0 0x7fb8ac0769c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc009a10 tx=0x7fb8bc008040 comp rx=0 tx=0).stop 2024-04-25T19:15:46.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.763+0000 7fb8d186e640 1 -- 172.21.15.76:0/855984906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8cc074650 msgr2=0x7fb8cc0cbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:46.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.763+0000 7fb8d186e640 1 --2- 172.21.15.76:0/855984906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8cc074650 0x7fb8cc0cbfd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb8b400e8a0 tx=0x7fb8b400ed60 comp rx=0 tx=0).stop 2024-04-25T19:15:46.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.763+0000 7fb8d186e640 1 -- 172.21.15.76:0/855984906 shutdown_connections 2024-04-25T19:15:46.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.763+0000 7fb8d186e640 1 --2- 172.21.15.76:0/855984906 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb8ac0744b0 0x7fb8ac0769c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.763+0000 7fb8d186e640 1 --2- 172.21.15.76:0/855984906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8cc0c96c0 0x7fb8cc1e5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.763+0000 7fb8d186e640 1 --2- 172.21.15.76:0/855984906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8cc074650 0x7fb8cc0cbfd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.763+0000 7fb8d186e640 1 --2- 172.21.15.76:0/855984906 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8cc06c5c0 0x7fb8cc1f3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.763+0000 7fb8d186e640 1 -- 172.21.15.76:0/855984906 >> 172.21.15.76:0/855984906 conn(0x7fb8cc0cecc0 msgr2=0x7fb8cc155ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:46.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.763+0000 7fb8d186e640 1 -- 172.21.15.76:0/855984906 shutdown_connections 2024-04-25T19:15:46.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.763+0000 7fb8d186e640 1 -- 172.21.15.76:0/855984906 wait complete. 2024-04-25T19:15:46.776 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T19:15:46.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.885+0000 7f3b8e557640 1 Processor -- start 2024-04-25T19:15:46.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.885+0000 7f3b8e557640 1 -- start start 2024-04-25T19:15:46.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.886+0000 7f3b8e557640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3b88158060 0x7f3b880c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:46.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.886+0000 7f3b8e557640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b880ca750 0x7f3b880cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:46.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.886+0000 7f3b8e557640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b880c56a0 con 0x7f3b880cab60 2024-04-25T19:15:46.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.886+0000 7f3b8e557640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f3b880c5960 con 0x7f3b88158060 2024-04-25T19:15:46.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.886+0000 7f3b8e557640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f3b880c5c40 con 0x7f3b880ca750 2024-04-25T19:15:46.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.886+0000 7f3b877fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b880ca750 0x7f3b880cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:46.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.886+0000 7f3b877fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b880ca750 0x7f3b880cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:35986/0 (socket says 172.21.15.76:35986) 2024-04-25T19:15:46.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.886+0000 7f3b877fe640 1 -- 172.21.15.76:0/3246884217 learned_addr learned my addr 172.21.15.76:0/3246884217 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:46.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.886+0000 7f3b87fff640 1 --2- 172.21.15.76:0/3246884217 >> v2:172.21.15.105:3300/0 conn(0x7f3b88158060 0x7f3b880c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:46.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.887+0000 7f3b877fe640 1 -- 172.21.15.76:0/3246884217 >> v2:172.21.15.105:3300/0 conn(0x7f3b88158060 msgr2=0x7f3b880c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:46.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.887+0000 7f3b877fe640 1 --2- 172.21.15.76:0/3246884217 >> v2:172.21.15.105:3300/0 conn(0x7f3b88158060 0x7f3b880c4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.887+0000 7f3b877fe640 1 -- 172.21.15.76:0/3246884217 >> v1:172.21.15.105:6789/0 conn(0x7f3b880cab60 legacy=0x7f3b8815af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:46.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.887+0000 7f3b877fe640 1 -- 172.21.15.76:0/3246884217 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b880c5f20 con 0x7f3b880ca750 2024-04-25T19:15:46.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.887+0000 7f3b877fe640 1 --2- 172.21.15.76:0/3246884217 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b880ca750 0x7f3b880cc7e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009870 tx=0x7f3b7c02a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f2c144b1121e88c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:46.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.888+0000 7f3b86ffd640 1 -- 172.21.15.76:0/3246884217 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c037070 con 0x7f3b880ca750 2024-04-25T19:15:46.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.888+0000 7f3b86ffd640 1 -- 172.21.15.76:0/3246884217 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b7c02ae50 con 0x7f3b880ca750 2024-04-25T19:15:46.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.888+0000 7f3b86ffd640 1 -- 172.21.15.76:0/3246884217 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c032860 con 0x7f3b880ca750 2024-04-25T19:15:46.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.888+0000 7f3b8e557640 1 -- 172.21.15.76:0/3246884217 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b880ca750 msgr2=0x7f3b880cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:46.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.888+0000 7f3b8e557640 1 --2- 172.21.15.76:0/3246884217 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b880ca750 0x7f3b880cc7e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009870 tx=0x7f3b7c02a300 comp rx=0 tx=0).stop 2024-04-25T19:15:46.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.888+0000 7f3b8e557640 1 -- 172.21.15.76:0/3246884217 shutdown_connections 2024-04-25T19:15:46.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.888+0000 7f3b8e557640 1 --2- 172.21.15.76:0/3246884217 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b880ca750 0x7f3b880cc7e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.888+0000 7f3b8e557640 1 --2- 172.21.15.76:0/3246884217 >> v2:172.21.15.105:3300/0 conn(0x7f3b88158060 0x7f3b880c4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.888+0000 7f3b8e557640 1 -- 172.21.15.76:0/3246884217 >> 172.21.15.76:0/3246884217 conn(0x7f3b88068b10 msgr2=0x7f3b88072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:46.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.888+0000 7f3b8e557640 1 -- 172.21.15.76:0/3246884217 shutdown_connections 2024-04-25T19:15:46.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.888+0000 7f3b8e557640 1 -- 172.21.15.76:0/3246884217 wait complete. 2024-04-25T19:15:46.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.890+0000 7f3b8e557640 1 Processor -- start 2024-04-25T19:15:46.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.890+0000 7f3b8e557640 1 -- start start 2024-04-25T19:15:46.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.891+0000 7f3b8e557640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b880ca750 0x7f3b881f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:46.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.891+0000 7f3b8e557640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b880cab60 0x7f3b880b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:46.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.891+0000 7f3b8e557640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b88158060 0x7f3b880c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:46.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.891+0000 7f3b8e557640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3b880c5320 con 0x7f3b880ca750 2024-04-25T19:15:46.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.891+0000 7f3b8e557640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3b880c6200 con 0x7f3b88158060 2024-04-25T19:15:46.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.891+0000 7f3b8e557640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f3b8815b690 con 0x7f3b880cab60 2024-04-25T19:15:46.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.891+0000 7f3b87fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b880ca750 0x7f3b881f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:46.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.891+0000 7f3b87fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b880ca750 0x7f3b881f2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40042/0 (socket says 172.21.15.76:40042) 2024-04-25T19:15:46.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.891+0000 7f3b87fff640 1 -- 172.21.15.76:0/1066774548 learned_addr learned my addr 172.21.15.76:0/1066774548 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:46.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.892+0000 7f3b877fe640 1 --2- 172.21.15.76:0/1066774548 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b880cab60 0x7f3b880b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:46.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.892+0000 7f3b8caf4640 1 --2- 172.21.15.76:0/1066774548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b88158060 0x7f3b880c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:46.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.892+0000 7f3b87fff640 1 -- 172.21.15.76:0/1066774548 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b880cab60 msgr2=0x7f3b880b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:46.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.892+0000 7f3b87fff640 1 --2- 172.21.15.76:0/1066774548 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b880cab60 0x7f3b880b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.892+0000 7f3b87fff640 1 -- 172.21.15.76:0/1066774548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b88158060 msgr2=0x7f3b880c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:46.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.892+0000 7f3b87fff640 1 --2- 172.21.15.76:0/1066774548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b88158060 0x7f3b880c8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:46.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.892+0000 7f3b87fff640 1 -- 172.21.15.76:0/1066774548 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b881f34c0 con 0x7f3b880ca750 2024-04-25T19:15:46.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.892+0000 7f3b8caf4640 1 --2- 172.21.15.76:0/1066774548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b88158060 0x7f3b880c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:15:46.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.892+0000 7f3b87fff640 1 --2- 172.21.15.76:0/1066774548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b880ca750 0x7f3b881f2f80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800e8a0 tx=0x7f3b7800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:46.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.892+0000 7f3b84ff9640 1 -- 172.21.15.76:0/1066774548 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b78017070 con 0x7f3b880ca750 2024-04-25T19:15:46.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.892+0000 7f3b84ff9640 1 -- 172.21.15.76:0/1066774548 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b7800c940 con 0x7f3b880ca750 2024-04-25T19:15:46.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.892+0000 7f3b8e557640 1 -- 172.21.15.76:0/1066774548 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b88154980 con 0x7f3b880ca750 2024-04-25T19:15:46.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.892+0000 7f3b84ff9640 1 -- 172.21.15.76:0/1066774548 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7800cd30 con 0x7f3b880ca750 2024-04-25T19:15:46.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.892+0000 7f3b8e557640 1 -- 172.21.15.76:0/1066774548 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b88154c60 con 0x7f3b880ca750 2024-04-25T19:15:46.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.893+0000 7f3b8e557640 1 -- 172.21.15.76:0/1066774548 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b880ca420 con 0x7f3b880ca750 2024-04-25T19:15:46.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.894+0000 7f3b84ff9640 1 -- 172.21.15.76:0/1066774548 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3b7801a070 con 0x7f3b880ca750 2024-04-25T19:15:46.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.894+0000 7f3b84ff9640 1 --2- 172.21.15.76:0/1066774548 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f3b740744b0 0x7f3b740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:46.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.894+0000 7f3b84ff9640 1 -- 172.21.15.76:0/1066774548 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3b78095500 con 0x7f3b880ca750 2024-04-25T19:15:46.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.894+0000 7f3b877fe640 1 --2- 172.21.15.76:0/1066774548 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f3b740744b0 0x7f3b740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:46.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.895+0000 7f3b877fe640 1 --2- 172.21.15.76:0/1066774548 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f3b740744b0 0x7f3b740769c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c034000 tx=0x7f3b7c02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:46.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:46.899+0000 7f3b84ff9640 1 -- 172.21.15.76:0/1066774548 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3b880ca420 con 0x7f3b880ca750 2024-04-25T19:15:47.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.078+0000 7f3b8e557640 1 -- 172.21.15.76:0/1066774548 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3b8806a390 con 0x7f3b880ca750 2024-04-25T19:15:47.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.079+0000 7f3b84ff9640 1 -- 172.21.15.76:0/1066774548 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f3b7805fc90 con 0x7f3b880ca750 2024-04-25T19:15:47.082 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-04-25T19:15:47.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.081+0000 7f3b8e557640 1 -- 172.21.15.76:0/1066774548 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f3b740744b0 msgr2=0x7f3b740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:47.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.081+0000 7f3b8e557640 1 --2- 172.21.15.76:0/1066774548 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f3b740744b0 0x7f3b740769c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c034000 tx=0x7f3b7c02e000 comp rx=0 tx=0).stop 2024-04-25T19:15:47.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.081+0000 7f3b8e557640 1 -- 172.21.15.76:0/1066774548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b880ca750 msgr2=0x7f3b881f2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:47.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.081+0000 7f3b8e557640 1 --2- 172.21.15.76:0/1066774548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b880ca750 0x7f3b881f2f80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800e8a0 tx=0x7f3b7800ed60 comp rx=0 tx=0).stop 2024-04-25T19:15:47.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.081+0000 7f3b8e557640 1 -- 172.21.15.76:0/1066774548 shutdown_connections 2024-04-25T19:15:47.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.081+0000 7f3b8e557640 1 --2- 172.21.15.76:0/1066774548 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f3b740744b0 0x7f3b740769c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.081+0000 7f3b8e557640 1 --2- 172.21.15.76:0/1066774548 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b88158060 0x7f3b880c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.081+0000 7f3b8e557640 1 --2- 172.21.15.76:0/1066774548 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3b880cab60 0x7f3b880b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.081+0000 7f3b8e557640 1 --2- 172.21.15.76:0/1066774548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b880ca750 0x7f3b881f2f80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.081+0000 7f3b8e557640 1 -- 172.21.15.76:0/1066774548 >> 172.21.15.76:0/1066774548 conn(0x7f3b88068b10 msgr2=0x7f3b88152550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:47.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.081+0000 7f3b8e557640 1 -- 172.21.15.76:0/1066774548 shutdown_connections 2024-04-25T19:15:47.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.081+0000 7f3b8e557640 1 -- 172.21.15.76:0/1066774548 wait complete. 2024-04-25T19:15:47.095 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:15:47.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.201+0000 7f553e842640 1 Processor -- start 2024-04-25T19:15:47.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.201+0000 7f553e842640 1 -- start start 2024-04-25T19:15:47.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.202+0000 7f553e842640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f55380c9cd0 0x7f5538156f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:47.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.202+0000 7f553e842640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55380c8fe0 con 0x7f55380c9780 2024-04-25T19:15:47.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.202+0000 7f553e842640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55380c92c0 con 0x7f5538160e20 2024-04-25T19:15:47.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.202+0000 7f553e842640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f5538160340 con 0x7f55380c9cd0 2024-04-25T19:15:47.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.202+0000 7f5537fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f55380c9780 0x7f5538068520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53604/0 (socket says 172.21.15.76:53604) 2024-04-25T19:15:47.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.202+0000 7f5537fff640 1 -- 172.21.15.76:0/327138463 learned_addr learned my addr 172.21.15.76:0/327138463 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:47.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.202+0000 7f5536ffd640 1 -- 172.21.15.76:0/327138463 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3523422290 0 0) 0x7f55380c8fe0 con 0x7f55380c9780 2024-04-25T19:15:47.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.202+0000 7f553cddf640 1 --2- 172.21.15.76:0/327138463 >> v2:172.21.15.105:3300/0 conn(0x7f55380c9cd0 0x7f5538156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:47.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.202+0000 7f5536ffd640 1 -- 172.21.15.76:0/327138463 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5520003520 con 0x7f55380c9780 2024-04-25T19:15:47.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.203+0000 7f5536ffd640 1 -- 172.21.15.76:0/327138463 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1775086653 0 0) 0x7f5520003520 con 0x7f55380c9780 2024-04-25T19:15:47.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.203+0000 7f5536ffd640 1 -- 172.21.15.76:0/327138463 >> v2:172.21.15.105:3300/0 conn(0x7f55380c9cd0 msgr2=0x7f5538156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:47.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.203+0000 7f5536ffd640 1 --2- 172.21.15.76:0/327138463 >> v2:172.21.15.105:3300/0 conn(0x7f55380c9cd0 0x7f5538156f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.203+0000 7f5536ffd640 1 -- 172.21.15.76:0/327138463 >> v1:172.21.15.105:6789/0 conn(0x7f5538160e20 legacy=0x7f5538074130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:47.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.203+0000 7f5536ffd640 1 -- 172.21.15.76:0/327138463 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5538160620 con 0x7f55380c9780 2024-04-25T19:15:47.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.203+0000 7f5536ffd640 1 -- 172.21.15.76:0/327138463 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f5528001670 con 0x7f55380c9780 2024-04-25T19:15:47.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.203+0000 7f553cddf640 1 --2- 172.21.15.76:0/327138463 >> v2:172.21.15.105:3300/0 conn(0x7f55380c9cd0 0x7f5538156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:47.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.203+0000 7f5536ffd640 1 -- 172.21.15.76:0/327138463 >> v1:172.21.15.76:6789/0 conn(0x7f55380c9780 legacy=0x7f5538068520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:47.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.203+0000 7f5536ffd640 1 --2- 172.21.15.76:0/327138463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55200039b0 0x7f5520003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:47.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.203+0000 7f5536ffd640 1 --2- 172.21.15.76:0/327138463 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5520004800 0x7f5520004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:47.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.203+0000 7f5536ffd640 1 --2- 172.21.15.76:0/327138463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5520005270 0x7f552000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:47.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.203+0000 7f5536ffd640 1 -- 172.21.15.76:0/327138463 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5538160340 con 0x7f55200039b0 2024-04-25T19:15:47.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.203+0000 7f5536ffd640 1 -- 172.21.15.76:0/327138463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f552000c200 con 0x7f5520005270 2024-04-25T19:15:47.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.203+0000 7f5536ffd640 1 -- 172.21.15.76:0/327138463 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f552000c400 con 0x7f5520004800 2024-04-25T19:15:47.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.203+0000 7f5537fff640 1 --2- 172.21.15.76:0/327138463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55200039b0 0x7f5520003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:47.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.203+0000 7f553cddf640 1 --2- 172.21.15.76:0/327138463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5520005270 0x7f552000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:47.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.203+0000 7f55377fe640 1 --2- 172.21.15.76:0/327138463 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5520004800 0x7f5520004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:47.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.203+0000 7f5537fff640 1 -- 172.21.15.76:0/327138463 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5520004800 msgr2=0x7f5520004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:47.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.203+0000 7f5537fff640 1 --2- 172.21.15.76:0/327138463 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5520004800 0x7f5520004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.203+0000 7f5537fff640 1 -- 172.21.15.76:0/327138463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5520005270 msgr2=0x7f552000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:47.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.203+0000 7f5537fff640 1 --2- 172.21.15.76:0/327138463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5520005270 0x7f552000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.203+0000 7f553cddf640 1 -- 172.21.15.76:0/327138463 reap_dead start 2024-04-25T19:15:47.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.203+0000 7f5537fff640 1 -- 172.21.15.76:0/327138463 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5520003520 con 0x7f55200039b0 2024-04-25T19:15:47.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.204+0000 7f5537fff640 1 --2- 172.21.15.76:0/327138463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55200039b0 0x7f5520003e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f552800c240 tx=0x7f5520005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6560d1be59528b27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:47.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.204+0000 7f5536ffd640 1 -- 172.21.15.76:0/327138463 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5528007d10 con 0x7f55200039b0 2024-04-25T19:15:47.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.204+0000 7f5536ffd640 1 -- 172.21.15.76:0/327138463 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55280058d0 con 0x7f55200039b0 2024-04-25T19:15:47.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.204+0000 7f5536ffd640 1 -- 172.21.15.76:0/327138463 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5528005d40 con 0x7f55200039b0 2024-04-25T19:15:47.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.205+0000 7f553e842640 1 -- 172.21.15.76:0/327138463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55200039b0 msgr2=0x7f5520003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:47.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.205+0000 7f553e842640 1 --2- 172.21.15.76:0/327138463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55200039b0 0x7f5520003e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f552800c240 tx=0x7f5520005830 comp rx=0 tx=0).stop 2024-04-25T19:15:47.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.205+0000 7f553e842640 1 -- 172.21.15.76:0/327138463 shutdown_connections 2024-04-25T19:15:47.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.205+0000 7f553e842640 1 --2- 172.21.15.76:0/327138463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55200039b0 0x7f5520003e20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.205+0000 7f553e842640 1 -- 172.21.15.76:0/327138463 >> 172.21.15.76:0/327138463 conn(0x7f553806c5c0 msgr2=0x7f553806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:47.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.205+0000 7f553e842640 1 -- 172.21.15.76:0/327138463 shutdown_connections 2024-04-25T19:15:47.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.205+0000 7f553e842640 1 -- 172.21.15.76:0/327138463 wait complete. 2024-04-25T19:15:47.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.207+0000 7f553e842640 1 Processor -- start 2024-04-25T19:15:47.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.207+0000 7f553e842640 1 -- start start 2024-04-25T19:15:47.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.208+0000 7f553e842640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f55200039b0 0x7f553815e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:47.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.208+0000 7f553e842640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f553806e8a0 0x7f5538148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:47.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.208+0000 7f553e842640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55381e9800 0x7f55381e9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:47.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.208+0000 7f553e842640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f55380c5590 con 0x7f553806e8a0 2024-04-25T19:15:47.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.208+0000 7f553e842640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f55380ca0e0 con 0x7f55381e9800 2024-04-25T19:15:47.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.208+0000 7f553e842640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f5538156f50 con 0x7f55200039b0 2024-04-25T19:15:47.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.208+0000 7f55377fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f553806e8a0 0x7f5538148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:47.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.208+0000 7f55377fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f553806e8a0 0x7f5538148fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40060/0 (socket says 172.21.15.76:40060) 2024-04-25T19:15:47.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.208+0000 7f55377fe640 1 -- 172.21.15.76:0/866344570 learned_addr learned my addr 172.21.15.76:0/866344570 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:47.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.208+0000 7f5537fff640 1 --2- 172.21.15.76:0/866344570 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f55200039b0 0x7f553815e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:47.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.208+0000 7f553cddf640 1 --2- 172.21.15.76:0/866344570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55381e9800 0x7f55381e9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:47.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.208+0000 7f55377fe640 1 -- 172.21.15.76:0/866344570 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f55200039b0 msgr2=0x7f553815e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:47.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.208+0000 7f55377fe640 1 --2- 172.21.15.76:0/866344570 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f55200039b0 0x7f553815e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.208+0000 7f55377fe640 1 -- 172.21.15.76:0/866344570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55381e9800 msgr2=0x7f55381e9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:47.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.208+0000 7f55377fe640 1 --2- 172.21.15.76:0/866344570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55381e9800 0x7f55381e9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.208+0000 7f55377fe640 1 -- 172.21.15.76:0/866344570 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55381f31b0 con 0x7f553806e8a0 2024-04-25T19:15:47.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.209+0000 7f55377fe640 1 --2- 172.21.15.76:0/866344570 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f553806e8a0 0x7f5538148fc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f552c00f5c0 tx=0x7f552c00fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:47.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.209+0000 7f5534ff9640 1 -- 172.21.15.76:0/866344570 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c0052e0 con 0x7f553806e8a0 2024-04-25T19:15:47.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.209+0000 7f5534ff9640 1 -- 172.21.15.76:0/866344570 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f552c018070 con 0x7f553806e8a0 2024-04-25T19:15:47.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.209+0000 7f553e842640 1 -- 172.21.15.76:0/866344570 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55381f3490 con 0x7f553806e8a0 2024-04-25T19:15:47.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.209+0000 7f5534ff9640 1 -- 172.21.15.76:0/866344570 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c003830 con 0x7f553806e8a0 2024-04-25T19:15:47.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.209+0000 7f553e842640 1 -- 172.21.15.76:0/866344570 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55381f3a50 con 0x7f553806e8a0 2024-04-25T19:15:47.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.210+0000 7f553e842640 1 -- 172.21.15.76:0/866344570 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f553806d6a0 con 0x7f553806e8a0 2024-04-25T19:15:47.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.211+0000 7f5534ff9640 1 -- 172.21.15.76:0/866344570 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f552c0039d0 con 0x7f553806e8a0 2024-04-25T19:15:47.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.211+0000 7f5534ff9640 1 --2- 172.21.15.76:0/866344570 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f55240744b0 0x7f55240769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:47.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.211+0000 7f5534ff9640 1 -- 172.21.15.76:0/866344570 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f552c094d20 con 0x7f553806e8a0 2024-04-25T19:15:47.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.211+0000 7f5537fff640 1 --2- 172.21.15.76:0/866344570 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f55240744b0 0x7f55240769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:47.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.212+0000 7f5537fff640 1 --2- 172.21.15.76:0/866344570 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f55240744b0 0x7f55240769c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5528033000 tx=0x7f552802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:47.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.216+0000 7f5534ff9640 1 -- 172.21.15.76:0/866344570 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f552c05ef80 con 0x7f553806e8a0 2024-04-25T19:15:47.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.455+0000 7f553e842640 1 -- 172.21.15.76:0/866344570 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f553806a390 con 0x7f553806e8a0 2024-04-25T19:15:47.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.455+0000 7f5534ff9640 1 -- 172.21.15.76:0/866344570 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+6911 (secure 0 0 0) 0x7f552c063280 con 0x7f553806e8a0 2024-04-25T19:15:47.458 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:47.458 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/3003966182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3003966182},{"type":"v1","addr":"172.21.15.76:6836","nonce":3003966182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T19:15:18.769449+0000","modified":"2024-04-25T19:15:42.486997+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4284,"mds_1":4278,"mds_2":4241,"mds_3":4238,"mds_4":4235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4235":{"gid":4235,"name":"b","rank":4,"incarnation":15,"state":"up:creating","state_seq":1,"addr":"172.21.15.105:6836/1809877606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1809877606},{"type":"v1","addr":"172.21.15.105:6836","nonce":1809877606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4238":{"gid":4238,"name":"f","rank":3,"incarnation":13,"state":"up:active","state_seq":9,"addr":"172.21.15.105:6838/2724142282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2724142282},{"type":"v1","addr":"172.21.15.105:6838","nonce":2724142282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4241":{"gid":4241,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.105:6839/1798232582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6837","nonce":1798232582},{"type":"v1","addr":"172.21.15.105:6839","nonce":1798232582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.76:6833/2101494685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2101494685},{"type":"v1","addr":"172.21.15.76:6833","nonce":2101494685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4284":{"gid":4284,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6837/4226277412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":4226277412},{"type":"v1","addr":"172.21.15.76:6837","nonce":4226277412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284,4278,4241,4238]},"id":1}]} 2024-04-25T19:15:47.459 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 15 2024-04-25T19:15:47.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.457+0000 7f553e842640 1 -- 172.21.15.76:0/866344570 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f55240744b0 msgr2=0x7f55240769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:47.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.457+0000 7f553e842640 1 --2- 172.21.15.76:0/866344570 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f55240744b0 0x7f55240769c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5528033000 tx=0x7f552802e000 comp rx=0 tx=0).stop 2024-04-25T19:15:47.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.457+0000 7f553e842640 1 -- 172.21.15.76:0/866344570 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f553806e8a0 msgr2=0x7f5538148fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:47.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.457+0000 7f553e842640 1 --2- 172.21.15.76:0/866344570 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f553806e8a0 0x7f5538148fc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f552c00f5c0 tx=0x7f552c00fa80 comp rx=0 tx=0).stop 2024-04-25T19:15:47.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.457+0000 7f553e842640 1 -- 172.21.15.76:0/866344570 shutdown_connections 2024-04-25T19:15:47.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.457+0000 7f553e842640 1 --2- 172.21.15.76:0/866344570 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f55240744b0 0x7f55240769c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.457+0000 7f553e842640 1 --2- 172.21.15.76:0/866344570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55381e9800 0x7f55381e9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.457+0000 7f553e842640 1 --2- 172.21.15.76:0/866344570 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f553806e8a0 0x7f5538148fc0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.457+0000 7f553e842640 1 --2- 172.21.15.76:0/866344570 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f55200039b0 0x7f553815e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.457+0000 7f553e842640 1 -- 172.21.15.76:0/866344570 >> 172.21.15.76:0/866344570 conn(0x7f553806c5c0 msgr2=0x7f55380ca260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:47.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.457+0000 7f553e842640 1 -- 172.21.15.76:0/866344570 shutdown_connections 2024-04-25T19:15:47.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.457+0000 7f553e842640 1 -- 172.21.15.76:0/866344570 wait complete. 2024-04-25T19:15:47.472 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:15:47.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.579+0000 7eff2d815640 1 Processor -- start 2024-04-25T19:15:47.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.580+0000 7eff2d815640 1 -- start start 2024-04-25T19:15:47.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.581+0000 7eff2d815640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff280c96c0 0x7eff2815cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:47.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.581+0000 7eff2d815640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7eff28074650 0x7eff28160a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:47.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.581+0000 7eff2d815640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff2815d1c0 con 0x7eff2806c5c0 2024-04-25T19:15:47.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.581+0000 7eff2d815640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7eff2815d4a0 con 0x7eff28074650 2024-04-25T19:15:47.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.581+0000 7eff2d815640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7eff2815d780 con 0x7eff280c96c0 2024-04-25T19:15:47.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.581+0000 7eff277fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7eff2806c5c0 0x7eff28071270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53612/0 (socket says 172.21.15.76:53612) 2024-04-25T19:15:47.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.581+0000 7eff277fe640 1 -- 172.21.15.76:0/3821864691 learned_addr learned my addr 172.21.15.76:0/3821864691 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:47.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.581+0000 7eff267fc640 1 --2- 172.21.15.76:0/3821864691 >> v2:172.21.15.105:3300/0 conn(0x7eff28074650 0x7eff28160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:47.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.581+0000 7eff26ffd640 1 --2- 172.21.15.76:0/3821864691 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff280c96c0 0x7eff2815cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:47.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.581+0000 7eff25ffb640 1 -- 172.21.15.76:0/3821864691 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2096182276 0 0) 0x7eff2815d1c0 con 0x7eff2806c5c0 2024-04-25T19:15:47.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.581+0000 7eff25ffb640 1 -- 172.21.15.76:0/3821864691 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff10003520 con 0x7eff2806c5c0 2024-04-25T19:15:47.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.581+0000 7eff267fc640 1 -- 172.21.15.76:0/3821864691 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff280c96c0 msgr2=0x7eff2815cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:47.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.581+0000 7eff267fc640 1 --2- 172.21.15.76:0/3821864691 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff280c96c0 0x7eff2815cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.581+0000 7eff267fc640 1 -- 172.21.15.76:0/3821864691 >> v1:172.21.15.76:6789/0 conn(0x7eff2806c5c0 legacy=0x7eff28071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:47.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.581+0000 7eff25ffb640 1 -- 172.21.15.76:0/3821864691 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2139631999 0 0) 0x7eff10003520 con 0x7eff2806c5c0 2024-04-25T19:15:47.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.582+0000 7eff267fc640 1 -- 172.21.15.76:0/3821864691 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff28160210 con 0x7eff28074650 2024-04-25T19:15:47.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.582+0000 7eff267fc640 1 --2- 172.21.15.76:0/3821864691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff28074650 0x7eff28160a10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0097a0 tx=0x7eff1c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5633ae75dd382f21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:47.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.582+0000 7eff25ffb640 1 -- 172.21.15.76:0/3821864691 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c037070 con 0x7eff28074650 2024-04-25T19:15:47.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.582+0000 7eff25ffb640 1 -- 172.21.15.76:0/3821864691 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff1c02ac30 con 0x7eff28074650 2024-04-25T19:15:47.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.582+0000 7eff25ffb640 1 -- 172.21.15.76:0/3821864691 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20001670 con 0x7eff28074650 2024-04-25T19:15:47.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.583+0000 7eff2d815640 1 -- 172.21.15.76:0/3821864691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff28074650 msgr2=0x7eff28160a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:47.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.583+0000 7eff2d815640 1 --2- 172.21.15.76:0/3821864691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff28074650 0x7eff28160a10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0097a0 tx=0x7eff1c02a1a0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.583+0000 7eff2d815640 1 -- 172.21.15.76:0/3821864691 shutdown_connections 2024-04-25T19:15:47.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.583+0000 7eff2d815640 1 --2- 172.21.15.76:0/3821864691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff28074650 0x7eff28160a10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.583+0000 7eff2d815640 1 --2- 172.21.15.76:0/3821864691 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff280c96c0 0x7eff2815cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.583+0000 7eff2d815640 1 -- 172.21.15.76:0/3821864691 >> 172.21.15.76:0/3821864691 conn(0x7eff280cecc0 msgr2=0x7eff280cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:47.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.583+0000 7eff2d815640 1 -- 172.21.15.76:0/3821864691 shutdown_connections 2024-04-25T19:15:47.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.583+0000 7eff2d815640 1 -- 172.21.15.76:0/3821864691 wait complete. 2024-04-25T19:15:47.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.585+0000 7eff2d815640 1 Processor -- start 2024-04-25T19:15:47.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.585+0000 7eff2d815640 1 -- start start 2024-04-25T19:15:47.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.586+0000 7eff2d815640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff2806c5c0 0x7eff281e8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:47.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.586+0000 7eff2d815640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff28074650 0x7eff28159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:47.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.586+0000 7eff2d815640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff280c96c0 0x7eff2815c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:47.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.586+0000 7eff2d815640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7eff280ca060 con 0x7eff28074650 2024-04-25T19:15:47.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.586+0000 7eff2d815640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7eff280c7330 con 0x7eff2806c5c0 2024-04-25T19:15:47.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.586+0000 7eff2d815640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7eff2815da60 con 0x7eff280c96c0 2024-04-25T19:15:47.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.586+0000 7eff267fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff28074650 0x7eff28159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:47.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.586+0000 7eff267fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff28074650 0x7eff28159ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40068/0 (socket says 172.21.15.76:40068) 2024-04-25T19:15:47.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.586+0000 7eff267fc640 1 -- 172.21.15.76:0/1921560367 learned_addr learned my addr 172.21.15.76:0/1921560367 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:47.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.586+0000 7eff277fe640 1 --2- 172.21.15.76:0/1921560367 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff280c96c0 0x7eff2815c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:47.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.586+0000 7eff26ffd640 1 --2- 172.21.15.76:0/1921560367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff2806c5c0 0x7eff281e8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:47.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.587+0000 7eff267fc640 1 -- 172.21.15.76:0/1921560367 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff280c96c0 msgr2=0x7eff2815c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:47.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.587+0000 7eff267fc640 1 --2- 172.21.15.76:0/1921560367 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff280c96c0 0x7eff2815c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.587+0000 7eff267fc640 1 -- 172.21.15.76:0/1921560367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff2806c5c0 msgr2=0x7eff281e8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:47.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.587+0000 7eff267fc640 1 --2- 172.21.15.76:0/1921560367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff2806c5c0 0x7eff281e8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.587+0000 7eff267fc640 1 -- 172.21.15.76:0/1921560367 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff281f2970 con 0x7eff28074650 2024-04-25T19:15:47.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.587+0000 7eff277fe640 1 --2- 172.21.15.76:0/1921560367 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff280c96c0 0x7eff2815c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:47.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.587+0000 7eff267fc640 1 --2- 172.21.15.76:0/1921560367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff28074650 0x7eff28159ea0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0098c0 tx=0x7eff1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:47.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.587+0000 7eff0ffff640 1 -- 172.21.15.76:0/1921560367 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c037050 con 0x7eff28074650 2024-04-25T19:15:47.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.587+0000 7eff0ffff640 1 -- 172.21.15.76:0/1921560367 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff1c043070 con 0x7eff28074650 2024-04-25T19:15:47.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.587+0000 7eff2d815640 1 -- 172.21.15.76:0/1921560367 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff281f2c50 con 0x7eff28074650 2024-04-25T19:15:47.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.587+0000 7eff0ffff640 1 -- 172.21.15.76:0/1921560367 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c006e10 con 0x7eff28074650 2024-04-25T19:15:47.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.587+0000 7eff2d815640 1 -- 172.21.15.76:0/1921560367 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff281f3210 con 0x7eff28074650 2024-04-25T19:15:47.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.588+0000 7eff2d815640 1 -- 172.21.15.76:0/1921560367 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff2806a390 con 0x7eff28074650 2024-04-25T19:15:47.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.589+0000 7eff0ffff640 1 -- 172.21.15.76:0/1921560367 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7eff1c03e670 con 0x7eff28074650 2024-04-25T19:15:47.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.589+0000 7eff0ffff640 1 --2- 172.21.15.76:0/1921560367 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7eff140744e0 0x7eff140769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:47.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.589+0000 7eff0ffff640 1 -- 172.21.15.76:0/1921560367 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7eff1c0b59f0 con 0x7eff28074650 2024-04-25T19:15:47.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.589+0000 7eff26ffd640 1 --2- 172.21.15.76:0/1921560367 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7eff140744e0 0x7eff140769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:47.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.590+0000 7eff26ffd640 1 --2- 172.21.15.76:0/1921560367 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7eff140744e0 0x7eff140769f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7eff18011340 tx=0x7eff1800b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:47.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.594+0000 7eff0ffff640 1 -- 172.21.15.76:0/1921560367 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7eff1c0802a0 con 0x7eff28074650 2024-04-25T19:15:47.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.773+0000 7eff2d815640 1 -- 172.21.15.76:0/1921560367 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff28074430 con 0x7eff28074650 2024-04-25T19:15:47.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.774+0000 7eff0ffff640 1 -- 172.21.15.76:0/1921560367 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7eff1c07fba0 con 0x7eff28074650 2024-04-25T19:15:47.777 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:47.777 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":21,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","created":"2024-04-25T19:15:01.693137+0000","modified":"2024-04-25T19:15:20.401211+0000","last_up_change":"2024-04-25T19:15:11.891681+0000","last_in_change":"2024-04-25T19:15:03.898065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:15:13.846514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:15:16.606533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:15:17.719197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72a16c02-48f9-45aa-9447-fcf755ebf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409574998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409574998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409574998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409574998}]},"public_addr":"172.21.15.76:6817/1409574998","cluster_addr":"172.21.15.76:6819/1409574998","heartbeat_back_addr":"172.21.15.76:6823/1409574998","heartbeat_front_addr":"172.21.15.76:6821/1409574998","state":["exists","up"]},{"osd":1,"uuid":"818d50f0-76c6-45f8-bf79-5e304f14c5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6801","nonce":1302663821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6803","nonce":1302663821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6807","nonce":1302663821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6805","nonce":1302663821}]},"public_addr":"172.21.15.76:6801/1302663821","cluster_addr":"172.21.15.76:6803/1302663821","heartbeat_back_addr":"172.21.15.76:6807/1302663821","heartbeat_front_addr":"172.21.15.76:6805/1302663821","state":["exists","up"]},{"osd":2,"uuid":"5714c8f5-ae23-473c-8cf1-db73811acded","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6809","nonce":1992938181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6811","nonce":1992938181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6815","nonce":1992938181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6813","nonce":1992938181}]},"public_addr":"172.21.15.76:6809/1992938181","cluster_addr":"172.21.15.76:6811/1992938181","heartbeat_back_addr":"172.21.15.76:6815/1992938181","heartbeat_front_addr":"172.21.15.76:6813/1992938181","state":["exists","up"]},{"osd":3,"uuid":"13204cdb-d8e6-4f97-bc6e-9561f442decf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6825","nonce":3418257163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6827","nonce":3418257163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6831","nonce":3418257163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6829","nonce":3418257163}]},"public_addr":"172.21.15.76:6825/3418257163","cluster_addr":"172.21.15.76:6827/3418257163","heartbeat_back_addr":"172.21.15.76:6831/3418257163","heartbeat_front_addr":"172.21.15.76:6829/3418257163","state":["exists","up"]},{"osd":4,"uuid":"5bea5166-1986-4d56-b5f5-ecc8cbe7f3ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6801","nonce":2125002468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6803","nonce":2125002468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6807","nonce":2125002468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6805","nonce":2125002468}]},"public_addr":"172.21.15.105:6801/2125002468","cluster_addr":"172.21.15.105:6803/2125002468","heartbeat_back_addr":"172.21.15.105:6807/2125002468","heartbeat_front_addr":"172.21.15.105:6805/2125002468","state":["exists","up"]},{"osd":5,"uuid":"96f89dcc-270d-4421-b8e8-3b92ba5f9844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6819","nonce":3413103121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6821","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6823","nonce":3413103121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6831","nonce":3413103121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6828","nonce":3413103121}]},"public_addr":"172.21.15.105:6819/3413103121","cluster_addr":"172.21.15.105:6823/3413103121","heartbeat_back_addr":"172.21.15.105:6831/3413103121","heartbeat_front_addr":"172.21.15.105:6828/3413103121","state":["exists","up"]},{"osd":6,"uuid":"93eb087a-d9ba-4096-b85d-b74d96e819a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6809","nonce":2721832954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6811","nonce":2721832954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6815","nonce":2721832954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6813","nonce":2721832954}]},"public_addr":"172.21.15.105:6809/2721832954","cluster_addr":"172.21.15.105:6811/2721832954","heartbeat_back_addr":"172.21.15.105:6815/2721832954","heartbeat_front_addr":"172.21.15.105:6813/2721832954","state":["exists","up"]},{"osd":7,"uuid":"2f80f790-ad2d-460e-a37d-47bf1771ef43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6818","nonce":548350430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6822","nonce":548350430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6829","nonce":548350430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6825","nonce":548350430}]},"public_addr":"172.21.15.105:6818/548350430","cluster_addr":"172.21.15.105:6822/548350430","heartbeat_back_addr":"172.21.15.105:6829/548350430","heartbeat_front_addr":"172.21.15.105:6825/548350430","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.106901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.072819+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.126865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.053464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:08.982614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.104825+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.112714+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.107387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:15:47.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.776+0000 7eff2d815640 1 -- 172.21.15.76:0/1921560367 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7eff140744e0 msgr2=0x7eff140769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:47.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.776+0000 7eff2d815640 1 --2- 172.21.15.76:0/1921560367 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7eff140744e0 0x7eff140769f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7eff18011340 tx=0x7eff1800b040 comp rx=0 tx=0).stop 2024-04-25T19:15:47.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.776+0000 7eff2d815640 1 -- 172.21.15.76:0/1921560367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff28074650 msgr2=0x7eff28159ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:47.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.776+0000 7eff2d815640 1 --2- 172.21.15.76:0/1921560367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff28074650 0x7eff28159ea0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0098c0 tx=0x7eff1c0046c0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.776+0000 7eff2d815640 1 -- 172.21.15.76:0/1921560367 shutdown_connections 2024-04-25T19:15:47.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.776+0000 7eff2d815640 1 --2- 172.21.15.76:0/1921560367 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7eff140744e0 0x7eff140769f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.776+0000 7eff2d815640 1 --2- 172.21.15.76:0/1921560367 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff280c96c0 0x7eff2815c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.776+0000 7eff2d815640 1 --2- 172.21.15.76:0/1921560367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff28074650 0x7eff28159ea0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.776+0000 7eff2d815640 1 --2- 172.21.15.76:0/1921560367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff2806c5c0 0x7eff281e8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.776+0000 7eff2d815640 1 -- 172.21.15.76:0/1921560367 >> 172.21.15.76:0/1921560367 conn(0x7eff280cecc0 msgr2=0x7eff28152960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:47.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.776+0000 7eff2d815640 1 -- 172.21.15.76:0/1921560367 shutdown_connections 2024-04-25T19:15:47.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.776+0000 7eff2d815640 1 -- 172.21.15.76:0/1921560367 wait complete. 2024-04-25T19:15:47.791 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:15:47.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.897+0000 7f9fcafba640 1 Processor -- start 2024-04-25T19:15:47.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.897+0000 7f9fcafba640 1 -- start start 2024-04-25T19:15:47.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.899+0000 7f9fcafba640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9fc415cd10 0x7f9fc4156c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:47.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.899+0000 7f9fcafba640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9fc406e0c0 0x7f9fc406e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:47.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.899+0000 7f9fcafba640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fc41571c0 con 0x7f9fc41609c0 2024-04-25T19:15:47.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.899+0000 7f9fcafba640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f9fc415d250 con 0x7f9fc415cd10 2024-04-25T19:15:47.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.899+0000 7f9fcafba640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc415d510 con 0x7f9fc406e0c0 2024-04-25T19:15:47.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.899+0000 7f9fc9557640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f9fc41609c0 0x7f9fc4072ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53626/0 (socket says 172.21.15.76:53626) 2024-04-25T19:15:47.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.899+0000 7f9fc9557640 1 -- 172.21.15.76:0/1929959199 learned_addr learned my addr 172.21.15.76:0/1929959199 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:47.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.899+0000 7f9fc37fe640 1 -- 172.21.15.76:0/1929959199 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 658702031 0 0) 0x7f9fc41571c0 con 0x7f9fc41609c0 2024-04-25T19:15:47.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.899+0000 7f9fc8d56640 1 --2- 172.21.15.76:0/1929959199 >> v2:172.21.15.105:3300/0 conn(0x7f9fc415cd10 0x7f9fc4156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:47.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.899+0000 7f9fc3fff640 1 --2- 172.21.15.76:0/1929959199 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9fc406e0c0 0x7f9fc406e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:47.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.899+0000 7f9fc37fe640 1 -- 172.21.15.76:0/1929959199 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fb0003520 con 0x7f9fc41609c0 2024-04-25T19:15:47.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.899+0000 7f9fc37fe640 1 -- 172.21.15.76:0/1929959199 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3748963990 0 0) 0x7f9fb0003520 con 0x7f9fc41609c0 2024-04-25T19:15:47.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.899+0000 7f9fc37fe640 1 -- 172.21.15.76:0/1929959199 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9fc406e0c0 msgr2=0x7f9fc406e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:47.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.899+0000 7f9fc37fe640 1 --2- 172.21.15.76:0/1929959199 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9fc406e0c0 0x7f9fc406e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.899+0000 7f9fc37fe640 1 -- 172.21.15.76:0/1929959199 >> v2:172.21.15.105:3300/0 conn(0x7f9fc415cd10 msgr2=0x7f9fc4156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:47.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.899+0000 7f9fc37fe640 1 --2- 172.21.15.76:0/1929959199 >> v2:172.21.15.105:3300/0 conn(0x7f9fc415cd10 0x7f9fc4156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.899+0000 7f9fc37fe640 1 -- 172.21.15.76:0/1929959199 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc415d7f0 con 0x7f9fc41609c0 2024-04-25T19:15:47.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.899+0000 7f9fc37fe640 1 -- 172.21.15.76:0/1929959199 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f9fac001670 con 0x7f9fc41609c0 2024-04-25T19:15:47.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.899+0000 7f9fc8d56640 1 --2- 172.21.15.76:0/1929959199 >> v2:172.21.15.105:3300/0 conn(0x7f9fc415cd10 0x7f9fc4156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:47.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.899+0000 7f9fc37fe640 1 -- 172.21.15.76:0/1929959199 >> v1:172.21.15.76:6789/0 conn(0x7f9fc41609c0 legacy=0x7f9fc4072ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:47.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.899+0000 7f9fc37fe640 1 --2- 172.21.15.76:0/1929959199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb00039b0 0x7f9fb0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:47.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.899+0000 7f9fc37fe640 1 --2- 172.21.15.76:0/1929959199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fb0004800 0x7f9fb0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:47.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.899+0000 7f9fc37fe640 1 --2- 172.21.15.76:0/1929959199 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9fc406e0c0 0x7f9fb0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:47.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.899+0000 7f9fc37fe640 1 -- 172.21.15.76:0/1929959199 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc415d250 con 0x7f9fb0004800 2024-04-25T19:15:47.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.899+0000 7f9fc37fe640 1 -- 172.21.15.76:0/1929959199 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc415d510 con 0x7f9fb00039b0 2024-04-25T19:15:47.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.899+0000 7f9fc37fe640 1 -- 172.21.15.76:0/1929959199 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f9fb000bb50 con 0x7f9fc406e0c0 2024-04-25T19:15:47.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.900+0000 7f9fc3fff640 1 --2- 172.21.15.76:0/1929959199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fb0004800 0x7f9fb0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:47.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.900+0000 7f9fc8d56640 1 --2- 172.21.15.76:0/1929959199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb00039b0 0x7f9fb0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:47.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.900+0000 7f9fc9557640 1 --2- 172.21.15.76:0/1929959199 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9fc406e0c0 0x7f9fb0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:47.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.900+0000 7f9fc3fff640 1 -- 172.21.15.76:0/1929959199 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9fc406e0c0 msgr2=0x7f9fb0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:47.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.900+0000 7f9fc3fff640 1 --2- 172.21.15.76:0/1929959199 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9fc406e0c0 0x7f9fb0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.900+0000 7f9fc3fff640 1 -- 172.21.15.76:0/1929959199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb00039b0 msgr2=0x7f9fb0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:47.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.900+0000 7f9fc3fff640 1 --2- 172.21.15.76:0/1929959199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fb00039b0 0x7f9fb0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.900+0000 7f9fc3fff640 1 -- 172.21.15.76:0/1929959199 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb0003520 con 0x7f9fb0004800 2024-04-25T19:15:47.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.900+0000 7f9fc9557640 1 --2- 172.21.15.76:0/1929959199 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9fc406e0c0 0x7f9fb0005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.900+0000 7f9fc3fff640 1 --2- 172.21.15.76:0/1929959199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fb0004800 0x7f9fb0004c10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400ba00 tx=0x7f9fb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a59646ffc409e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.901+0000 7f9fc37fe640 1 -- 172.21.15.76:0/1929959199 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4030070 con 0x7f9fb0004800 2024-04-25T19:15:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.901+0000 7f9fc37fe640 1 -- 172.21.15.76:0/1929959199 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb402adb0 con 0x7f9fb0004800 2024-04-25T19:15:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.901+0000 7f9fc37fe640 1 -- 172.21.15.76:0/1929959199 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb400b600 con 0x7f9fb0004800 2024-04-25T19:15:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.901+0000 7f9fcafba640 1 -- 172.21.15.76:0/1929959199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fb0004800 msgr2=0x7f9fb0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.901+0000 7f9fcafba640 1 --2- 172.21.15.76:0/1929959199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fb0004800 0x7f9fb0004c10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400ba00 tx=0x7f9fb0005830 comp rx=0 tx=0).stop 2024-04-25T19:15:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.901+0000 7f9fc9557640 1 -- 172.21.15.76:0/1929959199 reap_dead start 2024-04-25T19:15:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.901+0000 7f9fcafba640 1 -- 172.21.15.76:0/1929959199 shutdown_connections 2024-04-25T19:15:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.901+0000 7f9fcafba640 1 -- 172.21.15.76:0/1929959199 >> 172.21.15.76:0/1929959199 conn(0x7f9fc4070b00 msgr2=0x7f9fc4070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.901+0000 7f9fcafba640 1 -- 172.21.15.76:0/1929959199 shutdown_connections 2024-04-25T19:15:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.901+0000 7f9fcafba640 1 -- 172.21.15.76:0/1929959199 wait complete. 2024-04-25T19:15:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.903+0000 7f9fcafba640 1 Processor -- start 2024-04-25T19:15:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.903+0000 7f9fcafba640 1 -- start start 2024-04-25T19:15:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.904+0000 7f9fcafba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fb0004800 0x7f9fc40b3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.904+0000 7f9fcafba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fc415a3b0 0x7f9fc41533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.904+0000 7f9fcafba640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9fc40b4a60 0x7f9fc41e77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.904+0000 7f9fcafba640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc40c5210 con 0x7f9fb0004800 2024-04-25T19:15:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.904+0000 7f9fcafba640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc41604a0 con 0x7f9fc415a3b0 2024-04-25T19:15:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.904+0000 7f9fcafba640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc40c5390 con 0x7f9fc40b4a60 2024-04-25T19:15:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.904+0000 7f9fc8d56640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fb0004800 0x7f9fc40b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.905+0000 7f9fc8d56640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fb0004800 0x7f9fc40b3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40086/0 (socket says 172.21.15.76:40086) 2024-04-25T19:15:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.905+0000 7f9fc8d56640 1 -- 172.21.15.76:0/1300594782 learned_addr learned my addr 172.21.15.76:0/1300594782 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.905+0000 7f9fc8d56640 1 -- 172.21.15.76:0/1300594782 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9fc40b4a60 msgr2=0x7f9fc41e77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:47.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.905+0000 7f9fc8d56640 1 --2- 172.21.15.76:0/1300594782 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9fc40b4a60 0x7f9fc41e77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.905+0000 7f9fc8d56640 1 -- 172.21.15.76:0/1300594782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fc415a3b0 msgr2=0x7f9fc41533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:47.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.905+0000 7f9fc8d56640 1 --2- 172.21.15.76:0/1300594782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fc415a3b0 0x7f9fc41533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:47.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.905+0000 7f9fc8d56640 1 -- 172.21.15.76:0/1300594782 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc41f4390 con 0x7f9fb0004800 2024-04-25T19:15:47.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.905+0000 7f9fc8d56640 1 --2- 172.21.15.76:0/1300594782 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fb0004800 0x7f9fc40b3770 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc0103e0 tx=0x7f9fbc0108a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:47.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.905+0000 7f9fc17fa640 1 -- 172.21.15.76:0/1300594782 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc003990 con 0x7f9fb0004800 2024-04-25T19:15:47.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.905+0000 7f9fc17fa640 1 -- 172.21.15.76:0/1300594782 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fbc003b30 con 0x7f9fb0004800 2024-04-25T19:15:47.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.905+0000 7f9fcafba640 1 -- 172.21.15.76:0/1300594782 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc40b3cb0 con 0x7f9fb0004800 2024-04-25T19:15:47.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.905+0000 7f9fc17fa640 1 -- 172.21.15.76:0/1300594782 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc003d20 con 0x7f9fb0004800 2024-04-25T19:15:47.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.905+0000 7f9fcafba640 1 -- 172.21.15.76:0/1300594782 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc40b4620 con 0x7f9fb0004800 2024-04-25T19:15:47.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.906+0000 7f9fcafba640 1 -- 172.21.15.76:0/1300594782 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fc4077f40 con 0x7f9fb0004800 2024-04-25T19:15:47.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.907+0000 7f9fc17fa640 1 -- 172.21.15.76:0/1300594782 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9fbc01e020 con 0x7f9fb0004800 2024-04-25T19:15:47.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.907+0000 7f9fc17fa640 1 --2- 172.21.15.76:0/1300594782 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9fa40744e0 0x7f9fa40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:47.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.907+0000 7f9fc17fa640 1 -- 172.21.15.76:0/1300594782 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9fbc095bb0 con 0x7f9fb0004800 2024-04-25T19:15:47.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.907+0000 7f9fc3fff640 1 --2- 172.21.15.76:0/1300594782 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9fa40744e0 0x7f9fa40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:47.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.908+0000 7f9fc3fff640 1 --2- 172.21.15.76:0/1300594782 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9fa40744e0 0x7f9fa40769f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9fc406d8f0 tx=0x7f9fb402c000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:47.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:47.912+0000 7f9fc17fa640 1 -- 172.21.15.76:0/1300594782 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9fc4077f40 con 0x7f9fb0004800 2024-04-25T19:15:48.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:48.127+0000 7f9fcafba640 1 -- 172.21.15.76:0/1300594782 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9fc406a390 con 0x7f9fb0004800 2024-04-25T19:15:48.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:48.128+0000 7f9fc17fa640 1 -- 172.21.15.76:0/1300594782 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+6911 (secure 0 0 0) 0x7f9fbc004020 con 0x7f9fb0004800 2024-04-25T19:15:48.131 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:48.131 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/3003966182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3003966182},{"type":"v1","addr":"172.21.15.76:6836","nonce":3003966182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T19:15:18.769449+0000","modified":"2024-04-25T19:15:42.486997+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4284,"mds_1":4278,"mds_2":4241,"mds_3":4238,"mds_4":4235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4235":{"gid":4235,"name":"b","rank":4,"incarnation":15,"state":"up:creating","state_seq":1,"addr":"172.21.15.105:6836/1809877606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1809877606},{"type":"v1","addr":"172.21.15.105:6836","nonce":1809877606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4238":{"gid":4238,"name":"f","rank":3,"incarnation":13,"state":"up:active","state_seq":9,"addr":"172.21.15.105:6838/2724142282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2724142282},{"type":"v1","addr":"172.21.15.105:6838","nonce":2724142282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4241":{"gid":4241,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.105:6839/1798232582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6837","nonce":1798232582},{"type":"v1","addr":"172.21.15.105:6839","nonce":1798232582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.76:6833/2101494685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2101494685},{"type":"v1","addr":"172.21.15.76:6833","nonce":2101494685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4284":{"gid":4284,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6837/4226277412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":4226277412},{"type":"v1","addr":"172.21.15.76:6837","nonce":4226277412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284,4278,4241,4238]},"id":1}]} 2024-04-25T19:15:48.131 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 15 2024-04-25T19:15:48.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:48.130+0000 7f9fcafba640 1 -- 172.21.15.76:0/1300594782 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9fa40744e0 msgr2=0x7f9fa40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:48.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:48.130+0000 7f9fcafba640 1 --2- 172.21.15.76:0/1300594782 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9fa40744e0 0x7f9fa40769f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9fc406d8f0 tx=0x7f9fb402c000 comp rx=0 tx=0).stop 2024-04-25T19:15:48.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:48.130+0000 7f9fcafba640 1 -- 172.21.15.76:0/1300594782 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fb0004800 msgr2=0x7f9fc40b3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:48.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:48.130+0000 7f9fcafba640 1 --2- 172.21.15.76:0/1300594782 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fb0004800 0x7f9fc40b3770 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc0103e0 tx=0x7f9fbc0108a0 comp rx=0 tx=0).stop 2024-04-25T19:15:48.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:48.130+0000 7f9fcafba640 1 -- 172.21.15.76:0/1300594782 shutdown_connections 2024-04-25T19:15:48.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:48.130+0000 7f9fcafba640 1 --2- 172.21.15.76:0/1300594782 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9fa40744e0 0x7f9fa40769f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:48.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:48.130+0000 7f9fcafba640 1 --2- 172.21.15.76:0/1300594782 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9fc40b4a60 0x7f9fc41e77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:48.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:48.130+0000 7f9fcafba640 1 --2- 172.21.15.76:0/1300594782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9fc415a3b0 0x7f9fc41533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:48.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:48.130+0000 7f9fcafba640 1 --2- 172.21.15.76:0/1300594782 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fb0004800 0x7f9fc40b3770 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:48.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:48.131+0000 7f9fcafba640 1 -- 172.21.15.76:0/1300594782 >> 172.21.15.76:0/1300594782 conn(0x7f9fc4070b00 msgr2=0x7f9fc415ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:48.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:48.131+0000 7f9fcafba640 1 -- 172.21.15.76:0/1300594782 shutdown_connections 2024-04-25T19:15:48.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:48.131+0000 7f9fcafba640 1 -- 172.21.15.76:0/1300594782 wait complete. 2024-04-25T19:15:48.748 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T19:15:18.769449+0000', 'modified': '2024-04-25T19:15:42.486997+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4284, 'mds_1': 4278, 'mds_2': 4241, 'mds_3': 4238, 'mds_4': 4235}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4235': {'gid': 4235, 'name': 'b', 'rank': 4, 'incarnation': 15, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.105:6836/1809877606', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 1809877606}, {'type': 'v1', 'addr': '172.21.15.105:6836', 'nonce': 1809877606}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4238': {'gid': 4238, 'name': 'f', 'rank': 3, 'incarnation': 13, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.105:6838/2724142282', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6835', 'nonce': 2724142282}, {'type': 'v1', 'addr': '172.21.15.105:6838', 'nonce': 2724142282}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4241': {'gid': 4241, 'name': 'd', 'rank': 2, 'incarnation': 11, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.105:6839/1798232582', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6837', 'nonce': 1798232582}, {'type': 'v1', 'addr': '172.21.15.105:6839', 'nonce': 1798232582}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4278': {'gid': 4278, 'name': 'a', 'rank': 1, 'incarnation': 9, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.76:6833/2101494685', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 2101494685}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 2101494685}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4284': {'gid': 4284, 'name': 'e', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.76:6837/4226277412', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6835', 'nonce': 4226277412}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 4226277412}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4284, 'qdb_cluster': [4284, 4278, 4241, 4238]} 2024-04-25T19:15:48.749 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4235:up:creating 2024-04-25T19:15:49.749 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:15:49.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.856+0000 7f1aaf870640 1 Processor -- start 2024-04-25T19:15:49.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.856+0000 7f1aaf870640 1 -- start start 2024-04-25T19:15:49.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.858+0000 7f1aaf870640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1aa80cc840 0x7f1aa80cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:49.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.858+0000 7f1aaf870640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1aa806e0e0 0x7f1aa806e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:49.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.858+0000 7f1aaf870640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1aa80cb830 con 0x7f1aa80cab50 2024-04-25T19:15:49.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.858+0000 7f1aaf870640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f1aa80cbb10 con 0x7f1aa806e0e0 2024-04-25T19:15:49.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.858+0000 7f1aaf870640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f1aa80ce080 con 0x7f1aa80cc840 2024-04-25T19:15:49.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.858+0000 7f1aade0d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1aa806e0e0 0x7f1aa806e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:49.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.858+0000 7f1aade0d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1aa806e0e0 0x7f1aa806e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37750/0 (socket says 172.21.15.76:37750) 2024-04-25T19:15:49.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.858+0000 7f1aade0d640 1 -- 172.21.15.76:0/3518104370 learned_addr learned my addr 172.21.15.76:0/3518104370 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:49.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.858+0000 7f1aad60c640 1 --2- 172.21.15.76:0/3518104370 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1aa80cc840 0x7f1aa80cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:49.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.858+0000 7f1aade0d640 1 -- 172.21.15.76:0/3518104370 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1aa80cc840 msgr2=0x7f1aa80cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:49.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.858+0000 7f1aade0d640 1 --2- 172.21.15.76:0/3518104370 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1aa80cc840 0x7f1aa80cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:49.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.858+0000 7f1aade0d640 1 -- 172.21.15.76:0/3518104370 >> v1:172.21.15.76:6789/0 conn(0x7f1aa80cab50 legacy=0x7f1aa80ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:49.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.858+0000 7f1aade0d640 1 -- 172.21.15.76:0/3518104370 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa80ce360 con 0x7f1aa806e0e0 2024-04-25T19:15:49.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.859+0000 7f1aade0d640 1 --2- 172.21.15.76:0/3518104370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa806e0e0 0x7f1aa806e4f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009e50 tx=0x7f1aa002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b037e2298b1de6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:49.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.859+0000 7f1a9ffff640 1 -- 172.21.15.76:0/3518104370 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0036070 con 0x7f1aa806e0e0 2024-04-25T19:15:49.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.859+0000 7f1a9ffff640 1 -- 172.21.15.76:0/3518104370 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aa0031600 con 0x7f1aa806e0e0 2024-04-25T19:15:49.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.859+0000 7f1a9ffff640 1 -- 172.21.15.76:0/3518104370 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa00319d0 con 0x7f1aa806e0e0 2024-04-25T19:15:49.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.860+0000 7f1aaf870640 1 -- 172.21.15.76:0/3518104370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa806e0e0 msgr2=0x7f1aa806e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:49.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.860+0000 7f1aaf870640 1 --2- 172.21.15.76:0/3518104370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa806e0e0 0x7f1aa806e4f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009e50 tx=0x7f1aa002a710 comp rx=0 tx=0).stop 2024-04-25T19:15:49.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.860+0000 7f1aaf870640 1 -- 172.21.15.76:0/3518104370 shutdown_connections 2024-04-25T19:15:49.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.860+0000 7f1aaf870640 1 --2- 172.21.15.76:0/3518104370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa806e0e0 0x7f1aa806e4f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:49.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.860+0000 7f1aaf870640 1 --2- 172.21.15.76:0/3518104370 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1aa80cc840 0x7f1aa80cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:49.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.860+0000 7f1aaf870640 1 -- 172.21.15.76:0/3518104370 >> 172.21.15.76:0/3518104370 conn(0x7f1aa8072b60 msgr2=0x7f1aa8072f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:49.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.860+0000 7f1aaf870640 1 -- 172.21.15.76:0/3518104370 shutdown_connections 2024-04-25T19:15:49.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.860+0000 7f1aaf870640 1 -- 172.21.15.76:0/3518104370 wait complete. 2024-04-25T19:15:49.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.861+0000 7f1aaf870640 1 Processor -- start 2024-04-25T19:15:49.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.862+0000 7f1aaf870640 1 -- start start 2024-04-25T19:15:49.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.863+0000 7f1aaf870640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa806e0e0 0x7f1aa81f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:49.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.863+0000 7f1aaf870640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1aa80cab50 0x7f1aa80b2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:49.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.863+0000 7f1aaf870640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1aa80cc840 0x7f1aa8155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:49.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.863+0000 7f1aaf870640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa80cb0e0 con 0x7f1aa806e0e0 2024-04-25T19:15:49.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.863+0000 7f1aaf870640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa80caf60 con 0x7f1aa80cc840 2024-04-25T19:15:49.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.863+0000 7f1aaf870640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f1aa8160050 con 0x7f1aa80cab50 2024-04-25T19:15:49.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.863+0000 7f1aad60c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa806e0e0 0x7f1aa81f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:49.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.863+0000 7f1aad60c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa806e0e0 0x7f1aa81f6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37766/0 (socket says 172.21.15.76:37766) 2024-04-25T19:15:49.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.863+0000 7f1aad60c640 1 -- 172.21.15.76:0/1498827289 learned_addr learned my addr 172.21.15.76:0/1498827289 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:49.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.863+0000 7f1aace0b640 1 --2- 172.21.15.76:0/1498827289 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1aa80cab50 0x7f1aa80b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:49.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.863+0000 7f1aad60c640 1 -- 172.21.15.76:0/1498827289 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1aa80cab50 msgr2=0x7f1aa80b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:49.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.863+0000 7f1aad60c640 1 --2- 172.21.15.76:0/1498827289 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1aa80cab50 0x7f1aa80b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:49.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.863+0000 7f1aad60c640 1 -- 172.21.15.76:0/1498827289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1aa80cc840 msgr2=0x7f1aa8155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:49.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.863+0000 7f1aade0d640 1 --2- 172.21.15.76:0/1498827289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1aa80cc840 0x7f1aa8155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:49.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.863+0000 7f1aad60c640 1 --2- 172.21.15.76:0/1498827289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1aa80cc840 0x7f1aa8155f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:49.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.863+0000 7f1aad60c640 1 -- 172.21.15.76:0/1498827289 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa81f7180 con 0x7f1aa806e0e0 2024-04-25T19:15:49.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.863+0000 7f1aace0b640 1 --2- 172.21.15.76:0/1498827289 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1aa80cab50 0x7f1aa80b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:15:49.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.863+0000 7f1aad60c640 1 --2- 172.21.15.76:0/1498827289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa806e0e0 0x7f1aa81f6c40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800d620 tx=0x7f1a9800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:49.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.863+0000 7f1a9dffb640 1 -- 172.21.15.76:0/1498827289 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9800b620 con 0x7f1aa806e0e0 2024-04-25T19:15:49.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.863+0000 7f1a9dffb640 1 -- 172.21.15.76:0/1498827289 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a98019070 con 0x7f1aa806e0e0 2024-04-25T19:15:49.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.863+0000 7f1aaf870640 1 -- 172.21.15.76:0/1498827289 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa80b5130 con 0x7f1aa806e0e0 2024-04-25T19:15:49.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.863+0000 7f1a9dffb640 1 -- 172.21.15.76:0/1498827289 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9800ba00 con 0x7f1aa806e0e0 2024-04-25T19:15:49.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.864+0000 7f1aaf870640 1 -- 172.21.15.76:0/1498827289 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa80b53d0 con 0x7f1aa806e0e0 2024-04-25T19:15:49.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.865+0000 7f1aaf870640 1 -- 172.21.15.76:0/1498827289 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aa806a390 con 0x7f1aa806e0e0 2024-04-25T19:15:49.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.865+0000 7f1a9dffb640 1 -- 172.21.15.76:0/1498827289 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1a9800bba0 con 0x7f1aa806e0e0 2024-04-25T19:15:49.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.865+0000 7f1a9dffb640 1 --2- 172.21.15.76:0/1498827289 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1a8c0744b0 0x7f1a8c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:49.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.866+0000 7f1a9dffb640 1 -- 172.21.15.76:0/1498827289 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1a980956b0 con 0x7f1aa806e0e0 2024-04-25T19:15:49.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.866+0000 7f1aace0b640 1 --2- 172.21.15.76:0/1498827289 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1a8c0744b0 0x7f1a8c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:49.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.866+0000 7f1aace0b640 1 --2- 172.21.15.76:0/1498827289 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1a8c0744b0 0x7f1a8c0769c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1aa81edbf0 tx=0x7f1aa4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:49.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:49.870+0000 7f1a9dffb640 1 -- 172.21.15.76:0/1498827289 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1a9805fe40 con 0x7f1aa806e0e0 2024-04-25T19:15:50.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.090+0000 7f1aaf870640 1 -- 172.21.15.76:0/1498827289 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1aa80b55d0 con 0x7f1aa806e0e0 2024-04-25T19:15:50.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.091+0000 7f1a9dffb640 1 -- 172.21.15.76:0/1498827289 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6915 (secure 0 0 0) 0x7f1a9805f740 con 0x7f1aa806e0e0 2024-04-25T19:15:50.094 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:50.094 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/3003966182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3003966182},{"type":"v1","addr":"172.21.15.76:6836","nonce":3003966182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T19:15:18.769449+0000","modified":"2024-04-25T19:15:48.500543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4284,"mds_1":4278,"mds_2":4241,"mds_3":4238,"mds_4":4235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4235":{"gid":4235,"name":"b","rank":4,"incarnation":15,"state":"up:active","state_seq":10,"addr":"172.21.15.105:6836/1809877606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1809877606},{"type":"v1","addr":"172.21.15.105:6836","nonce":1809877606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4238":{"gid":4238,"name":"f","rank":3,"incarnation":13,"state":"up:active","state_seq":9,"addr":"172.21.15.105:6838/2724142282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2724142282},{"type":"v1","addr":"172.21.15.105:6838","nonce":2724142282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4241":{"gid":4241,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.105:6839/1798232582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6837","nonce":1798232582},{"type":"v1","addr":"172.21.15.105:6839","nonce":1798232582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.76:6833/2101494685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2101494685},{"type":"v1","addr":"172.21.15.76:6833","nonce":2101494685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4284":{"gid":4284,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6837/4226277412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":4226277412},{"type":"v1","addr":"172.21.15.76:6837","nonce":4226277412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4235,4238,4241,4278,4284]},"id":1}]} 2024-04-25T19:15:50.094 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 16 2024-04-25T19:15:50.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.093+0000 7f1aaf870640 1 -- 172.21.15.76:0/1498827289 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1a8c0744b0 msgr2=0x7f1a8c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:50.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.093+0000 7f1aaf870640 1 --2- 172.21.15.76:0/1498827289 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1a8c0744b0 0x7f1a8c0769c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1aa81edbf0 tx=0x7f1aa4009040 comp rx=0 tx=0).stop 2024-04-25T19:15:50.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.093+0000 7f1aaf870640 1 -- 172.21.15.76:0/1498827289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa806e0e0 msgr2=0x7f1aa81f6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:50.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.093+0000 7f1aaf870640 1 --2- 172.21.15.76:0/1498827289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa806e0e0 0x7f1aa81f6c40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800d620 tx=0x7f1a9800dae0 comp rx=0 tx=0).stop 2024-04-25T19:15:50.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.093+0000 7f1aaf870640 1 -- 172.21.15.76:0/1498827289 shutdown_connections 2024-04-25T19:15:50.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.093+0000 7f1aaf870640 1 --2- 172.21.15.76:0/1498827289 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1a8c0744b0 0x7f1a8c0769c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:50.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.093+0000 7f1aaf870640 1 --2- 172.21.15.76:0/1498827289 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1aa80cc840 0x7f1aa8155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:50.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.093+0000 7f1aaf870640 1 --2- 172.21.15.76:0/1498827289 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1aa80cab50 0x7f1aa80b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:50.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.093+0000 7f1aaf870640 1 --2- 172.21.15.76:0/1498827289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aa806e0e0 0x7f1aa81f6c40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:50.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.093+0000 7f1aaf870640 1 -- 172.21.15.76:0/1498827289 >> 172.21.15.76:0/1498827289 conn(0x7f1aa8072b60 msgr2=0x7f1aa80c66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:50.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.093+0000 7f1aaf870640 1 -- 172.21.15.76:0/1498827289 shutdown_connections 2024-04-25T19:15:50.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.093+0000 7f1aaf870640 1 -- 172.21.15.76:0/1498827289 wait complete. 2024-04-25T19:15:50.710 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T19:15:18.769449+0000', 'modified': '2024-04-25T19:15:48.500543+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4284, 'mds_1': 4278, 'mds_2': 4241, 'mds_3': 4238, 'mds_4': 4235}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4235': {'gid': 4235, 'name': 'b', 'rank': 4, 'incarnation': 15, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.105:6836/1809877606', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 1809877606}, {'type': 'v1', 'addr': '172.21.15.105:6836', 'nonce': 1809877606}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4238': {'gid': 4238, 'name': 'f', 'rank': 3, 'incarnation': 13, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.105:6838/2724142282', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6835', 'nonce': 2724142282}, {'type': 'v1', 'addr': '172.21.15.105:6838', 'nonce': 2724142282}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4241': {'gid': 4241, 'name': 'd', 'rank': 2, 'incarnation': 11, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.105:6839/1798232582', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6837', 'nonce': 1798232582}, {'type': 'v1', 'addr': '172.21.15.105:6839', 'nonce': 1798232582}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4278': {'gid': 4278, 'name': 'a', 'rank': 1, 'incarnation': 9, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.76:6833/2101494685', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6832', 'nonce': 2101494685}, {'type': 'v1', 'addr': '172.21.15.76:6833', 'nonce': 2101494685}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4284': {'gid': 4284, 'name': 'e', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.76:6837/4226277412', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6835', 'nonce': 4226277412}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 4226277412}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4284, 'qdb_cluster': [4235, 4238, 4241, 4278, 4284]} 2024-04-25T19:15:50.711 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T19:15:50.711 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T19:15:50.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.818+0000 7ff3cb71f640 1 Processor -- start 2024-04-25T19:15:50.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.818+0000 7ff3cb71f640 1 -- start start 2024-04-25T19:15:50.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.820+0000 7ff3cb71f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3c40c5350 0x7ff3c4070a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:50.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.820+0000 7ff3cb71f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff3c40c5760 0x7ff3c4070f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:50.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.820+0000 7ff3cb71f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3c40ca940 con 0x7ff3c415dd40 2024-04-25T19:15:50.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.820+0000 7ff3cb71f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7ff3c40cb7f0 con 0x7ff3c40c5760 2024-04-25T19:15:50.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.820+0000 7ff3cb71f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff3c40cabe0 con 0x7ff3c40c5350 2024-04-25T19:15:50.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.820+0000 7ff3c8cba640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3c40c5350 0x7ff3c4070a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:50.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.820+0000 7ff3c9cbc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff3c40c5760 0x7ff3c4070f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:50.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.820+0000 7ff3c8cba640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3c40c5350 0x7ff3c4070a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:44282/0 (socket says 172.21.15.76:44282) 2024-04-25T19:15:50.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.820+0000 7ff3c94bb640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff3c415dd40 0x7ff3c41549c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.76:47622/0 (socket says 172.21.15.76:47622) 2024-04-25T19:15:50.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.820+0000 7ff3c8cba640 1 -- 172.21.15.76:0/1258659763 learned_addr learned my addr 172.21.15.76:0/1258659763 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:50.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.820+0000 7ff3bbfff640 1 -- 172.21.15.76:0/1258659763 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3825518973 0 0) 0x7ff3c40ca940 con 0x7ff3c415dd40 2024-04-25T19:15:50.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.820+0000 7ff3bbfff640 1 -- 172.21.15.76:0/1258659763 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3b0003520 con 0x7ff3c415dd40 2024-04-25T19:15:50.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.820+0000 7ff3c9cbc640 1 -- 172.21.15.76:0/1258659763 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3c40c5350 msgr2=0x7ff3c4070a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:50.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.820+0000 7ff3c9cbc640 1 --2- 172.21.15.76:0/1258659763 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3c40c5350 0x7ff3c4070a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:50.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.820+0000 7ff3c9cbc640 1 -- 172.21.15.76:0/1258659763 >> v1:172.21.15.105:6789/0 conn(0x7ff3c415dd40 legacy=0x7ff3c41549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:50.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.820+0000 7ff3c9cbc640 1 -- 172.21.15.76:0/1258659763 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c40caec0 con 0x7ff3c40c5760 2024-04-25T19:15:50.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.820+0000 7ff3c8cba640 1 --2- 172.21.15.76:0/1258659763 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3c40c5350 0x7ff3c4070a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:50.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.821+0000 7ff3c9cbc640 1 --2- 172.21.15.76:0/1258659763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3c40c5760 0x7ff3c4070f60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009d90 tx=0x7ff3bc02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1c745d7fc5b0fb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:50.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.821+0000 7ff3bbfff640 1 -- 172.21.15.76:0/1258659763 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc007740 con 0x7ff3c40c5760 2024-04-25T19:15:50.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.821+0000 7ff3bbfff640 1 -- 172.21.15.76:0/1258659763 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3bc0078e0 con 0x7ff3c40c5760 2024-04-25T19:15:50.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.821+0000 7ff3bbfff640 1 -- 172.21.15.76:0/1258659763 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc007cb0 con 0x7ff3c40c5760 2024-04-25T19:15:50.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.822+0000 7ff3cb71f640 1 -- 172.21.15.76:0/1258659763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3c40c5760 msgr2=0x7ff3c4070f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:50.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.822+0000 7ff3cb71f640 1 --2- 172.21.15.76:0/1258659763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3c40c5760 0x7ff3c4070f60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009d90 tx=0x7ff3bc02a6f0 comp rx=0 tx=0).stop 2024-04-25T19:15:50.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.822+0000 7ff3cb71f640 1 -- 172.21.15.76:0/1258659763 shutdown_connections 2024-04-25T19:15:50.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.822+0000 7ff3cb71f640 1 --2- 172.21.15.76:0/1258659763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3c40c5760 0x7ff3c4070f60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:50.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.822+0000 7ff3cb71f640 1 --2- 172.21.15.76:0/1258659763 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3c40c5350 0x7ff3c4070a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:50.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.822+0000 7ff3cb71f640 1 -- 172.21.15.76:0/1258659763 >> 172.21.15.76:0/1258659763 conn(0x7ff3c40cfdc0 msgr2=0x7ff3c40d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:50.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.822+0000 7ff3cb71f640 1 -- 172.21.15.76:0/1258659763 shutdown_connections 2024-04-25T19:15:50.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.822+0000 7ff3cb71f640 1 -- 172.21.15.76:0/1258659763 wait complete. 2024-04-25T19:15:50.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.824+0000 7ff3cb71f640 1 Processor -- start 2024-04-25T19:15:50.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.824+0000 7ff3cb71f640 1 -- start start 2024-04-25T19:15:50.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.825+0000 7ff3cb71f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3c40c5350 0x7ff3c41f48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:50.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.825+0000 7ff3cb71f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3c40c5760 0x7ff3c40c6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:50.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.825+0000 7ff3cb71f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3c415dd40 0x7ff3c41f1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:50.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.825+0000 7ff3cb71f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c4071e20 con 0x7ff3c40c5760 2024-04-25T19:15:50.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.825+0000 7ff3cb71f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c40cb480 con 0x7ff3c40c5350 2024-04-25T19:15:50.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.825+0000 7ff3cb71f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff3c4071fa0 con 0x7ff3c415dd40 2024-04-25T19:15:50.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.825+0000 7ff3c8cba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3c40c5760 0x7ff3c40c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:50.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.825+0000 7ff3c8cba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3c40c5760 0x7ff3c40c6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37770/0 (socket says 172.21.15.76:37770) 2024-04-25T19:15:50.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.825+0000 7ff3c8cba640 1 -- 172.21.15.76:0/476542158 learned_addr learned my addr 172.21.15.76:0/476542158 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:50.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.825+0000 7ff3c94bb640 1 --2- 172.21.15.76:0/476542158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3c40c5350 0x7ff3c41f48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:50.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.825+0000 7ff3c9cbc640 1 --2- 172.21.15.76:0/476542158 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3c415dd40 0x7ff3c41f1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:50.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.825+0000 7ff3c8cba640 1 -- 172.21.15.76:0/476542158 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3c415dd40 msgr2=0x7ff3c41f1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:50.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.825+0000 7ff3c8cba640 1 --2- 172.21.15.76:0/476542158 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3c415dd40 0x7ff3c41f1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:50.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.825+0000 7ff3c8cba640 1 -- 172.21.15.76:0/476542158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3c40c5350 msgr2=0x7ff3c41f48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:50.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.825+0000 7ff3c8cba640 1 --2- 172.21.15.76:0/476542158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3c40c5350 0x7ff3c41f48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:50.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.825+0000 7ff3c8cba640 1 -- 172.21.15.76:0/476542158 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c41f4e30 con 0x7ff3c40c5760 2024-04-25T19:15:50.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.826+0000 7ff3c8cba640 1 --2- 172.21.15.76:0/476542158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3c40c5760 0x7ff3c40c6260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff3c000e900 tx=0x7ff3c000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:50.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.826+0000 7ff3b9ffb640 1 -- 172.21.15.76:0/476542158 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0017070 con 0x7ff3c40c5760 2024-04-25T19:15:50.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.826+0000 7ff3b9ffb640 1 -- 172.21.15.76:0/476542158 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3c000ca00 con 0x7ff3c40c5760 2024-04-25T19:15:50.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.826+0000 7ff3b9ffb640 1 -- 172.21.15.76:0/476542158 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c000cdf0 con 0x7ff3c40c5760 2024-04-25T19:15:50.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.826+0000 7ff3cb71f640 1 -- 172.21.15.76:0/476542158 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c4158070 con 0x7ff3c40c5760 2024-04-25T19:15:50.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.826+0000 7ff3cb71f640 1 -- 172.21.15.76:0/476542158 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c41f40f0 con 0x7ff3c40c5760 2024-04-25T19:15:50.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.828+0000 7ff3b9ffb640 1 -- 172.21.15.76:0/476542158 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff3c001a070 con 0x7ff3c40c5760 2024-04-25T19:15:50.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.828+0000 7ff3b9ffb640 1 --2- 172.21.15.76:0/476542158 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff3a80744b0 0x7ff3a80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:50.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.828+0000 7ff3b9ffb640 1 -- 172.21.15.76:0/476542158 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff3c0094f60 con 0x7ff3c40c5760 2024-04-25T19:15:50.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.829+0000 7ff3c94bb640 1 --2- 172.21.15.76:0/476542158 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff3a80744b0 0x7ff3a80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:50.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.829+0000 7ff3c94bb640 1 --2- 172.21.15.76:0/476542158 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff3a80744b0 0x7ff3a80769c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4000f60 tx=0x7ff3b4000ed0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:50.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.829+0000 7ff3d25d3740 1 Processor -- start 2024-04-25T19:15:50.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.829+0000 7ff3d25d3740 1 -- start start 2024-04-25T19:15:50.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.830+0000 7ff3d25d3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56500773a270 0x565007ae09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:50.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.830+0000 7ff3d25d3740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5650077113b0 0x565007adac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:50.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.830+0000 7ff3d25d3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565007719590 0x565007adb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:50.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.830+0000 7ff3d25d3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x565007a40740 con 0x565007719590 2024-04-25T19:15:50.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.830+0000 7ff3d25d3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5650079dc3a0 con 0x56500773a270 2024-04-25T19:15:50.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.830+0000 7ff3d25d3740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x565007a3c700 con 0x5650077113b0 2024-04-25T19:15:50.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.831+0000 7ff3c9cbc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565007719590 0x565007adb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:50.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.831+0000 7ff3c9cbc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565007719590 0x565007adb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37782/0 (socket says 172.21.15.76:37782) 2024-04-25T19:15:50.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.831+0000 7ff3c9cbc640 1 -- 172.21.15.76:0/2774103926 learned_addr learned my addr 172.21.15.76:0/2774103926 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:50.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.831+0000 7ff3c94bb640 1 --2- 172.21.15.76:0/2774103926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56500773a270 0x565007ae09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:50.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.831+0000 7ff3c8cba640 1 --2- 172.21.15.76:0/2774103926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5650077113b0 0x565007adac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:50.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.831+0000 7ff3c9cbc640 1 -- 172.21.15.76:0/2774103926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5650077113b0 msgr2=0x565007adac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:50.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.831+0000 7ff3c9cbc640 1 --2- 172.21.15.76:0/2774103926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5650077113b0 0x565007adac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:50.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.831+0000 7ff3c9cbc640 1 -- 172.21.15.76:0/2774103926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56500773a270 msgr2=0x565007ae09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:50.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.831+0000 7ff3c9cbc640 1 --2- 172.21.15.76:0/2774103926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56500773a270 0x565007ae09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:50.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.831+0000 7ff3c9cbc640 1 -- 172.21.15.76:0/2774103926 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650079f8750 con 0x565007719590 2024-04-25T19:15:50.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.831+0000 7ff3c9cbc640 1 --2- 172.21.15.76:0/2774103926 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565007719590 0x565007adb1d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009eb0 tx=0x7ff3bc02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f63e6a2b1e675f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:50.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.831+0000 7ff3a6ffd640 1 -- 172.21.15.76:0/2774103926 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc007880 con 0x565007719590 2024-04-25T19:15:50.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.831+0000 7ff3a6ffd640 1 -- 172.21.15.76:0/2774103926 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3bc007a20 con 0x565007719590 2024-04-25T19:15:50.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.831+0000 7ff3a6ffd640 1 -- 172.21.15.76:0/2774103926 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc007e10 con 0x565007719590 2024-04-25T19:15:50.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.832+0000 7ff3d25d3740 1 -- 172.21.15.76:0/2774103926 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565007719590 msgr2=0x565007adb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:50.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.832+0000 7ff3d25d3740 1 --2- 172.21.15.76:0/2774103926 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565007719590 0x565007adb1d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009eb0 tx=0x7ff3bc02e000 comp rx=0 tx=0).stop 2024-04-25T19:15:50.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.832+0000 7ff3d25d3740 1 -- 172.21.15.76:0/2774103926 shutdown_connections 2024-04-25T19:15:50.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.832+0000 7ff3d25d3740 1 --2- 172.21.15.76:0/2774103926 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565007719590 0x565007adb1d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:50.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.832+0000 7ff3d25d3740 1 --2- 172.21.15.76:0/2774103926 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5650077113b0 0x565007adac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:50.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.832+0000 7ff3d25d3740 1 --2- 172.21.15.76:0/2774103926 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56500773a270 0x565007ae09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:50.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.832+0000 7ff3d25d3740 1 -- 172.21.15.76:0/2774103926 >> 172.21.15.76:0/2774103926 conn(0x56500772e3c0 msgr2=0x565007adf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:50.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.832+0000 7ff3d25d3740 1 -- 172.21.15.76:0/2774103926 shutdown_connections 2024-04-25T19:15:50.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.832+0000 7ff3d25d3740 1 -- 172.21.15.76:0/2774103926 wait complete. 2024-04-25T19:15:50.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.833+0000 7ff3d25d3740 1 -- start start 2024-04-25T19:15:50.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.833+0000 7ff3d25d3740 1 Processor -- start 2024-04-25T19:15:50.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.834+0000 7ff3d25d3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650077113b0 0x565007b07c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:50.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.834+0000 7ff3d25d3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565007719590 0x565007b202e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:50.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.834+0000 7ff3d25d3740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x56500773a270 0x565007b20820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:50.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.834+0000 7ff3d25d3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x565007b1f110 con 0x5650077113b0 2024-04-25T19:15:50.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.834+0000 7ff3d25d3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5650079c7fe0 con 0x565007719590 2024-04-25T19:15:50.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.834+0000 7ff3d25d3740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x565007adeff0 con 0x56500773a270 2024-04-25T19:15:50.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.834+0000 7ff3d25d3740 10 client.0 _ll_register_callbacks cb 0x5650079de150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:15:50.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.834+0000 7ff3c94bb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650077113b0 0x565007b07c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.834+0000 7ff3c94bb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650077113b0 0x565007b07c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37796/0 (socket says 172.21.15.76:37796) 2024-04-25T19:15:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.834+0000 7ff3c94bb640 1 -- 172.21.15.76:0/447657812 learned_addr learned my addr 172.21.15.76:0/447657812 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.835+0000 7ff3c8cba640 1 --2- 172.21.15.76:0/447657812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565007719590 0x565007b202e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.835+0000 7ff3c9cbc640 1 --2- 172.21.15.76:0/447657812 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x56500773a270 0x565007b20820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.835+0000 7ff3c94bb640 1 -- 172.21.15.76:0/447657812 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x56500773a270 msgr2=0x565007b20820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.835+0000 7ff3c94bb640 1 --2- 172.21.15.76:0/447657812 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x56500773a270 0x565007b20820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.835+0000 7ff3c94bb640 1 -- 172.21.15.76:0/447657812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565007719590 msgr2=0x565007b202e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.835+0000 7ff3c94bb640 1 --2- 172.21.15.76:0/447657812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565007719590 0x565007b202e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.835+0000 7ff3c94bb640 1 -- 172.21.15.76:0/447657812 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650077522d0 con 0x5650077113b0 2024-04-25T19:15:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.835+0000 7ff3c94bb640 1 --2- 172.21.15.76:0/447657812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650077113b0 0x565007b07c90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4015640 tx=0x7ff3b4001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3102f724af8b796 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.835+0000 7ff39ffff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-04-25T19:15:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.835+0000 7ff39ffff640 1 -- 172.21.15.76:0/447657812 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4027060 con 0x5650077113b0 2024-04-25T19:15:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.835+0000 7ff39ffff640 1 -- 172.21.15.76:0/447657812 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff3b4010040 con 0x5650077113b0 2024-04-25T19:15:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.835+0000 7ff3d25d3740 1 -- 172.21.15.76:0/447657812 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3bc007a20 con 0x5650077113b0 2024-04-25T19:15:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.835+0000 7ff39ffff640 1 -- 172.21.15.76:0/447657812 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7ff3b40133f0 con 0x5650077113b0 2024-04-25T19:15:50.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.835+0000 7ff3d25d3740 10 client.4710 fetch_fsmap learned FSMap version 16 2024-04-25T19:15:50.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.835+0000 7ff3d25d3740 1 -- 172.21.15.76:0/447657812 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x56500781c6b0 con 0x5650077113b0 2024-04-25T19:15:50.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.835+0000 7ff39ffff640 1 -- 172.21.15.76:0/447657812 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3bc007a20 con 0x5650077113b0 2024-04-25T19:15:50.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.836+0000 7ff39ffff640 1 -- 172.21.15.76:0/447657812 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 16) ==== 4784+0+0 (secure 0 0 0) 0x7ff3b40226f0 con 0x5650077113b0 2024-04-25T19:15:50.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.836+0000 7ff3d25d3740 10 client.4710 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T19:15:50.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.836+0000 7ff3d25d3740 10 client.4710 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T19:15:50.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.836+0000 7ff3d25d3740 20 client.4710 populate_metadata read hostname 'smithi076' 2024-04-25T19:15:50.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.836+0000 7ff3d25d3740 1 --2- 172.21.15.76:0/447657812 >> [v2:172.21.15.105:6834/1809877606,v1:172.21.15.105:6836/1809877606] conn(0x5650076f8e40 0x565007b3f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:50.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.836+0000 7ff3d25d3740 4 client.4710 mds_command: new command op to 4235 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:15:50.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.836+0000 7ff3d25d3740 1 -- 172.21.15.76:0/447657812 --> [v2:172.21.15.105:6834/1809877606,v1:172.21.15.105:6836/1809877606] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565007a4e350 con 0x5650076f8e40 2024-04-25T19:15:50.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.836+0000 7ff3c8cba640 1 --2- 172.21.15.76:0/447657812 >> [v2:172.21.15.105:6834/1809877606,v1:172.21.15.105:6836/1809877606] conn(0x5650076f8e40 0x565007b3f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:50.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.836+0000 7ff39ffff640 10 client.4710 ms_handle_connect on v2:172.21.15.105:6834/1809877606 2024-04-25T19:15:50.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:50.836+0000 7ff3c8cba640 1 --2- 172.21.15.76:0/447657812 >> [v2:172.21.15.105:6834/1809877606,v1:172.21.15.105:6836/1809877606] conn(0x5650076f8e40 0x565007b3f330 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5c377e830dca8a0 server_cookie=f03ba795f803fbea in_seq=0 out_seq=0 2024-04-25T19:15:53.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.506+0000 7ff39ffff640 1 -- 172.21.15.76:0/447657812 <== mds.4 v2:172.21.15.105:6834/1809877606 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x565007a4e350 con 0x5650076f8e40 2024-04-25T19:15:53.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.506+0000 7ff39ffff640 10 client.4710 handle_command_reply: tid=0 2024-04-25T19:15:53.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.506+0000 7ff3d25d3740 1 client.4710 shutdown 2024-04-25T19:15:53.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.506+0000 7ff3d25d3740 1 -- 172.21.15.76:0/447657812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650077113b0 msgr2=0x565007b07c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:53.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.506+0000 7ff3d25d3740 1 --2- 172.21.15.76:0/447657812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650077113b0 0x565007b07c90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4015640 tx=0x7ff3b4001150 comp rx=0 tx=0).stop 2024-04-25T19:15:53.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.506+0000 7ff3d25d3740 1 -- 172.21.15.76:0/447657812 shutdown_connections 2024-04-25T19:15:53.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.506+0000 7ff3d25d3740 1 --2- 172.21.15.76:0/447657812 >> [v2:172.21.15.105:6834/1809877606,v1:172.21.15.105:6836/1809877606] conn(0x5650076f8e40 0x565007b3f330 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.506+0000 7ff3d25d3740 1 --2- 172.21.15.76:0/447657812 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x56500773a270 0x565007b20820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.506+0000 7ff3d25d3740 1 --2- 172.21.15.76:0/447657812 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565007719590 0x565007b202e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.506+0000 7ff39ffff640 0 client.4710 ms_handle_reset on v2:172.21.15.105:6834/1809877606 2024-04-25T19:15:53.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.506+0000 7ff3d25d3740 1 --2- 172.21.15.76:0/447657812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650077113b0 0x565007b07c90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.506+0000 7ff3d25d3740 1 -- 172.21.15.76:0/447657812 >> 172.21.15.76:0/447657812 conn(0x56500772e3c0 msgr2=0x565007b097b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:53.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.507+0000 7ff3d25d3740 1 -- 172.21.15.76:0/447657812 shutdown_connections 2024-04-25T19:15:53.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.507+0000 7ff3d25d3740 1 -- 172.21.15.76:0/447657812 wait complete. 2024-04-25T19:15:53.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.507+0000 7ff3d25d3740 20 client.4710 trim_cache size 0 max 16384 2024-04-25T19:15:53.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.526+0000 7ff3d25d3740 1 Processor -- start 2024-04-25T19:15:53.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.526+0000 7ff3d25d3740 1 -- start start 2024-04-25T19:15:53.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.527+0000 7ff3d25d3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650077113b0 0x565007adae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.527+0000 7ff3d25d3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565007719590 0x565007b92b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.527+0000 7ff3d25d3740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x56500773a270 0x565007b93060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.527+0000 7ff3d25d3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c415d5a0 con 0x565007719590 2024-04-25T19:15:53.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.527+0000 7ff3d25d3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x565007b30d50 con 0x5650077113b0 2024-04-25T19:15:53.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.527+0000 7ff3d25d3740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x565007b17e90 con 0x56500773a270 2024-04-25T19:15:53.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.527+0000 7ff3c8cba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565007719590 0x565007b92b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:53.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.527+0000 7ff3c8cba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565007719590 0x565007b92b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37810/0 (socket says 172.21.15.76:37810) 2024-04-25T19:15:53.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.527+0000 7ff3c8cba640 1 -- 172.21.15.76:0/744049166 learned_addr learned my addr 172.21.15.76:0/744049166 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:53.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.528+0000 7ff3c8cba640 1 -- 172.21.15.76:0/744049166 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x56500773a270 msgr2=0x565007b93060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:53.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.528+0000 7ff3c8cba640 1 --2- 172.21.15.76:0/744049166 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x56500773a270 0x565007b93060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.528+0000 7ff3c8cba640 1 -- 172.21.15.76:0/744049166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650077113b0 msgr2=0x565007adae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:53.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.528+0000 7ff3c8cba640 1 --2- 172.21.15.76:0/744049166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650077113b0 0x565007adae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.528+0000 7ff3c8cba640 1 -- 172.21.15.76:0/744049166 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565007adbf40 con 0x565007719590 2024-04-25T19:15:53.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.528+0000 7ff3c8cba640 1 --2- 172.21.15.76:0/744049166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565007719590 0x565007b92b20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff3c005eec0 tx=0x7ff3c005b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790c49e3475e8aac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:53.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.528+0000 7ff3a67fc640 1 -- 172.21.15.76:0/744049166 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0091070 con 0x565007719590 2024-04-25T19:15:53.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.528+0000 7ff3a67fc640 1 -- 172.21.15.76:0/744049166 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3c00634c0 con 0x565007719590 2024-04-25T19:15:53.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.528+0000 7ff3a67fc640 1 -- 172.21.15.76:0/744049166 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0021070 con 0x565007719590 2024-04-25T19:15:53.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.529+0000 7ff3d25d3740 1 -- 172.21.15.76:0/744049166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565007719590 msgr2=0x565007b92b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:53.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.529+0000 7ff3d25d3740 1 --2- 172.21.15.76:0/744049166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565007719590 0x565007b92b20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff3c005eec0 tx=0x7ff3c005b8a0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.529+0000 7ff3d25d3740 1 -- 172.21.15.76:0/744049166 shutdown_connections 2024-04-25T19:15:53.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.529+0000 7ff3d25d3740 1 --2- 172.21.15.76:0/744049166 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x56500773a270 0x565007b93060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.529+0000 7ff3d25d3740 1 --2- 172.21.15.76:0/744049166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565007719590 0x565007b92b20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.529+0000 7ff3d25d3740 1 --2- 172.21.15.76:0/744049166 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5650077113b0 0x565007adae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.529+0000 7ff3d25d3740 1 -- 172.21.15.76:0/744049166 >> 172.21.15.76:0/744049166 conn(0x56500772e3c0 msgr2=0x565007ae1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:53.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.529+0000 7ff3d25d3740 1 -- 172.21.15.76:0/744049166 shutdown_connections 2024-04-25T19:15:53.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.529+0000 7ff3d25d3740 1 -- 172.21.15.76:0/744049166 wait complete. 2024-04-25T19:15:53.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.530+0000 7ff3d25d3740 1 -- start start 2024-04-25T19:15:53.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.530+0000 7ff3d25d3740 1 Processor -- start 2024-04-25T19:15:53.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.531+0000 7ff3d25d3740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5650077113b0 0x565007b96f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.531+0000 7ff3d25d3740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565007719590 0x565007b05100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.531+0000 7ff3d25d3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56500773a270 0x565007b90f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.531+0000 7ff3d25d3740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x565007b07280 con 0x565007719590 2024-04-25T19:15:53.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.531+0000 7ff3d25d3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x565007b19e40 con 0x56500773a270 2024-04-25T19:15:53.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.531+0000 7ff3d25d3740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x565007b33f00 con 0x5650077113b0 2024-04-25T19:15:53.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.531+0000 7ff3d25d3740 10 client.0 _ll_register_callbacks cb 0x7ff3b40070a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:15:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.531+0000 7ff3c8cba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565007719590 0x565007b05100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.531+0000 7ff3c8cba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565007719590 0x565007b05100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37826/0 (socket says 172.21.15.76:37826) 2024-04-25T19:15:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.531+0000 7ff3c8cba640 1 -- 172.21.15.76:0/629567303 learned_addr learned my addr 172.21.15.76:0/629567303 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.531+0000 7ff3c94bb640 1 --2- 172.21.15.76:0/629567303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5650077113b0 0x565007b96f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.531+0000 7ff3c9cbc640 1 --2- 172.21.15.76:0/629567303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56500773a270 0x565007b90f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.531+0000 7ff3c8cba640 1 -- 172.21.15.76:0/629567303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5650077113b0 msgr2=0x565007b96f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.531+0000 7ff3c8cba640 1 --2- 172.21.15.76:0/629567303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5650077113b0 0x565007b96f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.531+0000 7ff3c8cba640 1 -- 172.21.15.76:0/629567303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56500773a270 msgr2=0x565007b90f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.531+0000 7ff3c8cba640 1 --2- 172.21.15.76:0/629567303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56500773a270 0x565007b90f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.531+0000 7ff3c8cba640 1 -- 172.21.15.76:0/629567303 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565007b933a0 con 0x565007719590 2024-04-25T19:15:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.532+0000 7ff3c8cba640 1 --2- 172.21.15.76:0/629567303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565007719590 0x565007b05100 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff3c000f010 tx=0x7ff3c0062040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b54346efd48c9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.532+0000 7ff39ffff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-04-25T19:15:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.532+0000 7ff39ffff640 1 -- 172.21.15.76:0/629567303 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0091030 con 0x565007719590 2024-04-25T19:15:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.532+0000 7ff39ffff640 1 -- 172.21.15.76:0/629567303 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff3c005b450 con 0x565007719590 2024-04-25T19:15:53.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.532+0000 7ff3d25d3740 1 -- 172.21.15.76:0/629567303 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3c00634c0 con 0x565007719590 2024-04-25T19:15:53.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.532+0000 7ff39ffff640 1 -- 172.21.15.76:0/629567303 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7ff3c005ecd0 con 0x565007719590 2024-04-25T19:15:53.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.532+0000 7ff3d25d3740 10 client.4716 fetch_fsmap learned FSMap version 16 2024-04-25T19:15:53.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.532+0000 7ff3d25d3740 1 -- 172.21.15.76:0/629567303 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x565007b06490 con 0x565007719590 2024-04-25T19:15:53.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.532+0000 7ff39ffff640 1 -- 172.21.15.76:0/629567303 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3c00634c0 con 0x565007719590 2024-04-25T19:15:53.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.532+0000 7ff39ffff640 1 -- 172.21.15.76:0/629567303 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 16) ==== 4784+0+0 (secure 0 0 0) 0x7ff3c0024060 con 0x565007719590 2024-04-25T19:15:53.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.532+0000 7ff3d25d3740 10 client.4716 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T19:15:53.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.532+0000 7ff3d25d3740 10 client.4716 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T19:15:53.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.532+0000 7ff3d25d3740 20 client.4716 populate_metadata read hostname 'smithi076' 2024-04-25T19:15:53.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.532+0000 7ff3d25d3740 1 --2- 172.21.15.76:0/629567303 >> [v2:172.21.15.105:6834/1809877606,v1:172.21.15.105:6836/1809877606] conn(0x565007b97a60 0x565007b91480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.532+0000 7ff3d25d3740 4 client.4716 mds_command: new command op to 4235 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T19:15:53.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.532+0000 7ff3d25d3740 1 -- 172.21.15.76:0/629567303 --> [v2:172.21.15.105:6834/1809877606,v1:172.21.15.105:6836/1809877606] -- command(tid 0: {"prefix": "status"}) -- 0x565007a481c0 con 0x565007b97a60 2024-04-25T19:15:53.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.533+0000 7ff3c94bb640 1 --2- 172.21.15.76:0/629567303 >> [v2:172.21.15.105:6834/1809877606,v1:172.21.15.105:6836/1809877606] conn(0x565007b97a60 0x565007b91480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:53.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.533+0000 7ff39ffff640 10 client.4716 ms_handle_connect on v2:172.21.15.105:6834/1809877606 2024-04-25T19:15:53.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.533+0000 7ff3c94bb640 1 --2- 172.21.15.76:0/629567303 >> [v2:172.21.15.105:6834/1809877606,v1:172.21.15.105:6836/1809877606] conn(0x565007b97a60 0x565007b91480 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a6840808b24963f3 server_cookie=61c6aee6eef02eaa in_seq=0 out_seq=0 2024-04-25T19:15:53.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.533+0000 7ff39ffff640 1 -- 172.21.15.76:0/629567303 <== mds.4 v2:172.21.15.105:6834/1809877606 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x565007a481c0 con 0x565007b97a60 2024-04-25T19:15:53.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.533+0000 7ff39ffff640 10 client.4716 handle_command_reply: tid=0 2024-04-25T19:15:53.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.533+0000 7ff3d25d3740 1 client.4716 shutdown 2024-04-25T19:15:53.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.533+0000 7ff3d25d3740 1 -- 172.21.15.76:0/629567303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565007719590 msgr2=0x565007b05100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:53.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.533+0000 7ff3d25d3740 1 --2- 172.21.15.76:0/629567303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565007719590 0x565007b05100 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff3c000f010 tx=0x7ff3c0062040 comp rx=0 tx=0).stop 2024-04-25T19:15:53.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.533+0000 7ff3d25d3740 1 -- 172.21.15.76:0/629567303 shutdown_connections 2024-04-25T19:15:53.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.533+0000 7ff3d25d3740 1 --2- 172.21.15.76:0/629567303 >> [v2:172.21.15.105:6834/1809877606,v1:172.21.15.105:6836/1809877606] conn(0x565007b97a60 0x565007b91480 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.533+0000 7ff3d25d3740 1 --2- 172.21.15.76:0/629567303 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56500773a270 0x565007b90f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.533+0000 7ff39ffff640 0 client.4716 ms_handle_reset on v2:172.21.15.105:6834/1809877606 2024-04-25T19:15:53.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.533+0000 7ff3d25d3740 1 --2- 172.21.15.76:0/629567303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565007719590 0x565007b05100 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.533+0000 7ff3d25d3740 1 --2- 172.21.15.76:0/629567303 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x5650077113b0 0x565007b96f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.533+0000 7ff3d25d3740 1 -- 172.21.15.76:0/629567303 >> 172.21.15.76:0/629567303 conn(0x56500772e3c0 msgr2=0x565007adae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:53.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.533+0000 7ff3d25d3740 1 -- 172.21.15.76:0/629567303 shutdown_connections 2024-04-25T19:15:53.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.533+0000 7ff3d25d3740 1 -- 172.21.15.76:0/629567303 wait complete. 2024-04-25T19:15:53.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.534+0000 7ff3d25d3740 20 client.4716 trim_cache size 0 max 16384 2024-04-25T19:15:53.541 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:15:53.541 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:15:53.541 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 4, 2024-04-25T19:15:53.541 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4235, 2024-04-25T19:15:53.541 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-04-25T19:15:53.541 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-04-25T19:15:53.541 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-04-25T19:15:53.541 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 11.045805758, 2024-04-25T19:15:53.541 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 16, 2024-04-25T19:15:53.541 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 21, 2024-04-25T19:15:53.541 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 21, 2024-04-25T19:15:53.541 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 41.156828298999997 2024-04-25T19:15:53.541 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:15:53.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.534+0000 7ff3cb71f640 1 -- 172.21.15.76:0/476542158 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff3a80744b0 msgr2=0x7ff3a80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:53.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.534+0000 7ff3cb71f640 1 --2- 172.21.15.76:0/476542158 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff3a80744b0 0x7ff3a80769c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4000f60 tx=0x7ff3b4000ed0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.534+0000 7ff3cb71f640 1 -- 172.21.15.76:0/476542158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3c40c5760 msgr2=0x7ff3c40c6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:53.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.534+0000 7ff3cb71f640 1 --2- 172.21.15.76:0/476542158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3c40c5760 0x7ff3c40c6260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff3c000e900 tx=0x7ff3c000edc0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.534+0000 7ff3cb71f640 1 -- 172.21.15.76:0/476542158 shutdown_connections 2024-04-25T19:15:53.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.534+0000 7ff3cb71f640 1 --2- 172.21.15.76:0/476542158 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff3a80744b0 0x7ff3a80769c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.534+0000 7ff3cb71f640 1 --2- 172.21.15.76:0/476542158 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff3c415dd40 0x7ff3c41f1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.534+0000 7ff3cb71f640 1 --2- 172.21.15.76:0/476542158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3c40c5760 0x7ff3c40c6260 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.534+0000 7ff3cb71f640 1 --2- 172.21.15.76:0/476542158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3c40c5350 0x7ff3c41f48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.534+0000 7ff3cb71f640 1 -- 172.21.15.76:0/476542158 >> 172.21.15.76:0/476542158 conn(0x7ff3c40cfdc0 msgr2=0x7ff3c4147a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:53.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.534+0000 7ff3cb71f640 1 -- 172.21.15.76:0/476542158 shutdown_connections 2024-04-25T19:15:53.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.534+0000 7ff3cb71f640 1 -- 172.21.15.76:0/476542158 wait complete. 2024-04-25T19:15:53.548 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-25T19:15:53.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.653+0000 7f24324d8640 1 Processor -- start 2024-04-25T19:15:53.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.653+0000 7f24324d8640 1 -- start start 2024-04-25T19:15:53.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.654+0000 7f24324d8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f242c159840 0x7f242c153b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.654+0000 7f24324d8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f242c15d880 0x7f242c0c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.654+0000 7f24324d8640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f242c0715d0 con 0x7f242c0c5b20 2024-04-25T19:15:53.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.654+0000 7f24324d8640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f242c15ce30 con 0x7f242c159840 2024-04-25T19:15:53.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.654+0000 7f24324d8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f242c0718b0 con 0x7f242c15d880 2024-04-25T19:15:53.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.654+0000 7f242bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f242c159840 0x7f242c153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:53.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.654+0000 7f242bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f242c159840 0x7f242c153b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37828/0 (socket says 172.21.15.76:37828) 2024-04-25T19:15:53.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.654+0000 7f242bfff640 1 -- 172.21.15.76:0/3363318722 learned_addr learned my addr 172.21.15.76:0/3363318722 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:53.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.654+0000 7f2430a75640 1 --2- 172.21.15.76:0/3363318722 >> v2:172.21.15.105:3300/0 conn(0x7f242c15d880 0x7f242c0c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:53.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.654+0000 7f242bfff640 1 -- 172.21.15.76:0/3363318722 >> v2:172.21.15.105:3300/0 conn(0x7f242c15d880 msgr2=0x7f242c0c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:53.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.654+0000 7f242bfff640 1 --2- 172.21.15.76:0/3363318722 >> v2:172.21.15.105:3300/0 conn(0x7f242c15d880 0x7f242c0c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.654+0000 7f242bfff640 1 -- 172.21.15.76:0/3363318722 >> v1:172.21.15.76:6789/0 conn(0x7f242c0c5b20 legacy=0x7f242c0c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:53.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.654+0000 7f242bfff640 1 -- 172.21.15.76:0/3363318722 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242c071b90 con 0x7f242c159840 2024-04-25T19:15:53.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.655+0000 7f242bfff640 1 --2- 172.21.15.76:0/3363318722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f242c159840 0x7f242c153b70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f242c0c8810 tx=0x7f241c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a7eabb81a7e80ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:53.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.655+0000 7f242affd640 1 -- 172.21.15.76:0/3363318722 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c037070 con 0x7f242c159840 2024-04-25T19:15:53.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.655+0000 7f242affd640 1 -- 172.21.15.76:0/3363318722 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f241c032440 con 0x7f242c159840 2024-04-25T19:15:53.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.655+0000 7f242affd640 1 -- 172.21.15.76:0/3363318722 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c032810 con 0x7f242c159840 2024-04-25T19:15:53.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.656+0000 7f24324d8640 1 -- 172.21.15.76:0/3363318722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f242c159840 msgr2=0x7f242c153b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:53.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.656+0000 7f24324d8640 1 --2- 172.21.15.76:0/3363318722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f242c159840 0x7f242c153b70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f242c0c8810 tx=0x7f241c02a270 comp rx=0 tx=0).stop 2024-04-25T19:15:53.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.656+0000 7f24324d8640 1 -- 172.21.15.76:0/3363318722 shutdown_connections 2024-04-25T19:15:53.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.656+0000 7f24324d8640 1 --2- 172.21.15.76:0/3363318722 >> v2:172.21.15.105:3300/0 conn(0x7f242c15d880 0x7f242c0c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.656+0000 7f24324d8640 1 --2- 172.21.15.76:0/3363318722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f242c159840 0x7f242c153b70 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.656+0000 7f24324d8640 1 -- 172.21.15.76:0/3363318722 >> 172.21.15.76:0/3363318722 conn(0x7f242c0cecc0 msgr2=0x7f242c0cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:53.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.656+0000 7f24324d8640 1 -- 172.21.15.76:0/3363318722 shutdown_connections 2024-04-25T19:15:53.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.656+0000 7f24324d8640 1 -- 172.21.15.76:0/3363318722 wait complete. 2024-04-25T19:15:53.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.658+0000 7f24324d8640 1 Processor -- start 2024-04-25T19:15:53.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.658+0000 7f24324d8640 1 -- start start 2024-04-25T19:15:53.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.659+0000 7f24324d8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f242c0c5b20 0x7f242c154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.659+0000 7f24324d8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f242c159840 0x7f242c149840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.659+0000 7f24324d8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f242c15d880 0x7f242c152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.659+0000 7f24324d8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f242c15dc90 con 0x7f242c159840 2024-04-25T19:15:53.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.659+0000 7f24324d8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f242c071270 con 0x7f242c15d880 2024-04-25T19:15:53.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.659+0000 7f24324d8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f242c0c8e30 con 0x7f242c0c5b20 2024-04-25T19:15:53.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.659+0000 7f242b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f242c159840 0x7f242c149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:53.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.659+0000 7f242b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f242c159840 0x7f242c149840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37836/0 (socket says 172.21.15.76:37836) 2024-04-25T19:15:53.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.659+0000 7f242b7fe640 1 -- 172.21.15.76:0/3251576980 learned_addr learned my addr 172.21.15.76:0/3251576980 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:53.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.659+0000 7f242bfff640 1 --2- 172.21.15.76:0/3251576980 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f242c0c5b20 0x7f242c154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:53.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.659+0000 7f2430a75640 1 --2- 172.21.15.76:0/3251576980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f242c15d880 0x7f242c152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:53.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.659+0000 7f242b7fe640 1 -- 172.21.15.76:0/3251576980 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f242c0c5b20 msgr2=0x7f242c154cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:53.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.659+0000 7f242b7fe640 1 --2- 172.21.15.76:0/3251576980 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f242c0c5b20 0x7f242c154cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.660+0000 7f242b7fe640 1 -- 172.21.15.76:0/3251576980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f242c15d880 msgr2=0x7f242c152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:53.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.660+0000 7f242b7fe640 1 --2- 172.21.15.76:0/3251576980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f242c15d880 0x7f242c152d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.660+0000 7f242b7fe640 1 -- 172.21.15.76:0/3251576980 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242c155200 con 0x7f242c159840 2024-04-25T19:15:53.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.660+0000 7f242b7fe640 1 --2- 172.21.15.76:0/3251576980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f242c159840 0x7f242c149840 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f242000c3f0 tx=0x7f242000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:53.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.660+0000 7f2428ff9640 1 -- 172.21.15.76:0/3251576980 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24200052d0 con 0x7f242c159840 2024-04-25T19:15:53.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.660+0000 7f2428ff9640 1 -- 172.21.15.76:0/3251576980 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2420005490 con 0x7f242c159840 2024-04-25T19:15:53.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.660+0000 7f2428ff9640 1 -- 172.21.15.76:0/3251576980 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2420007260 con 0x7f242c159840 2024-04-25T19:15:53.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.660+0000 7f24324d8640 1 -- 172.21.15.76:0/3251576980 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f242c156e10 con 0x7f242c159840 2024-04-25T19:15:53.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.660+0000 7f24324d8640 1 -- 172.21.15.76:0/3251576980 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f242c1f15c0 con 0x7f242c159840 2024-04-25T19:15:53.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.662+0000 7f2428ff9640 1 -- 172.21.15.76:0/3251576980 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2420007400 con 0x7f242c159840 2024-04-25T19:15:53.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.662+0000 7f2428ff9640 1 --2- 172.21.15.76:0/3251576980 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f24180744b0 0x7f24180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.662+0000 7f2428ff9640 1 -- 172.21.15.76:0/3251576980 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f24200945a0 con 0x7f242c159840 2024-04-25T19:15:53.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.662+0000 7f242bfff640 1 --2- 172.21.15.76:0/3251576980 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f24180744b0 0x7f24180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:53.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.663+0000 7f2439385740 1 Processor -- start 2024-04-25T19:15:53.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.663+0000 7f242bfff640 1 --2- 172.21.15.76:0/3251576980 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f24180744b0 0x7f24180769c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f241c034000 tx=0x7f241c02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:53.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.663+0000 7f2439385740 1 -- start start 2024-04-25T19:15:53.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.664+0000 7f2439385740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55a892dd0270 0x55a8931b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.664+0000 7f2439385740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a892da73b0 0x55a8931a8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.664+0000 7f2439385740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892daf590 0x55a8931a91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.664+0000 7f2439385740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a8930d6740 con 0x55a892daf590 2024-04-25T19:15:53.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.664+0000 7f2439385740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a8930723a0 con 0x55a892da73b0 2024-04-25T19:15:53.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.664+0000 7f2439385740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x55a8930d2700 con 0x55a892dd0270 2024-04-25T19:15:53.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.664+0000 7f2430a75640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892daf590 0x55a8931a91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:53.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.664+0000 7f2430a75640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892daf590 0x55a8931a91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37840/0 (socket says 172.21.15.76:37840) 2024-04-25T19:15:53.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.664+0000 7f2430a75640 1 -- 172.21.15.76:0/1309292571 learned_addr learned my addr 172.21.15.76:0/1309292571 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:53.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.664+0000 7f242b7fe640 1 --2- 172.21.15.76:0/1309292571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a892da73b0 0x55a8931a8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:53.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.664+0000 7f242bfff640 1 --2- 172.21.15.76:0/1309292571 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55a892dd0270 0x55a8931b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:53.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.664+0000 7f2430a75640 1 -- 172.21.15.76:0/1309292571 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55a892dd0270 msgr2=0x55a8931b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:53.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.664+0000 7f2430a75640 1 --2- 172.21.15.76:0/1309292571 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55a892dd0270 0x55a8931b0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.664+0000 7f2430a75640 1 -- 172.21.15.76:0/1309292571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a892da73b0 msgr2=0x55a8931a8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:53.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.664+0000 7f2430a75640 1 --2- 172.21.15.76:0/1309292571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a892da73b0 0x55a8931a8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.664+0000 7f2430a75640 1 -- 172.21.15.76:0/1309292571 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a89308e750 con 0x55a892daf590 2024-04-25T19:15:53.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.665+0000 7f2430a75640 1 --2- 172.21.15.76:0/1309292571 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892daf590 0x55a8931a91e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2424011420 tx=0x7f2424009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce0a8adcf422b92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:53.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.665+0000 7f2411ffb640 1 -- 172.21.15.76:0/1309292571 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2424015900 con 0x55a892daf590 2024-04-25T19:15:53.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.665+0000 7f2411ffb640 1 -- 172.21.15.76:0/1309292571 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2424015aa0 con 0x55a892daf590 2024-04-25T19:15:53.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.665+0000 7f2411ffb640 1 -- 172.21.15.76:0/1309292571 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2424015e50 con 0x55a892daf590 2024-04-25T19:15:53.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.665+0000 7f2439385740 1 -- 172.21.15.76:0/1309292571 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892daf590 msgr2=0x55a8931a91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:53.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.665+0000 7f2439385740 1 --2- 172.21.15.76:0/1309292571 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892daf590 0x55a8931a91e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2424011420 tx=0x7f2424009750 comp rx=0 tx=0).stop 2024-04-25T19:15:53.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.666+0000 7f2439385740 1 -- 172.21.15.76:0/1309292571 shutdown_connections 2024-04-25T19:15:53.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.666+0000 7f2439385740 1 --2- 172.21.15.76:0/1309292571 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892daf590 0x55a8931a91e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.666+0000 7f2439385740 1 --2- 172.21.15.76:0/1309292571 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a892da73b0 0x55a8931a8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.666+0000 7f2439385740 1 --2- 172.21.15.76:0/1309292571 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55a892dd0270 0x55a8931b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.666+0000 7f2439385740 1 -- 172.21.15.76:0/1309292571 >> 172.21.15.76:0/1309292571 conn(0x55a892dc43c0 msgr2=0x55a893170d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:53.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.666+0000 7f2439385740 1 -- 172.21.15.76:0/1309292571 shutdown_connections 2024-04-25T19:15:53.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.666+0000 7f2439385740 1 -- 172.21.15.76:0/1309292571 wait complete. 2024-04-25T19:15:53.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.666+0000 7f2439385740 1 -- start start 2024-04-25T19:15:53.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.667+0000 7f2439385740 1 Processor -- start 2024-04-25T19:15:53.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.668+0000 7f2439385740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892da73b0 0x55a893197100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.668+0000 7f2439385740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55a892daf590 0x55a8931aef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.668+0000 7f2439385740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a892dd0270 0x55a8931aacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.668+0000 7f2439385740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a89305dfe0 con 0x55a892da73b0 2024-04-25T19:15:53.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.668+0000 7f2439385740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a8931af930 con 0x55a892dd0270 2024-04-25T19:15:53.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.668+0000 7f2439385740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x55a8931aa310 con 0x55a892daf590 2024-04-25T19:15:53.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.668+0000 7f2439385740 10 client.0 _ll_register_callbacks cb 0x55a893074150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:15:53.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.668+0000 7f242bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892da73b0 0x55a893197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:53.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.668+0000 7f242bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892da73b0 0x55a893197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37854/0 (socket says 172.21.15.76:37854) 2024-04-25T19:15:53.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.668+0000 7f242bfff640 1 -- 172.21.15.76:0/3010697345 learned_addr learned my addr 172.21.15.76:0/3010697345 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:53.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.668+0000 7f242b7fe640 1 --2- 172.21.15.76:0/3010697345 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55a892daf590 0x55a8931aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:53.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.668+0000 7f2430a75640 1 --2- 172.21.15.76:0/3010697345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a892dd0270 0x55a8931aacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:53.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.668+0000 7f242bfff640 1 -- 172.21.15.76:0/3010697345 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55a892daf590 msgr2=0x55a8931aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:53.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.668+0000 7f242bfff640 1 --2- 172.21.15.76:0/3010697345 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55a892daf590 0x55a8931aef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.668+0000 7f242bfff640 1 -- 172.21.15.76:0/3010697345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a892dd0270 msgr2=0x55a8931aacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:53.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.668+0000 7f242bfff640 1 --2- 172.21.15.76:0/3010697345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a892dd0270 0x55a8931aacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.668+0000 7f242bfff640 1 -- 172.21.15.76:0/3010697345 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a892de82d0 con 0x55a892da73b0 2024-04-25T19:15:53.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.668+0000 7f242b7fe640 1 --2- 172.21.15.76:0/3010697345 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55a892daf590 0x55a8931aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:53.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.668+0000 7f242bfff640 1 --2- 172.21.15.76:0/3010697345 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892da73b0 0x55a893197100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f241c02a2b0 tx=0x7f241c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4118ad5271d45912 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:53.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.668+0000 7f240affd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-04-25T19:15:53.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.669+0000 7f240affd640 1 -- 172.21.15.76:0/3010697345 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c037040 con 0x55a892da73b0 2024-04-25T19:15:53.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.669+0000 7f240affd640 1 -- 172.21.15.76:0/3010697345 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f241c03ed40 con 0x55a892da73b0 2024-04-25T19:15:53.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.669+0000 7f2439385740 1 -- 172.21.15.76:0/3010697345 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2424015aa0 con 0x55a892da73b0 2024-04-25T19:15:53.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.669+0000 7f240affd640 1 -- 172.21.15.76:0/3010697345 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f241c057050 con 0x55a892da73b0 2024-04-25T19:15:53.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.669+0000 7f2439385740 10 client.4728 fetch_fsmap learned FSMap version 16 2024-04-25T19:15:53.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.669+0000 7f2439385740 1 -- 172.21.15.76:0/3010697345 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55a892eb26b0 con 0x55a892da73b0 2024-04-25T19:15:53.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.669+0000 7f240affd640 1 -- 172.21.15.76:0/3010697345 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2424015aa0 con 0x55a892da73b0 2024-04-25T19:15:53.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.669+0000 7f240affd640 1 -- 172.21.15.76:0/3010697345 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 16) ==== 4784+0+0 (secure 0 0 0) 0x7f241c05a020 con 0x55a892da73b0 2024-04-25T19:15:53.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.669+0000 7f2439385740 10 client.4728 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T19:15:53.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.669+0000 7f2439385740 10 client.4728 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T19:15:53.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.669+0000 7f2439385740 20 client.4728 populate_metadata read hostname 'smithi076' 2024-04-25T19:15:53.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.669+0000 7f2439385740 1 --2- 172.21.15.76:0/3010697345 >> [v2:172.21.15.105:6835/2724142282,v1:172.21.15.105:6838/2724142282] conn(0x55a892d8ee40 0x55a8931d4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.669+0000 7f2439385740 4 client.4728 mds_command: new command op to 4238 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:15:53.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.669+0000 7f2439385740 1 -- 172.21.15.76:0/3010697345 --> [v2:172.21.15.105:6835/2724142282,v1:172.21.15.105:6838/2724142282] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8930e4350 con 0x55a892d8ee40 2024-04-25T19:15:53.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.669+0000 7f242b7fe640 1 --2- 172.21.15.76:0/3010697345 >> [v2:172.21.15.105:6835/2724142282,v1:172.21.15.105:6838/2724142282] conn(0x55a892d8ee40 0x55a8931d4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:53.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.670+0000 7f242b7fe640 1 --2- 172.21.15.76:0/3010697345 >> [v2:172.21.15.105:6835/2724142282,v1:172.21.15.105:6838/2724142282] conn(0x55a892d8ee40 0x55a8931d4fa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6de50533f5df9b0 server_cookie=a1ddf2d05d6b5359 in_seq=0 out_seq=0 2024-04-25T19:15:53.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.670+0000 7f240affd640 10 client.4728 ms_handle_connect on v2:172.21.15.105:6835/2724142282 2024-04-25T19:15:53.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.670+0000 7f240affd640 1 -- 172.21.15.76:0/3010697345 <== mds.3 v2:172.21.15.105:6835/2724142282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a8930e4350 con 0x55a892d8ee40 2024-04-25T19:15:53.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.670+0000 7f240affd640 10 client.4728 handle_command_reply: tid=0 2024-04-25T19:15:53.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.671+0000 7f2439385740 1 client.4728 shutdown 2024-04-25T19:15:53.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.671+0000 7f2439385740 1 -- 172.21.15.76:0/3010697345 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892da73b0 msgr2=0x55a893197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:53.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.671+0000 7f2439385740 1 --2- 172.21.15.76:0/3010697345 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892da73b0 0x55a893197100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f241c02a2b0 tx=0x7f241c004730 comp rx=0 tx=0).stop 2024-04-25T19:15:53.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.671+0000 7f2439385740 1 -- 172.21.15.76:0/3010697345 shutdown_connections 2024-04-25T19:15:53.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.671+0000 7f2439385740 1 --2- 172.21.15.76:0/3010697345 >> [v2:172.21.15.105:6835/2724142282,v1:172.21.15.105:6838/2724142282] conn(0x55a892d8ee40 0x55a8931d4fa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.671+0000 7f2439385740 1 --2- 172.21.15.76:0/3010697345 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a892dd0270 0x55a8931aacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.671+0000 7f240affd640 0 client.4728 ms_handle_reset on v2:172.21.15.105:6835/2724142282 2024-04-25T19:15:53.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.671+0000 7f2439385740 1 --2- 172.21.15.76:0/3010697345 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55a892daf590 0x55a8931aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.671+0000 7f2439385740 1 --2- 172.21.15.76:0/3010697345 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892da73b0 0x55a893197100 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.671+0000 7f2439385740 1 -- 172.21.15.76:0/3010697345 >> 172.21.15.76:0/3010697345 conn(0x55a892dc43c0 msgr2=0x55a8931707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:53.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.671+0000 7f2439385740 1 -- 172.21.15.76:0/3010697345 shutdown_connections 2024-04-25T19:15:53.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.671+0000 7f2439385740 1 -- 172.21.15.76:0/3010697345 wait complete. 2024-04-25T19:15:53.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.671+0000 7f2439385740 20 client.4728 trim_cache size 0 max 16384 2024-04-25T19:15:53.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.689+0000 7f2439385740 1 Processor -- start 2024-04-25T19:15:53.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.689+0000 7f2439385740 1 -- start start 2024-04-25T19:15:53.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.691+0000 7f2439385740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a892da73b0 0x55a8931aba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.691+0000 7f2439385740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892daf590 0x55a8931abf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.691+0000 7f2439385740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55a892dd0270 0x55a893171a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.691+0000 7f2439385740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f242c1f1790 con 0x55a892daf590 2024-04-25T19:15:53.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.691+0000 7f2439385740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a8931aa030 con 0x55a892da73b0 2024-04-25T19:15:53.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.691+0000 7f2439385740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x55a8931af710 con 0x55a892dd0270 2024-04-25T19:15:53.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.691+0000 7f242b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892daf590 0x55a8931abf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:53.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.691+0000 7f242b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892daf590 0x55a8931abf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37866/0 (socket says 172.21.15.76:37866) 2024-04-25T19:15:53.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.691+0000 7f242b7fe640 1 -- 172.21.15.76:0/1765192056 learned_addr learned my addr 172.21.15.76:0/1765192056 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:53.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.691+0000 7f242bfff640 1 --2- 172.21.15.76:0/1765192056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a892da73b0 0x55a8931aba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:53.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.691+0000 7f2430a75640 1 --2- 172.21.15.76:0/1765192056 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55a892dd0270 0x55a893171a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:53.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.691+0000 7f242b7fe640 1 -- 172.21.15.76:0/1765192056 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55a892dd0270 msgr2=0x55a893171a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:53.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.691+0000 7f242b7fe640 1 --2- 172.21.15.76:0/1765192056 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55a892dd0270 0x55a893171a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.691+0000 7f242b7fe640 1 -- 172.21.15.76:0/1765192056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a892da73b0 msgr2=0x55a8931aba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:53.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.691+0000 7f242b7fe640 1 --2- 172.21.15.76:0/1765192056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a892da73b0 0x55a8931aba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.691+0000 7f242b7fe640 1 -- 172.21.15.76:0/1765192056 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8930dba00 con 0x55a892daf590 2024-04-25T19:15:53.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.691+0000 7f242b7fe640 1 --2- 172.21.15.76:0/1765192056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892daf590 0x55a8931abf80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f242005af30 tx=0x7f2420014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc1ca91a4af5f82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:53.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.691+0000 7f24117fa640 1 -- 172.21.15.76:0/1765192056 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242005aa00 con 0x55a892daf590 2024-04-25T19:15:53.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.691+0000 7f24117fa640 1 -- 172.21.15.76:0/1765192056 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24200940d0 con 0x55a892daf590 2024-04-25T19:15:53.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.691+0000 7f24117fa640 1 -- 172.21.15.76:0/1765192056 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242005aba0 con 0x55a892daf590 2024-04-25T19:15:53.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.692+0000 7f2439385740 1 -- 172.21.15.76:0/1765192056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892daf590 msgr2=0x55a8931abf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:53.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.692+0000 7f2439385740 1 --2- 172.21.15.76:0/1765192056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892daf590 0x55a8931abf80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f242005af30 tx=0x7f2420014040 comp rx=0 tx=0).stop 2024-04-25T19:15:53.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.692+0000 7f2439385740 1 -- 172.21.15.76:0/1765192056 shutdown_connections 2024-04-25T19:15:53.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.692+0000 7f2439385740 1 --2- 172.21.15.76:0/1765192056 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55a892dd0270 0x55a893171a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.692+0000 7f2439385740 1 --2- 172.21.15.76:0/1765192056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892daf590 0x55a8931abf80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.692+0000 7f2439385740 1 --2- 172.21.15.76:0/1765192056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a892da73b0 0x55a8931aba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.692+0000 7f2439385740 1 -- 172.21.15.76:0/1765192056 >> 172.21.15.76:0/1765192056 conn(0x55a892dc43c0 msgr2=0x55a89319b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:53.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.692+0000 7f2439385740 1 -- 172.21.15.76:0/1765192056 shutdown_connections 2024-04-25T19:15:53.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.692+0000 7f2439385740 1 -- 172.21.15.76:0/1765192056 wait complete. 2024-04-25T19:15:53.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.693+0000 7f2439385740 1 -- start start 2024-04-25T19:15:53.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.693+0000 7f2439385740 1 Processor -- start 2024-04-25T19:15:53.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.694+0000 7f2439385740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55a892da73b0 0x55a89316c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.694+0000 7f2439385740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a892daf590 0x55a89322e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.694+0000 7f2439385740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892dd0270 0x55a89322ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.694+0000 7f2439385740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a8931ae160 con 0x55a892dd0270 2024-04-25T19:15:53.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.694+0000 7f2439385740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55a8931ab240 con 0x55a892daf590 2024-04-25T19:15:53.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.694+0000 7f2439385740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x55a8931c9c50 con 0x55a892da73b0 2024-04-25T19:15:53.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.694+0000 7f2439385740 10 client.0 _ll_register_callbacks cb 0x7f241c0096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:15:53.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.694+0000 7f2430a75640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892dd0270 0x55a89322ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:53.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.694+0000 7f2430a75640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892dd0270 0x55a89322ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37872/0 (socket says 172.21.15.76:37872) 2024-04-25T19:15:53.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.694+0000 7f2430a75640 1 -- 172.21.15.76:0/4028493156 learned_addr learned my addr 172.21.15.76:0/4028493156 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:53.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.694+0000 7f242bfff640 1 --2- 172.21.15.76:0/4028493156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55a892da73b0 0x55a89316c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:53.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.694+0000 7f242b7fe640 1 --2- 172.21.15.76:0/4028493156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a892daf590 0x55a89322e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:53.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.695+0000 7f2430a75640 1 -- 172.21.15.76:0/4028493156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55a892da73b0 msgr2=0x55a89316c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:53.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.695+0000 7f2430a75640 1 --2- 172.21.15.76:0/4028493156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55a892da73b0 0x55a89316c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.695+0000 7f2430a75640 1 -- 172.21.15.76:0/4028493156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a892daf590 msgr2=0x55a89322e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:53.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.695+0000 7f2430a75640 1 --2- 172.21.15.76:0/4028493156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a892daf590 0x55a89322e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:53.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.695+0000 7f2430a75640 1 -- 172.21.15.76:0/4028493156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a893172090 con 0x55a892dd0270 2024-04-25T19:15:53.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.695+0000 7f2430a75640 1 --2- 172.21.15.76:0/4028493156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892dd0270 0x55a89322ed20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2424019000 tx=0x7f2424004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fdf2d3faaac9fe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:53.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.695+0000 7f240affd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-04-25T19:15:53.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.695+0000 7f240affd640 1 -- 172.21.15.76:0/4028493156 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2424015900 con 0x55a892dd0270 2024-04-25T19:15:53.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.695+0000 7f240affd640 1 -- 172.21.15.76:0/4028493156 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2424015aa0 con 0x55a892dd0270 2024-04-25T19:15:53.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.695+0000 7f2439385740 1 -- 172.21.15.76:0/4028493156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24200940d0 con 0x55a892dd0270 2024-04-25T19:15:53.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.695+0000 7f240affd640 1 -- 172.21.15.76:0/4028493156 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f2424036050 con 0x55a892dd0270 2024-04-25T19:15:53.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.695+0000 7f2439385740 10 client.4734 fetch_fsmap learned FSMap version 16 2024-04-25T19:15:53.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.695+0000 7f2439385740 1 -- 172.21.15.76:0/4028493156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55a89322f260 con 0x55a892dd0270 2024-04-25T19:15:53.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.695+0000 7f240affd640 1 -- 172.21.15.76:0/4028493156 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24200940d0 con 0x55a892dd0270 2024-04-25T19:15:53.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.695+0000 7f240affd640 1 -- 172.21.15.76:0/4028493156 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 16) ==== 4784+0+0 (secure 0 0 0) 0x7f2424039020 con 0x55a892dd0270 2024-04-25T19:15:53.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.695+0000 7f2439385740 10 client.4734 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T19:15:53.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.696+0000 7f2439385740 10 client.4734 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T19:15:53.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.696+0000 7f2439385740 20 client.4734 populate_metadata read hostname 'smithi076' 2024-04-25T19:15:53.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.696+0000 7f2439385740 1 --2- 172.21.15.76:0/4028493156 >> [v2:172.21.15.105:6835/2724142282,v1:172.21.15.105:6838/2724142282] conn(0x55a893196870 0x55a893241400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:53.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.696+0000 7f2439385740 4 client.4734 mds_command: new command op to 4238 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T19:15:53.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.696+0000 7f2439385740 1 -- 172.21.15.76:0/4028493156 --> [v2:172.21.15.105:6835/2724142282,v1:172.21.15.105:6838/2724142282] -- command(tid 0: {"prefix": "status"}) -- 0x55a8930de1c0 con 0x55a893196870 2024-04-25T19:15:53.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.696+0000 7f242bfff640 1 --2- 172.21.15.76:0/4028493156 >> [v2:172.21.15.105:6835/2724142282,v1:172.21.15.105:6838/2724142282] conn(0x55a893196870 0x55a893241400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:53.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.696+0000 7f240affd640 10 client.4734 ms_handle_connect on v2:172.21.15.105:6835/2724142282 2024-04-25T19:15:53.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:53.696+0000 7f242bfff640 1 --2- 172.21.15.76:0/4028493156 >> [v2:172.21.15.105:6835/2724142282,v1:172.21.15.105:6838/2724142282] conn(0x55a893196870 0x55a893241400 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bc3c0a4d22a3712d server_cookie=1dc1fc22de1b7a3 in_seq=0 out_seq=0 2024-04-25T19:15:54.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.011+0000 7f240affd640 1 -- 172.21.15.76:0/4028493156 <== mds.3 v2:172.21.15.105:6835/2724142282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55a8930de1c0 con 0x55a893196870 2024-04-25T19:15:54.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.011+0000 7f240affd640 10 client.4734 handle_command_reply: tid=0 2024-04-25T19:15:54.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.011+0000 7f2439385740 1 client.4734 shutdown 2024-04-25T19:15:54.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.011+0000 7f2439385740 1 -- 172.21.15.76:0/4028493156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892dd0270 msgr2=0x55a89322ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.011+0000 7f2439385740 1 --2- 172.21.15.76:0/4028493156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892dd0270 0x55a89322ed20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2424019000 tx=0x7f2424004ab0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.011+0000 7f2439385740 1 -- 172.21.15.76:0/4028493156 shutdown_connections 2024-04-25T19:15:54.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.011+0000 7f2439385740 1 --2- 172.21.15.76:0/4028493156 >> [v2:172.21.15.105:6835/2724142282,v1:172.21.15.105:6838/2724142282] conn(0x55a893196870 0x55a893241400 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.011+0000 7f2439385740 1 --2- 172.21.15.76:0/4028493156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a892dd0270 0x55a89322ed20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.011+0000 7f2439385740 1 --2- 172.21.15.76:0/4028493156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55a892daf590 0x55a89322e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.011+0000 7f2439385740 1 --2- 172.21.15.76:0/4028493156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55a892da73b0 0x55a89316c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.011+0000 7f240affd640 0 client.4734 ms_handle_reset on v2:172.21.15.105:6835/2724142282 2024-04-25T19:15:54.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.011+0000 7f2439385740 1 -- 172.21.15.76:0/4028493156 >> 172.21.15.76:0/4028493156 conn(0x55a892dc43c0 msgr2=0x55a8931ac120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:54.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.011+0000 7f2439385740 1 -- 172.21.15.76:0/4028493156 shutdown_connections 2024-04-25T19:15:54.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.011+0000 7f2439385740 1 -- 172.21.15.76:0/4028493156 wait complete. 2024-04-25T19:15:54.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.011+0000 7f2439385740 20 client.4734 trim_cache size 0 max 16384 2024-04-25T19:15:54.015 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:15:54.015 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:15:54.015 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 3, 2024-04-25T19:15:54.015 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4238, 2024-04-25T19:15:54.015 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-04-25T19:15:54.015 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-04-25T19:15:54.015 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-04-25T19:15:54.015 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 17.528968856999999, 2024-04-25T19:15:54.015 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 16, 2024-04-25T19:15:54.015 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 21, 2024-04-25T19:15:54.015 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 21, 2024-04-25T19:15:54.015 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 41.633399429000001 2024-04-25T19:15:54.016 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:15:54.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.011+0000 7f24324d8640 1 -- 172.21.15.76:0/3251576980 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f24180744b0 msgr2=0x7f24180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.011+0000 7f24324d8640 1 --2- 172.21.15.76:0/3251576980 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f24180744b0 0x7f24180769c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f241c034000 tx=0x7f241c02e000 comp rx=0 tx=0).stop 2024-04-25T19:15:54.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.011+0000 7f24324d8640 1 -- 172.21.15.76:0/3251576980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f242c159840 msgr2=0x7f242c149840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.011+0000 7f24324d8640 1 --2- 172.21.15.76:0/3251576980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f242c159840 0x7f242c149840 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f242000c3f0 tx=0x7f242000c8b0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.011+0000 7f24324d8640 1 -- 172.21.15.76:0/3251576980 shutdown_connections 2024-04-25T19:15:54.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.011+0000 7f24324d8640 1 --2- 172.21.15.76:0/3251576980 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f24180744b0 0x7f24180769c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.011+0000 7f24324d8640 1 --2- 172.21.15.76:0/3251576980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f242c15d880 0x7f242c152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.011+0000 7f24324d8640 1 --2- 172.21.15.76:0/3251576980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f242c159840 0x7f242c149840 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.011+0000 7f24324d8640 1 --2- 172.21.15.76:0/3251576980 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f242c0c5b20 0x7f242c154cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.011+0000 7f24324d8640 1 -- 172.21.15.76:0/3251576980 >> 172.21.15.76:0/3251576980 conn(0x7f242c0cecc0 msgr2=0x7f242c1572a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:54.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.012+0000 7f24324d8640 1 -- 172.21.15.76:0/3251576980 shutdown_connections 2024-04-25T19:15:54.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.012+0000 7f24324d8640 1 -- 172.21.15.76:0/3251576980 wait complete. 2024-04-25T19:15:54.026 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T19:15:54.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.135+0000 7f1e87292640 1 Processor -- start 2024-04-25T19:15:54.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.135+0000 7f1e87292640 1 -- start start 2024-04-25T19:15:54.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.135+0000 7f1e87292640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1e800cb880 0x7f1e8015d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.135+0000 7f1e87292640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e80072ba0 0x7f1e80072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.135+0000 7f1e87292640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e800cbd50 con 0x7f1e800cad30 2024-04-25T19:15:54.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.135+0000 7f1e87292640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f1e80161340 con 0x7f1e800cb880 2024-04-25T19:15:54.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.135+0000 7f1e87292640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f1e801607a0 con 0x7f1e80072ba0 2024-04-25T19:15:54.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.137+0000 7f1e8582f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e80072ba0 0x7f1e80072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.137+0000 7f1e8502e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f1e800cb880 0x7f1e8015d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.137+0000 7f1e8582f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e80072ba0 0x7f1e80072fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:44390/0 (socket says 172.21.15.76:44390) 2024-04-25T19:15:54.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.137+0000 7f1e8582f640 1 -- 172.21.15.76:0/2153259543 learned_addr learned my addr 172.21.15.76:0/2153259543 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:54.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.137+0000 7f1e8582f640 1 -- 172.21.15.76:0/2153259543 >> v2:172.21.15.105:3300/0 conn(0x7f1e800cb880 msgr2=0x7f1e8015d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.137+0000 7f1e8582f640 1 --2- 172.21.15.76:0/2153259543 >> v2:172.21.15.105:3300/0 conn(0x7f1e800cb880 0x7f1e8015d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.137+0000 7f1e8582f640 1 -- 172.21.15.76:0/2153259543 >> v1:172.21.15.105:6789/0 conn(0x7f1e800cad30 legacy=0x7f1e80160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.137+0000 7f1e8582f640 1 -- 172.21.15.76:0/2153259543 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e800cc010 con 0x7f1e80072ba0 2024-04-25T19:15:54.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.137+0000 7f1e8502e640 1 --2- 172.21.15.76:0/2153259543 >> v2:172.21.15.105:3300/0 conn(0x7f1e800cb880 0x7f1e8015d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:54.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.137+0000 7f1e8582f640 1 --2- 172.21.15.76:0/2153259543 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e80072ba0 0x7f1e80072fb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009870 tx=0x7f1e7c02a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1cb795a63eddbc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:54.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.137+0000 7f1e77fff640 1 -- 172.21.15.76:0/2153259543 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c037070 con 0x7f1e80072ba0 2024-04-25T19:15:54.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.137+0000 7f1e77fff640 1 -- 172.21.15.76:0/2153259543 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e7c02ae50 con 0x7f1e80072ba0 2024-04-25T19:15:54.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.137+0000 7f1e77fff640 1 -- 172.21.15.76:0/2153259543 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c032860 con 0x7f1e80072ba0 2024-04-25T19:15:54.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.138+0000 7f1e87292640 1 -- 172.21.15.76:0/2153259543 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e80072ba0 msgr2=0x7f1e80072fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.138+0000 7f1e87292640 1 --2- 172.21.15.76:0/2153259543 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e80072ba0 0x7f1e80072fb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009870 tx=0x7f1e7c02a300 comp rx=0 tx=0).stop 2024-04-25T19:15:54.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.138+0000 7f1e87292640 1 -- 172.21.15.76:0/2153259543 shutdown_connections 2024-04-25T19:15:54.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.138+0000 7f1e87292640 1 --2- 172.21.15.76:0/2153259543 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e80072ba0 0x7f1e80072fb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.138+0000 7f1e87292640 1 --2- 172.21.15.76:0/2153259543 >> v2:172.21.15.105:3300/0 conn(0x7f1e800cb880 0x7f1e8015d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.138+0000 7f1e87292640 1 -- 172.21.15.76:0/2153259543 >> 172.21.15.76:0/2153259543 conn(0x7f1e80070b00 msgr2=0x7f1e80070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:54.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.138+0000 7f1e87292640 1 -- 172.21.15.76:0/2153259543 shutdown_connections 2024-04-25T19:15:54.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.138+0000 7f1e87292640 1 -- 172.21.15.76:0/2153259543 wait complete. 2024-04-25T19:15:54.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.140+0000 7f1e87292640 1 Processor -- start 2024-04-25T19:15:54.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.140+0000 7f1e87292640 1 -- start start 2024-04-25T19:15:54.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.142+0000 7f1e87292640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e80072ba0 0x7f1e80151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.142+0000 7f1e87292640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e800cad30 0x7f1e8015f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.142+0000 7f1e87292640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e800cb880 0x7f1e80150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.142+0000 7f1e87292640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8006e7f0 con 0x7f1e80072ba0 2024-04-25T19:15:54.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.142+0000 7f1e87292640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1e80160430 con 0x7f1e800cad30 2024-04-25T19:15:54.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.142+0000 7f1e87292640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f1e800cc5d0 con 0x7f1e800cb880 2024-04-25T19:15:54.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.142+0000 7f1e8502e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e80072ba0 0x7f1e80151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.142+0000 7f1e8502e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e80072ba0 0x7f1e80151ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37876/0 (socket says 172.21.15.76:37876) 2024-04-25T19:15:54.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.142+0000 7f1e8502e640 1 -- 172.21.15.76:0/3106091277 learned_addr learned my addr 172.21.15.76:0/3106091277 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:54.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.142+0000 7f1e8482d640 1 --2- 172.21.15.76:0/3106091277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e800cad30 0x7f1e8015f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.142+0000 7f1e8582f640 1 --2- 172.21.15.76:0/3106091277 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e800cb880 0x7f1e80150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.142+0000 7f1e8502e640 1 -- 172.21.15.76:0/3106091277 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e800cb880 msgr2=0x7f1e80150880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.142+0000 7f1e8502e640 1 --2- 172.21.15.76:0/3106091277 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e800cb880 0x7f1e80150880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.142+0000 7f1e8502e640 1 -- 172.21.15.76:0/3106091277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e800cad30 msgr2=0x7f1e8015f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.142+0000 7f1e8502e640 1 --2- 172.21.15.76:0/3106091277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e800cad30 0x7f1e8015f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.142+0000 7f1e8502e640 1 -- 172.21.15.76:0/3106091277 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8015fd80 con 0x7f1e80072ba0 2024-04-25T19:15:54.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.142+0000 7f1e8502e640 1 --2- 172.21.15.76:0/3106091277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e80072ba0 0x7f1e80151ba0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1e78009ec0 tx=0x7f1e7800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:54.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.142+0000 7f1e75ffb640 1 -- 172.21.15.76:0/3106091277 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e78018020 con 0x7f1e80072ba0 2024-04-25T19:15:54.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.142+0000 7f1e75ffb640 1 -- 172.21.15.76:0/3106091277 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e78014680 con 0x7f1e80072ba0 2024-04-25T19:15:54.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.142+0000 7f1e87292640 1 -- 172.21.15.76:0/3106091277 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e801520e0 con 0x7f1e80072ba0 2024-04-25T19:15:54.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.142+0000 7f1e75ffb640 1 -- 172.21.15.76:0/3106091277 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e78014a50 con 0x7f1e80072ba0 2024-04-25T19:15:54.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.142+0000 7f1e87292640 1 -- 172.21.15.76:0/3106091277 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e800d01c0 con 0x7f1e80072ba0 2024-04-25T19:15:54.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.143+0000 7f1e75ffb640 1 -- 172.21.15.76:0/3106091277 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1e78005d10 con 0x7f1e80072ba0 2024-04-25T19:15:54.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.144+0000 7f1e75ffb640 1 --2- 172.21.15.76:0/3106091277 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1e640744e0 0x7f1e640769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.144+0000 7f1e75ffb640 1 -- 172.21.15.76:0/3106091277 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1e78095d60 con 0x7f1e80072ba0 2024-04-25T19:15:54.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.144+0000 7f1e8482d640 1 --2- 172.21.15.76:0/3106091277 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1e640744e0 0x7f1e640769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.145+0000 7f1e8482d640 1 --2- 172.21.15.76:0/3106091277 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1e640744e0 0x7f1e640769f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1e8015ac50 tx=0x7f1e70009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:54.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.145+0000 7f1e8e218740 1 Processor -- start 2024-04-25T19:15:54.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.145+0000 7f1e8e218740 1 -- start start 2024-04-25T19:15:54.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.146+0000 7f1e8e218740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55928726b270 0x55928760d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.146+0000 7f1e8e218740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592872423b0 0x559287607880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.146+0000 7f1e8e218740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55928724a590 0x559287607dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.146+0000 7f1e8e218740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559287571740 con 0x5592872423b0 2024-04-25T19:15:54.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.146+0000 7f1e8e218740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x55928750d3a0 con 0x55928726b270 2024-04-25T19:15:54.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.146+0000 7f1e8e218740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x55928756d700 con 0x55928724a590 2024-04-25T19:15:54.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.146+0000 7f1e8482d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592872423b0 0x559287607880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.146+0000 7f1e8482d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592872423b0 0x559287607880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37878/0 (socket says 172.21.15.76:37878) 2024-04-25T19:15:54.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.146+0000 7f1e8482d640 1 -- 172.21.15.76:0/2894783128 learned_addr learned my addr 172.21.15.76:0/2894783128 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:54.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.146+0000 7f1e8502e640 1 --2- 172.21.15.76:0/2894783128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55928726b270 0x55928760d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.146+0000 7f1e8582f640 1 --2- 172.21.15.76:0/2894783128 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55928724a590 0x559287607dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.147+0000 7f1e8482d640 1 -- 172.21.15.76:0/2894783128 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55928724a590 msgr2=0x559287607dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.147+0000 7f1e8482d640 1 --2- 172.21.15.76:0/2894783128 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55928724a590 0x559287607dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.147+0000 7f1e8482d640 1 -- 172.21.15.76:0/2894783128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55928726b270 msgr2=0x55928760d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.147+0000 7f1e8482d640 1 --2- 172.21.15.76:0/2894783128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55928726b270 0x55928760d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.147+0000 7f1e8482d640 1 -- 172.21.15.76:0/2894783128 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559287529750 con 0x5592872423b0 2024-04-25T19:15:54.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.147+0000 7f1e8482d640 1 --2- 172.21.15.76:0/2894783128 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592872423b0 0x559287607880 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000fa10 tx=0x7f1e70012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0ff3efa6737389 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:54.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.147+0000 7f1e6affd640 1 -- 172.21.15.76:0/2894783128 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70025070 con 0x5592872423b0 2024-04-25T19:15:54.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.147+0000 7f1e6affd640 1 -- 172.21.15.76:0/2894783128 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e7001b050 con 0x5592872423b0 2024-04-25T19:15:54.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.147+0000 7f1e6affd640 1 -- 172.21.15.76:0/2894783128 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70001420 con 0x5592872423b0 2024-04-25T19:15:54.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.148+0000 7f1e8e218740 1 -- 172.21.15.76:0/2894783128 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592872423b0 msgr2=0x559287607880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.148+0000 7f1e8e218740 1 --2- 172.21.15.76:0/2894783128 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592872423b0 0x559287607880 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000fa10 tx=0x7f1e70012000 comp rx=0 tx=0).stop 2024-04-25T19:15:54.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.148+0000 7f1e8e218740 1 -- 172.21.15.76:0/2894783128 shutdown_connections 2024-04-25T19:15:54.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.148+0000 7f1e8e218740 1 --2- 172.21.15.76:0/2894783128 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55928724a590 0x559287607dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.148+0000 7f1e8e218740 1 --2- 172.21.15.76:0/2894783128 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592872423b0 0x559287607880 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.148+0000 7f1e8e218740 1 --2- 172.21.15.76:0/2894783128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55928726b270 0x55928760d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.148+0000 7f1e8e218740 1 -- 172.21.15.76:0/2894783128 >> 172.21.15.76:0/2894783128 conn(0x55928725f3c0 msgr2=0x55928760bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:54.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.148+0000 7f1e8e218740 1 -- 172.21.15.76:0/2894783128 shutdown_connections 2024-04-25T19:15:54.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.148+0000 7f1e8e218740 1 -- 172.21.15.76:0/2894783128 wait complete. 2024-04-25T19:15:54.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.149+0000 7f1e8e218740 1 -- start start 2024-04-25T19:15:54.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.149+0000 7f1e8e218740 1 Processor -- start 2024-04-25T19:15:54.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.150+0000 7f1e8e218740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592872423b0 0x55928764d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.150+0000 7f1e8e218740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55928724a590 0x559287634760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.150+0000 7f1e8e218740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55928726b270 0x559287646820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.150+0000 7f1e8e218740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55928764bd00 con 0x55928726b270 2024-04-25T19:15:54.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.150+0000 7f1e8e218740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5592874f8fe0 con 0x5592872423b0 2024-04-25T19:15:54.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.150+0000 7f1e8e218740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x55928760bbe0 con 0x55928724a590 2024-04-25T19:15:54.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.150+0000 7f1e8e218740 10 client.0 _ll_register_callbacks cb 0x55928750f150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:15:54.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.150+0000 7f1e8582f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55928726b270 0x559287646820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.150+0000 7f1e8582f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55928726b270 0x559287646820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37884/0 (socket says 172.21.15.76:37884) 2024-04-25T19:15:54.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.150+0000 7f1e8582f640 1 -- 172.21.15.76:0/3973639635 learned_addr learned my addr 172.21.15.76:0/3973639635 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:54.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.150+0000 7f1e8502e640 1 --2- 172.21.15.76:0/3973639635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592872423b0 0x55928764d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.150+0000 7f1e8482d640 1 --2- 172.21.15.76:0/3973639635 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55928724a590 0x559287634760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.150+0000 7f1e8582f640 1 -- 172.21.15.76:0/3973639635 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55928724a590 msgr2=0x559287634760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.150+0000 7f1e8582f640 1 --2- 172.21.15.76:0/3973639635 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55928724a590 0x559287634760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.150+0000 7f1e8582f640 1 -- 172.21.15.76:0/3973639635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592872423b0 msgr2=0x55928764d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.150+0000 7f1e8582f640 1 --2- 172.21.15.76:0/3973639635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592872423b0 0x55928764d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.150+0000 7f1e8582f640 1 -- 172.21.15.76:0/3973639635 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592872832d0 con 0x55928726b270 2024-04-25T19:15:54.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.151+0000 7f1e8582f640 1 --2- 172.21.15.76:0/3973639635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55928726b270 0x559287646820 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009990 tx=0x7f1e7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e08e3c2231e6804d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:54.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.151+0000 7f1e5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-04-25T19:15:54.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.151+0000 7f1e5ffff640 1 -- 172.21.15.76:0/3973639635 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c037040 con 0x55928726b270 2024-04-25T19:15:54.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.151+0000 7f1e5ffff640 1 -- 172.21.15.76:0/3973639635 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1e7c046410 con 0x55928726b270 2024-04-25T19:15:54.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.151+0000 7f1e8e218740 1 -- 172.21.15.76:0/3973639635 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e7001b050 con 0x55928726b270 2024-04-25T19:15:54.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.151+0000 7f1e5ffff640 1 -- 172.21.15.76:0/3973639635 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f1e7c04e050 con 0x55928726b270 2024-04-25T19:15:54.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.151+0000 7f1e8e218740 10 client.4743 fetch_fsmap learned FSMap version 16 2024-04-25T19:15:54.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.151+0000 7f1e8e218740 1 -- 172.21.15.76:0/3973639635 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55928734d6b0 con 0x55928726b270 2024-04-25T19:15:54.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.151+0000 7f1e5ffff640 1 -- 172.21.15.76:0/3973639635 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e7001b050 con 0x55928726b270 2024-04-25T19:15:54.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.151+0000 7f1e5ffff640 1 -- 172.21.15.76:0/3973639635 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 16) ==== 4784+0+0 (secure 0 0 0) 0x7f1e7c051020 con 0x55928726b270 2024-04-25T19:15:54.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.151+0000 7f1e8e218740 10 client.4743 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T19:15:54.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.151+0000 7f1e8e218740 10 client.4743 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T19:15:54.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.151+0000 7f1e8e218740 20 client.4743 populate_metadata read hostname 'smithi076' 2024-04-25T19:15:54.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.151+0000 7f1e8e218740 1 --2- 172.21.15.76:0/3973639635 >> [v2:172.21.15.105:6837/1798232582,v1:172.21.15.105:6839/1798232582] conn(0x559287229e40 0x55928762adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.151+0000 7f1e8e218740 4 client.4743 mds_command: new command op to 4241 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:15:54.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.151+0000 7f1e8e218740 1 -- 172.21.15.76:0/3973639635 --> [v2:172.21.15.105:6837/1798232582,v1:172.21.15.105:6839/1798232582] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55928757f350 con 0x559287229e40 2024-04-25T19:15:54.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.151+0000 7f1e8502e640 1 --2- 172.21.15.76:0/3973639635 >> [v2:172.21.15.105:6837/1798232582,v1:172.21.15.105:6839/1798232582] conn(0x559287229e40 0x55928762adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.152+0000 7f1e5ffff640 10 client.4743 ms_handle_connect on v2:172.21.15.105:6837/1798232582 2024-04-25T19:15:54.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.152+0000 7f1e8502e640 1 --2- 172.21.15.76:0/3973639635 >> [v2:172.21.15.105:6837/1798232582,v1:172.21.15.105:6839/1798232582] conn(0x559287229e40 0x55928762adf0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b75932b9a4c0a107 server_cookie=39337600de435d43 in_seq=0 out_seq=0 2024-04-25T19:15:54.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.153+0000 7f1e5ffff640 1 -- 172.21.15.76:0/3973639635 <== mds.2 v2:172.21.15.105:6837/1798232582 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55928757f350 con 0x559287229e40 2024-04-25T19:15:54.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.153+0000 7f1e5ffff640 10 client.4743 handle_command_reply: tid=0 2024-04-25T19:15:54.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.153+0000 7f1e8e218740 1 client.4743 shutdown 2024-04-25T19:15:54.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.153+0000 7f1e8e218740 1 -- 172.21.15.76:0/3973639635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55928726b270 msgr2=0x559287646820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.153+0000 7f1e8e218740 1 --2- 172.21.15.76:0/3973639635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55928726b270 0x559287646820 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009990 tx=0x7f1e7c004730 comp rx=0 tx=0).stop 2024-04-25T19:15:54.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.153+0000 7f1e8e218740 1 -- 172.21.15.76:0/3973639635 shutdown_connections 2024-04-25T19:15:54.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.153+0000 7f1e8e218740 1 --2- 172.21.15.76:0/3973639635 >> [v2:172.21.15.105:6837/1798232582,v1:172.21.15.105:6839/1798232582] conn(0x559287229e40 0x55928762adf0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.153+0000 7f1e8e218740 1 --2- 172.21.15.76:0/3973639635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55928726b270 0x559287646820 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.153+0000 7f1e8e218740 1 --2- 172.21.15.76:0/3973639635 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55928724a590 0x559287634760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.153+0000 7f1e8e218740 1 --2- 172.21.15.76:0/3973639635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592872423b0 0x55928764d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.153+0000 7f1e5ffff640 0 client.4743 ms_handle_reset on v2:172.21.15.105:6837/1798232582 2024-04-25T19:15:54.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.153+0000 7f1e8e218740 1 -- 172.21.15.76:0/3973639635 >> 172.21.15.76:0/3973639635 conn(0x55928725f3c0 msgr2=0x559287606b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:54.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.153+0000 7f1e8e218740 1 -- 172.21.15.76:0/3973639635 shutdown_connections 2024-04-25T19:15:54.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.153+0000 7f1e8e218740 1 -- 172.21.15.76:0/3973639635 wait complete. 2024-04-25T19:15:54.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.153+0000 7f1e8e218740 20 client.4743 trim_cache size 0 max 16384 2024-04-25T19:15:54.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.171+0000 7f1e8e218740 1 Processor -- start 2024-04-25T19:15:54.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.171+0000 7f1e8e218740 1 -- start start 2024-04-25T19:15:54.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.173+0000 7f1e8e218740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592872423b0 0x559287607880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.173+0000 7f1e8e218740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55928724a590 0x559287607dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.173+0000 7f1e8e218740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55928726b270 0x55928760bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.173+0000 7f1e8e218740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1e801ff680 con 0x55928724a590 2024-04-25T19:15:54.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.173+0000 7f1e8e218740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8015a1c0 con 0x5592872423b0 2024-04-25T19:15:54.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.173+0000 7f1e8e218740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x55928764e120 con 0x55928726b270 2024-04-25T19:15:54.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.173+0000 7f1e8482d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55928724a590 0x559287607dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.173+0000 7f1e8482d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55928724a590 0x559287607dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37898/0 (socket says 172.21.15.76:37898) 2024-04-25T19:15:54.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.173+0000 7f1e8482d640 1 -- 172.21.15.76:0/2634723641 learned_addr learned my addr 172.21.15.76:0/2634723641 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:54.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.173+0000 7f1e8582f640 1 --2- 172.21.15.76:0/2634723641 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55928726b270 0x55928760bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.173+0000 7f1e8502e640 1 --2- 172.21.15.76:0/2634723641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592872423b0 0x559287607880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.173+0000 7f1e8482d640 1 -- 172.21.15.76:0/2634723641 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55928726b270 msgr2=0x55928760bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.173+0000 7f1e8482d640 1 --2- 172.21.15.76:0/2634723641 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55928726b270 0x55928760bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.173+0000 7f1e8482d640 1 -- 172.21.15.76:0/2634723641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592872423b0 msgr2=0x559287607880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.173+0000 7f1e8482d640 1 --2- 172.21.15.76:0/2634723641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592872423b0 0x559287607880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.173+0000 7f1e8482d640 1 -- 172.21.15.76:0/2634723641 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55928760e010 con 0x55928724a590 2024-04-25T19:15:54.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.173+0000 7f1e8582f640 1 --2- 172.21.15.76:0/2634723641 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55928726b270 0x55928760bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:54.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.173+0000 7f1e8502e640 1 --2- 172.21.15.76:0/2634723641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592872423b0 0x559287607880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:54.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.173+0000 7f1e8482d640 1 --2- 172.21.15.76:0/2634723641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55928724a590 0x559287607dc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000f9e0 tx=0x7f1e70014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4aaa601cc4a6d36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:54.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.173+0000 7f1e6a7fc640 1 -- 172.21.15.76:0/2634723641 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70025030 con 0x55928724a590 2024-04-25T19:15:54.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.173+0000 7f1e6a7fc640 1 -- 172.21.15.76:0/2634723641 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e70026040 con 0x55928724a590 2024-04-25T19:15:54.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.173+0000 7f1e6a7fc640 1 -- 172.21.15.76:0/2634723641 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70007260 con 0x55928724a590 2024-04-25T19:15:54.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.174+0000 7f1e8e218740 1 -- 172.21.15.76:0/2634723641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55928724a590 msgr2=0x559287607dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.174+0000 7f1e8e218740 1 --2- 172.21.15.76:0/2634723641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55928724a590 0x559287607dc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000f9e0 tx=0x7f1e70014000 comp rx=0 tx=0).stop 2024-04-25T19:15:54.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.174+0000 7f1e8e218740 1 -- 172.21.15.76:0/2634723641 shutdown_connections 2024-04-25T19:15:54.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.174+0000 7f1e8e218740 1 --2- 172.21.15.76:0/2634723641 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55928726b270 0x55928760bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.174+0000 7f1e8e218740 1 --2- 172.21.15.76:0/2634723641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55928724a590 0x559287607dc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.174+0000 7f1e8e218740 1 --2- 172.21.15.76:0/2634723641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592872423b0 0x559287607880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.174+0000 7f1e8e218740 1 -- 172.21.15.76:0/2634723641 >> 172.21.15.76:0/2634723641 conn(0x55928725f3c0 msgr2=0x5592876465e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:54.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.174+0000 7f1e8e218740 1 -- 172.21.15.76:0/2634723641 shutdown_connections 2024-04-25T19:15:54.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.174+0000 7f1e8e218740 1 -- 172.21.15.76:0/2634723641 wait complete. 2024-04-25T19:15:54.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.175+0000 7f1e8e218740 1 -- start start 2024-04-25T19:15:54.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.175+0000 7f1e8e218740 1 Processor -- start 2024-04-25T19:15:54.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.176+0000 7f1e8e218740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592872423b0 0x559287631cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.176+0000 7f1e8e218740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55928724a590 0x5592876cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.176+0000 7f1e8e218740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55928726b270 0x5592876c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.176+0000 7f1e8e218740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5592876456f0 con 0x55928724a590 2024-04-25T19:15:54.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.176+0000 7f1e8e218740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x559287646d60 con 0x5592872423b0 2024-04-25T19:15:54.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.176+0000 7f1e8e218740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x559287645d80 con 0x55928726b270 2024-04-25T19:15:54.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.176+0000 7f1e8e218740 10 client.0 _ll_register_callbacks cb 0x7f1e7c0096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:15:54.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.176+0000 7f1e8482d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55928724a590 0x5592876cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.176+0000 7f1e8482d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55928724a590 0x5592876cf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37908/0 (socket says 172.21.15.76:37908) 2024-04-25T19:15:54.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.176+0000 7f1e8482d640 1 -- 172.21.15.76:0/828628246 learned_addr learned my addr 172.21.15.76:0/828628246 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:54.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.176+0000 7f1e8582f640 1 --2- 172.21.15.76:0/828628246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55928726b270 0x5592876c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.176+0000 7f1e8502e640 1 --2- 172.21.15.76:0/828628246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592872423b0 0x559287631cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.177+0000 7f1e8482d640 1 -- 172.21.15.76:0/828628246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55928726b270 msgr2=0x5592876c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.177+0000 7f1e8482d640 1 --2- 172.21.15.76:0/828628246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55928726b270 0x5592876c5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.177+0000 7f1e8482d640 1 -- 172.21.15.76:0/828628246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592872423b0 msgr2=0x559287631cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.177+0000 7f1e8482d640 1 --2- 172.21.15.76:0/828628246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592872423b0 0x559287631cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.177+0000 7f1e8482d640 1 -- 172.21.15.76:0/828628246 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592876cfb60 con 0x55928724a590 2024-04-25T19:15:54.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.177+0000 7f1e8482d640 1 --2- 172.21.15.76:0/828628246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55928724a590 0x5592876cf620 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1e70011010 tx=0x7f1e70001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f344071626424 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:54.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.177+0000 7f1e5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-04-25T19:15:54.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.177+0000 7f1e5ffff640 1 -- 172.21.15.76:0/828628246 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7003b060 con 0x55928724a590 2024-04-25T19:15:54.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.177+0000 7f1e5ffff640 1 -- 172.21.15.76:0/828628246 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1e70007430 con 0x55928724a590 2024-04-25T19:15:54.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.177+0000 7f1e8e218740 1 -- 172.21.15.76:0/828628246 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e70026040 con 0x55928724a590 2024-04-25T19:15:54.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.177+0000 7f1e5ffff640 1 -- 172.21.15.76:0/828628246 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f1e70025040 con 0x55928724a590 2024-04-25T19:15:54.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.177+0000 7f1e8e218740 10 client.4749 fetch_fsmap learned FSMap version 16 2024-04-25T19:15:54.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.177+0000 7f1e8e218740 1 -- 172.21.15.76:0/828628246 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x559287643b80 con 0x55928724a590 2024-04-25T19:15:54.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.177+0000 7f1e5ffff640 1 -- 172.21.15.76:0/828628246 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e70026040 con 0x55928724a590 2024-04-25T19:15:54.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.177+0000 7f1e5ffff640 1 -- 172.21.15.76:0/828628246 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 16) ==== 4784+0+0 (secure 0 0 0) 0x7f1e700067d0 con 0x55928724a590 2024-04-25T19:15:54.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.177+0000 7f1e8e218740 10 client.4749 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T19:15:54.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.177+0000 7f1e8e218740 10 client.4749 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T19:15:54.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.177+0000 7f1e8e218740 20 client.4749 populate_metadata read hostname 'smithi076' 2024-04-25T19:15:54.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.178+0000 7f1e8e218740 1 --2- 172.21.15.76:0/828628246 >> [v2:172.21.15.105:6837/1798232582,v1:172.21.15.105:6839/1798232582] conn(0x55928764a4e0 0x5592876d84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.178+0000 7f1e8e218740 4 client.4749 mds_command: new command op to 4241 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T19:15:54.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.178+0000 7f1e8e218740 1 -- 172.21.15.76:0/828628246 --> [v2:172.21.15.105:6837/1798232582,v1:172.21.15.105:6839/1798232582] -- command(tid 0: {"prefix": "status"}) -- 0x5592875791c0 con 0x55928764a4e0 2024-04-25T19:15:54.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.178+0000 7f1e8502e640 1 --2- 172.21.15.76:0/828628246 >> [v2:172.21.15.105:6837/1798232582,v1:172.21.15.105:6839/1798232582] conn(0x55928764a4e0 0x5592876d84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.178+0000 7f1e8502e640 1 --2- 172.21.15.76:0/828628246 >> [v2:172.21.15.105:6837/1798232582,v1:172.21.15.105:6839/1798232582] conn(0x55928764a4e0 0x5592876d84d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c1b7087b2842bbbf server_cookie=568f0d01b90327f7 in_seq=0 out_seq=0 2024-04-25T19:15:54.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.178+0000 7f1e5ffff640 10 client.4749 ms_handle_connect on v2:172.21.15.105:6837/1798232582 2024-04-25T19:15:54.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.178+0000 7f1e5ffff640 1 -- 172.21.15.76:0/828628246 <== mds.2 v2:172.21.15.105:6837/1798232582 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x5592875791c0 con 0x55928764a4e0 2024-04-25T19:15:54.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.178+0000 7f1e5ffff640 10 client.4749 handle_command_reply: tid=0 2024-04-25T19:15:54.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.178+0000 7f1e8e218740 1 client.4749 shutdown 2024-04-25T19:15:54.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.178+0000 7f1e8e218740 1 -- 172.21.15.76:0/828628246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55928724a590 msgr2=0x5592876cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.178+0000 7f1e8e218740 1 --2- 172.21.15.76:0/828628246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55928724a590 0x5592876cf620 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1e70011010 tx=0x7f1e70001080 comp rx=0 tx=0).stop 2024-04-25T19:15:54.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.178+0000 7f1e8e218740 1 -- 172.21.15.76:0/828628246 shutdown_connections 2024-04-25T19:15:54.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.178+0000 7f1e8e218740 1 --2- 172.21.15.76:0/828628246 >> [v2:172.21.15.105:6837/1798232582,v1:172.21.15.105:6839/1798232582] conn(0x55928764a4e0 0x5592876d84d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.178+0000 7f1e8e218740 1 --2- 172.21.15.76:0/828628246 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x55928726b270 0x5592876c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.178+0000 7f1e5ffff640 0 client.4749 ms_handle_reset on v2:172.21.15.105:6837/1798232582 2024-04-25T19:15:54.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.178+0000 7f1e8e218740 1 --2- 172.21.15.76:0/828628246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55928724a590 0x5592876cf620 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.178+0000 7f1e8e218740 1 --2- 172.21.15.76:0/828628246 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5592872423b0 0x559287631cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.179+0000 7f1e8e218740 1 -- 172.21.15.76:0/828628246 >> 172.21.15.76:0/828628246 conn(0x55928725f3c0 msgr2=0x55928760bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:54.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.179+0000 7f1e8e218740 1 -- 172.21.15.76:0/828628246 shutdown_connections 2024-04-25T19:15:54.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.179+0000 7f1e8e218740 1 -- 172.21.15.76:0/828628246 wait complete. 2024-04-25T19:15:54.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.179+0000 7f1e8e218740 20 client.4749 trim_cache size 0 max 16384 2024-04-25T19:15:54.190 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:15:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:15:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 2, 2024-04-25T19:15:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4241, 2024-04-25T19:15:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-04-25T19:15:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-04-25T19:15:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-04-25T19:15:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 23.719791506, 2024-04-25T19:15:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 16, 2024-04-25T19:15:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 21, 2024-04-25T19:15:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 21, 2024-04-25T19:15:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 41.800429023 2024-04-25T19:15:54.191 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:15:54.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.179+0000 7f1e87292640 1 -- 172.21.15.76:0/3106091277 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1e640744e0 msgr2=0x7f1e640769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.179+0000 7f1e87292640 1 --2- 172.21.15.76:0/3106091277 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1e640744e0 0x7f1e640769f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1e8015ac50 tx=0x7f1e70009040 comp rx=0 tx=0).stop 2024-04-25T19:15:54.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.179+0000 7f1e87292640 1 -- 172.21.15.76:0/3106091277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e80072ba0 msgr2=0x7f1e80151ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.179+0000 7f1e87292640 1 --2- 172.21.15.76:0/3106091277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e80072ba0 0x7f1e80151ba0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1e78009ec0 tx=0x7f1e7800c490 comp rx=0 tx=0).stop 2024-04-25T19:15:54.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.179+0000 7f1e87292640 1 -- 172.21.15.76:0/3106091277 shutdown_connections 2024-04-25T19:15:54.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.179+0000 7f1e87292640 1 --2- 172.21.15.76:0/3106091277 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1e640744e0 0x7f1e640769f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.179+0000 7f1e87292640 1 --2- 172.21.15.76:0/3106091277 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1e800cb880 0x7f1e80150880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.179+0000 7f1e87292640 1 --2- 172.21.15.76:0/3106091277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e800cad30 0x7f1e8015f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.179+0000 7f1e87292640 1 --2- 172.21.15.76:0/3106091277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e80072ba0 0x7f1e80151ba0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.179+0000 7f1e87292640 1 -- 172.21.15.76:0/3106091277 >> 172.21.15.76:0/3106091277 conn(0x7f1e80070b00 msgr2=0x7f1e80155610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:54.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.179+0000 7f1e87292640 1 -- 172.21.15.76:0/3106091277 shutdown_connections 2024-04-25T19:15:54.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.179+0000 7f1e87292640 1 -- 172.21.15.76:0/3106091277 wait complete. 2024-04-25T19:15:54.194 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T19:15:54.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.301+0000 7fa8b6dce640 1 Processor -- start 2024-04-25T19:15:54.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.301+0000 7fa8b6dce640 1 -- start start 2024-04-25T19:15:54.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.301+0000 7fa8b6dce640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8b00d0810 0x7fa8b006f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.301+0000 7fa8b6dce640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8b015ce80 con 0x7fa8b006e870 2024-04-25T19:15:54.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.301+0000 7fa8b6dce640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8b015d160 con 0x7fa8b00cea70 2024-04-25T19:15:54.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.301+0000 7fa8b6dce640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa8b015d440 con 0x7fa8b00d0810 2024-04-25T19:15:54.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.301+0000 7fa8b536b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa8b006e870 0x7fa8b01674f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56736/0 (socket says 172.21.15.76:56736) 2024-04-25T19:15:54.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.301+0000 7fa8b536b640 1 -- 172.21.15.76:0/1142117108 learned_addr learned my addr 172.21.15.76:0/1142117108 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:54.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.301+0000 7fa8af7fe640 1 -- 172.21.15.76:0/1142117108 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3446738048 0 0) 0x7fa8b015ce80 con 0x7fa8b006e870 2024-04-25T19:15:54.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.302+0000 7fa8b4b6a640 1 --2- 172.21.15.76:0/1142117108 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8b00d0810 0x7fa8b006f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.302+0000 7fa8af7fe640 1 -- 172.21.15.76:0/1142117108 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8a0003520 con 0x7fa8b006e870 2024-04-25T19:15:54.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.302+0000 7fa8af7fe640 1 -- 172.21.15.76:0/1142117108 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1806221176 0 0) 0x7fa8a0003520 con 0x7fa8b006e870 2024-04-25T19:15:54.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.302+0000 7fa8af7fe640 1 -- 172.21.15.76:0/1142117108 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8b00d0810 msgr2=0x7fa8b006f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.302+0000 7fa8af7fe640 1 --2- 172.21.15.76:0/1142117108 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8b00d0810 0x7fa8b006f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.302+0000 7fa8af7fe640 1 -- 172.21.15.76:0/1142117108 >> v1:172.21.15.105:6789/0 conn(0x7fa8b00cea70 legacy=0x7fa8b006e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.302+0000 7fa8af7fe640 1 -- 172.21.15.76:0/1142117108 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b015d720 con 0x7fa8b006e870 2024-04-25T19:15:54.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.302+0000 7fa8af7fe640 1 -- 172.21.15.76:0/1142117108 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7fa8a80019f0 con 0x7fa8b006e870 2024-04-25T19:15:54.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.302+0000 7fa8af7fe640 1 -- 172.21.15.76:0/1142117108 >> v1:172.21.15.76:6789/0 conn(0x7fa8b006e870 legacy=0x7fa8b01674f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.302+0000 7fa8af7fe640 1 --2- 172.21.15.76:0/1142117108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a00039e0 0x7fa8a0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.302+0000 7fa8af7fe640 1 --2- 172.21.15.76:0/1142117108 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8b00d0810 0x7fa8a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.302+0000 7fa8af7fe640 1 --2- 172.21.15.76:0/1142117108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8a0004d40 0x7fa8a0005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.302+0000 7fa8af7fe640 1 -- 172.21.15.76:0/1142117108 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b015d440 con 0x7fa8a00039e0 2024-04-25T19:15:54.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.302+0000 7fa8af7fe640 1 -- 172.21.15.76:0/1142117108 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a000ba70 con 0x7fa8a0004d40 2024-04-25T19:15:54.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.302+0000 7fa8af7fe640 1 -- 172.21.15.76:0/1142117108 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa8a000bc70 con 0x7fa8b00d0810 2024-04-25T19:15:54.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.302+0000 7fa8b4b6a640 1 --2- 172.21.15.76:0/1142117108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a00039e0 0x7fa8a0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.302+0000 7fa8b536b640 1 --2- 172.21.15.76:0/1142117108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8a0004d40 0x7fa8a0005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.302+0000 7fa8affff640 1 --2- 172.21.15.76:0/1142117108 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8b00d0810 0x7fa8a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.302+0000 7fa8b4b6a640 1 -- 172.21.15.76:0/1142117108 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8b00d0810 msgr2=0x7fa8a0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.302+0000 7fa8b4b6a640 1 --2- 172.21.15.76:0/1142117108 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8b00d0810 0x7fa8a0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.302+0000 7fa8b4b6a640 1 -- 172.21.15.76:0/1142117108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8a0004d40 msgr2=0x7fa8a0005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.302+0000 7fa8b4b6a640 1 --2- 172.21.15.76:0/1142117108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8a0004d40 0x7fa8a0005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.302+0000 7fa8b4b6a640 1 -- 172.21.15.76:0/1142117108 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a0003520 con 0x7fa8a00039e0 2024-04-25T19:15:54.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.302+0000 7fa8affff640 1 --2- 172.21.15.76:0/1142117108 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8b00d0810 0x7fa8a0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:54.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.303+0000 7fa8b4b6a640 1 --2- 172.21.15.76:0/1142117108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a00039e0 0x7fa8a0003e10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4009ac0 tx=0x7fa8a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931590aa0d45318e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:54.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.303+0000 7fa8af7fe640 1 -- 172.21.15.76:0/1142117108 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4036070 con 0x7fa8a00039e0 2024-04-25T19:15:54.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.303+0000 7fa8af7fe640 1 -- 172.21.15.76:0/1142117108 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8a4031400 con 0x7fa8a00039e0 2024-04-25T19:15:54.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.303+0000 7fa8af7fe640 1 -- 172.21.15.76:0/1142117108 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4009690 con 0x7fa8a00039e0 2024-04-25T19:15:54.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.304+0000 7fa8b6dce640 1 -- 172.21.15.76:0/1142117108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a00039e0 msgr2=0x7fa8a0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.304+0000 7fa8b6dce640 1 --2- 172.21.15.76:0/1142117108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a00039e0 0x7fa8a0003e10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4009ac0 tx=0x7fa8a0005830 comp rx=0 tx=0).stop 2024-04-25T19:15:54.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.304+0000 7fa8b536b640 1 -- 172.21.15.76:0/1142117108 reap_dead start 2024-04-25T19:15:54.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.304+0000 7fa8b6dce640 1 -- 172.21.15.76:0/1142117108 shutdown_connections 2024-04-25T19:15:54.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.304+0000 7fa8b6dce640 1 -- 172.21.15.76:0/1142117108 >> 172.21.15.76:0/1142117108 conn(0x7fa8b00b3c00 msgr2=0x7fa8b00b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:54.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.304+0000 7fa8b6dce640 1 -- 172.21.15.76:0/1142117108 shutdown_connections 2024-04-25T19:15:54.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.304+0000 7fa8b6dce640 1 -- 172.21.15.76:0/1142117108 wait complete. 2024-04-25T19:15:54.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.306+0000 7fa8b6dce640 1 Processor -- start 2024-04-25T19:15:54.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.306+0000 7fa8b6dce640 1 -- start start 2024-04-25T19:15:54.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.307+0000 7fa8b6dce640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a00039e0 0x7fa8b0159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.307+0000 7fa8b6dce640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8b01f27b0 0x7fa8b0153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.307+0000 7fa8b6dce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8b0160bd0 0x7fa8b01500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.307+0000 7fa8b6dce640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b015cb70 con 0x7fa8a00039e0 2024-04-25T19:15:54.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.307+0000 7fa8b6dce640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b006dc40 con 0x7fa8b0160bd0 2024-04-25T19:15:54.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.307+0000 7fa8b6dce640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa8b01570e0 con 0x7fa8b01f27b0 2024-04-25T19:15:54.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.307+0000 7fa8b4b6a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a00039e0 0x7fa8b0159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.307+0000 7fa8b4b6a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a00039e0 0x7fa8b0159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37920/0 (socket says 172.21.15.76:37920) 2024-04-25T19:15:54.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.307+0000 7fa8b4b6a640 1 -- 172.21.15.76:0/3429043810 learned_addr learned my addr 172.21.15.76:0/3429043810 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:54.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.307+0000 7fa8affff640 1 --2- 172.21.15.76:0/3429043810 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8b01f27b0 0x7fa8b0153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.307+0000 7fa8b536b640 1 --2- 172.21.15.76:0/3429043810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8b0160bd0 0x7fa8b01500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.307+0000 7fa8b4b6a640 1 -- 172.21.15.76:0/3429043810 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8b01f27b0 msgr2=0x7fa8b0153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.307+0000 7fa8b4b6a640 1 --2- 172.21.15.76:0/3429043810 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8b01f27b0 0x7fa8b0153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.307+0000 7fa8b4b6a640 1 -- 172.21.15.76:0/3429043810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8b0160bd0 msgr2=0x7fa8b01500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.307+0000 7fa8b4b6a640 1 --2- 172.21.15.76:0/3429043810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8b0160bd0 0x7fa8b01500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.307+0000 7fa8b4b6a640 1 -- 172.21.15.76:0/3429043810 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b00cdd00 con 0x7fa8a00039e0 2024-04-25T19:15:54.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.308+0000 7fa8b4b6a640 1 --2- 172.21.15.76:0/3429043810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a00039e0 0x7fa8b0159540 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4009ac0 tx=0x7fa8a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:54.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.308+0000 7fa8ad7fa640 1 -- 172.21.15.76:0/3429043810 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4040020 con 0x7fa8a00039e0 2024-04-25T19:15:54.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.308+0000 7fa8ad7fa640 1 -- 172.21.15.76:0/3429043810 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8a4003bc0 con 0x7fa8a00039e0 2024-04-25T19:15:54.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.308+0000 7fa8b6dce640 1 -- 172.21.15.76:0/3429043810 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8b015a1e0 con 0x7fa8a00039e0 2024-04-25T19:15:54.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.308+0000 7fa8ad7fa640 1 -- 172.21.15.76:0/3429043810 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4036040 con 0x7fa8a00039e0 2024-04-25T19:15:54.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.308+0000 7fa8b6dce640 1 -- 172.21.15.76:0/3429043810 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8b015faa0 con 0x7fa8a00039e0 2024-04-25T19:15:54.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.310+0000 7fa8ad7fa640 1 -- 172.21.15.76:0/3429043810 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa8a4043020 con 0x7fa8a00039e0 2024-04-25T19:15:54.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.310+0000 7fa8ad7fa640 1 --2- 172.21.15.76:0/3429043810 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa8940744e0 0x7fa8940769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.310+0000 7fa8ad7fa640 1 -- 172.21.15.76:0/3429043810 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa8a40b5220 con 0x7fa8a00039e0 2024-04-25T19:15:54.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.310+0000 7fa8affff640 1 --2- 172.21.15.76:0/3429043810 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa8940744e0 0x7fa8940769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.311+0000 7fa8affff640 1 --2- 172.21.15.76:0/3429043810 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa8940744e0 0x7fa8940769f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa8b01f38e0 tx=0x7fa89c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:54.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.311+0000 7fa8bdc80740 1 Processor -- start 2024-04-25T19:15:54.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.311+0000 7fa8bdc80740 1 -- start start 2024-04-25T19:15:54.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.312+0000 7fa8bdc80740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x563fb0e00270 0x563fb11a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.312+0000 7fa8bdc80740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0dd73b0 0x563fb119c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.312+0000 7fa8bdc80740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fb0ddf590 0x563fb119cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.312+0000 7fa8bdc80740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563fb1106740 con 0x563fb0dd73b0 2024-04-25T19:15:54.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.312+0000 7fa8bdc80740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563fb10a23a0 con 0x563fb0ddf590 2024-04-25T19:15:54.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.312+0000 7fa8bdc80740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x563fb1102700 con 0x563fb0e00270 2024-04-25T19:15:54.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.312+0000 7fa8affff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0dd73b0 0x563fb119c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.312+0000 7fa8affff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0dd73b0 0x563fb119c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37922/0 (socket says 172.21.15.76:37922) 2024-04-25T19:15:54.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.312+0000 7fa8affff640 1 -- 172.21.15.76:0/3112228855 learned_addr learned my addr 172.21.15.76:0/3112228855 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:54.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.313+0000 7fa8affff640 1 -- 172.21.15.76:0/3112228855 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x563fb0e00270 msgr2=0x563fb11a25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.313+0000 7fa8affff640 1 --2- 172.21.15.76:0/3112228855 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x563fb0e00270 0x563fb11a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.313+0000 7fa8affff640 1 -- 172.21.15.76:0/3112228855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fb0ddf590 msgr2=0x563fb119cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.313+0000 7fa8affff640 1 --2- 172.21.15.76:0/3112228855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fb0ddf590 0x563fb119cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.313+0000 7fa8affff640 1 -- 172.21.15.76:0/3112228855 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fb10be750 con 0x563fb0dd73b0 2024-04-25T19:15:54.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.313+0000 7fa8affff640 1 --2- 172.21.15.76:0/3112228855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0dd73b0 0x563fb119c880 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa89c011a50 tx=0x7fa89c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6df1a1e46de9536 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:54.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.313+0000 7fa89a7fc640 1 -- 172.21.15.76:0/3112228855 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89c015b40 con 0x563fb0dd73b0 2024-04-25T19:15:54.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.313+0000 7fa89a7fc640 1 -- 172.21.15.76:0/3112228855 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa89c015ce0 con 0x563fb0dd73b0 2024-04-25T19:15:54.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.313+0000 7fa89a7fc640 1 -- 172.21.15.76:0/3112228855 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89c007020 con 0x563fb0dd73b0 2024-04-25T19:15:54.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.314+0000 7fa8bdc80740 1 -- 172.21.15.76:0/3112228855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0dd73b0 msgr2=0x563fb119c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.314+0000 7fa8bdc80740 1 --2- 172.21.15.76:0/3112228855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0dd73b0 0x563fb119c880 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa89c011a50 tx=0x7fa89c014000 comp rx=0 tx=0).stop 2024-04-25T19:15:54.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.314+0000 7fa8bdc80740 1 -- 172.21.15.76:0/3112228855 shutdown_connections 2024-04-25T19:15:54.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.314+0000 7fa8bdc80740 1 --2- 172.21.15.76:0/3112228855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fb0ddf590 0x563fb119cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.314+0000 7fa8bdc80740 1 --2- 172.21.15.76:0/3112228855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0dd73b0 0x563fb119c880 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.314+0000 7fa8bdc80740 1 --2- 172.21.15.76:0/3112228855 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x563fb0e00270 0x563fb11a25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.314+0000 7fa8bdc80740 1 -- 172.21.15.76:0/3112228855 >> 172.21.15.76:0/3112228855 conn(0x563fb0df43c0 msgr2=0x563fb11a0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:54.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.314+0000 7fa8bdc80740 1 -- 172.21.15.76:0/3112228855 shutdown_connections 2024-04-25T19:15:54.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.314+0000 7fa8bdc80740 1 -- 172.21.15.76:0/3112228855 wait complete. 2024-04-25T19:15:54.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.315+0000 7fa8bdc80740 1 -- start start 2024-04-25T19:15:54.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.315+0000 7fa8bdc80740 1 Processor -- start 2024-04-25T19:15:54.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.316+0000 7fa8bdc80740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0dd73b0 0x563fb11e2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.316+0000 7fa8bdc80740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fb0ddf590 0x563fb11bf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.316+0000 7fa8bdc80740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x563fb0e00270 0x563fb11c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.316+0000 7fa8bdc80740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563fb11e0d00 con 0x563fb0dd73b0 2024-04-25T19:15:54.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.316+0000 7fa8bdc80740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563fb108dfe0 con 0x563fb0ddf590 2024-04-25T19:15:54.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.316+0000 7fa8bdc80740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x563fb11a0be0 con 0x563fb0e00270 2024-04-25T19:15:54.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.316+0000 7fa8b4b6a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0dd73b0 0x563fb11e2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.316+0000 7fa8bdc80740 10 client.0 _ll_register_callbacks cb 0x563fb10a4150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:15:54.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.316+0000 7fa8b4b6a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0dd73b0 0x563fb11e2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37934/0 (socket says 172.21.15.76:37934) 2024-04-25T19:15:54.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.316+0000 7fa8b4b6a640 1 -- 172.21.15.76:0/1416674527 learned_addr learned my addr 172.21.15.76:0/1416674527 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:54.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.316+0000 7fa8affff640 1 --2- 172.21.15.76:0/1416674527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fb0ddf590 0x563fb11bf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.316+0000 7fa8b536b640 1 --2- 172.21.15.76:0/1416674527 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x563fb0e00270 0x563fb11c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.316+0000 7fa8b4b6a640 1 -- 172.21.15.76:0/1416674527 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x563fb0e00270 msgr2=0x563fb11c32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.316+0000 7fa8b4b6a640 1 --2- 172.21.15.76:0/1416674527 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x563fb0e00270 0x563fb11c32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.316+0000 7fa8b4b6a640 1 -- 172.21.15.76:0/1416674527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fb0ddf590 msgr2=0x563fb11bf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.316+0000 7fa8b4b6a640 1 --2- 172.21.15.76:0/1416674527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fb0ddf590 0x563fb11bf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.316+0000 7fa8b4b6a640 1 -- 172.21.15.76:0/1416674527 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fb0e182d0 con 0x563fb0dd73b0 2024-04-25T19:15:54.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.316+0000 7fa8affff640 1 --2- 172.21.15.76:0/1416674527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fb0ddf590 0x563fb11bf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:15:54.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.316+0000 7fa8b536b640 1 --2- 172.21.15.76:0/1416674527 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x563fb0e00270 0x563fb11c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:15:54.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.317+0000 7fa8b4b6a640 1 --2- 172.21.15.76:0/1416674527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0dd73b0 0x563fb11e2550 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4034040 tx=0x7fa8a407ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc786d48baf7bf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:54.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.317+0000 7fa88f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-04-25T19:15:54.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.317+0000 7fa88f7fe640 1 -- 172.21.15.76:0/1416674527 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a407f360 con 0x563fb0dd73b0 2024-04-25T19:15:54.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.317+0000 7fa88f7fe640 1 -- 172.21.15.76:0/1416674527 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa8a4083170 con 0x563fb0dd73b0 2024-04-25T19:15:54.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.317+0000 7fa8bdc80740 1 -- 172.21.15.76:0/1416674527 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa89c015ce0 con 0x563fb0dd73b0 2024-04-25T19:15:54.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.317+0000 7fa88f7fe640 1 -- 172.21.15.76:0/1416674527 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fa8a4083660 con 0x563fb0dd73b0 2024-04-25T19:15:54.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.317+0000 7fa8bdc80740 10 client.4761 fetch_fsmap learned FSMap version 16 2024-04-25T19:15:54.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.317+0000 7fa8bdc80740 1 -- 172.21.15.76:0/1416674527 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x563fb0ee26b0 con 0x563fb0dd73b0 2024-04-25T19:15:54.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.317+0000 7fa88f7fe640 1 -- 172.21.15.76:0/1416674527 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa89c015ce0 con 0x563fb0dd73b0 2024-04-25T19:15:54.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.317+0000 7fa88f7fe640 1 -- 172.21.15.76:0/1416674527 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 16) ==== 4784+0+0 (secure 0 0 0) 0x7fa8a40b3b70 con 0x563fb0dd73b0 2024-04-25T19:15:54.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.317+0000 7fa8bdc80740 10 client.4761 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T19:15:54.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.317+0000 7fa8bdc80740 10 client.4761 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T19:15:54.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.317+0000 7fa8bdc80740 20 client.4761 populate_metadata read hostname 'smithi076' 2024-04-25T19:15:54.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.317+0000 7fa8bdc80740 1 --2- 172.21.15.76:0/1416674527 >> [v2:172.21.15.76:6832/2101494685,v1:172.21.15.76:6833/2101494685] conn(0x563fb0dbee40 0x563fb11f6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.317+0000 7fa8bdc80740 4 client.4761 mds_command: new command op to 4278 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:15:54.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.317+0000 7fa8bdc80740 1 -- 172.21.15.76:0/1416674527 --> [v2:172.21.15.76:6832/2101494685,v1:172.21.15.76:6833/2101494685] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fb1114350 con 0x563fb0dbee40 2024-04-25T19:15:54.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.317+0000 7fa8affff640 1 --2- 172.21.15.76:0/1416674527 >> [v2:172.21.15.76:6832/2101494685,v1:172.21.15.76:6833/2101494685] conn(0x563fb0dbee40 0x563fb11f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.318+0000 7fa88f7fe640 10 client.4761 ms_handle_connect on v2:172.21.15.76:6832/2101494685 2024-04-25T19:15:54.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.318+0000 7fa8affff640 1 --2- 172.21.15.76:0/1416674527 >> [v2:172.21.15.76:6832/2101494685,v1:172.21.15.76:6833/2101494685] conn(0x563fb0dbee40 0x563fb11f6880 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2a692ad1935043dc server_cookie=e9d9e9aa1b934b26 in_seq=0 out_seq=0 2024-04-25T19:15:54.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.318+0000 7fa88f7fe640 1 -- 172.21.15.76:0/1416674527 <== mds.1 v2:172.21.15.76:6832/2101494685 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563fb1114350 con 0x563fb0dbee40 2024-04-25T19:15:54.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.318+0000 7fa88f7fe640 10 client.4761 handle_command_reply: tid=0 2024-04-25T19:15:54.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.318+0000 7fa8bdc80740 1 client.4761 shutdown 2024-04-25T19:15:54.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.318+0000 7fa8bdc80740 1 -- 172.21.15.76:0/1416674527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0dd73b0 msgr2=0x563fb11e2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.318+0000 7fa8bdc80740 1 --2- 172.21.15.76:0/1416674527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0dd73b0 0x563fb11e2550 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4034040 tx=0x7fa8a407ee70 comp rx=0 tx=0).stop 2024-04-25T19:15:54.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.319+0000 7fa8bdc80740 1 -- 172.21.15.76:0/1416674527 shutdown_connections 2024-04-25T19:15:54.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.319+0000 7fa8bdc80740 1 --2- 172.21.15.76:0/1416674527 >> [v2:172.21.15.76:6832/2101494685,v1:172.21.15.76:6833/2101494685] conn(0x563fb0dbee40 0x563fb11f6880 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.319+0000 7fa8bdc80740 1 --2- 172.21.15.76:0/1416674527 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x563fb0e00270 0x563fb11c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.319+0000 7fa8bdc80740 1 --2- 172.21.15.76:0/1416674527 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fb0ddf590 0x563fb11bf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.319+0000 7fa88f7fe640 0 client.4761 ms_handle_reset on v2:172.21.15.76:6832/2101494685 2024-04-25T19:15:54.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.319+0000 7fa8bdc80740 1 --2- 172.21.15.76:0/1416674527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0dd73b0 0x563fb11e2550 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.319+0000 7fa8bdc80740 1 -- 172.21.15.76:0/1416674527 >> 172.21.15.76:0/1416674527 conn(0x563fb0df43c0 msgr2=0x563fb11dc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:54.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.319+0000 7fa8bdc80740 1 -- 172.21.15.76:0/1416674527 shutdown_connections 2024-04-25T19:15:54.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.319+0000 7fa8bdc80740 1 -- 172.21.15.76:0/1416674527 wait complete. 2024-04-25T19:15:54.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.319+0000 7fa8bdc80740 20 client.4761 trim_cache size 0 max 16384 2024-04-25T19:15:54.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.337+0000 7fa8bdc80740 1 Processor -- start 2024-04-25T19:15:54.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.337+0000 7fa8bdc80740 1 -- start start 2024-04-25T19:15:54.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.339+0000 7fa8bdc80740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fb0dd73b0 0x563fb1206ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.339+0000 7fa8bdc80740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x563fb0ddf590 0x563fb11a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.339+0000 7fa8bdc80740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0e00270 0x563fb11a1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.339+0000 7fa8bdc80740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b01f66b0 con 0x563fb0e00270 2024-04-25T19:15:54.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.339+0000 7fa8bdc80740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563fb11e0d00 con 0x563fb0dd73b0 2024-04-25T19:15:54.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.339+0000 7fa8bdc80740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x563fb11bf020 con 0x563fb0ddf590 2024-04-25T19:15:54.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.339+0000 7fa8b536b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0e00270 0x563fb11a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.339+0000 7fa8b536b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0e00270 0x563fb11a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37948/0 (socket says 172.21.15.76:37948) 2024-04-25T19:15:54.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.339+0000 7fa8b536b640 1 -- 172.21.15.76:0/2426858492 learned_addr learned my addr 172.21.15.76:0/2426858492 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:54.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.339+0000 7fa8b536b640 1 -- 172.21.15.76:0/2426858492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x563fb0ddf590 msgr2=0x563fb11a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.339+0000 7fa8affff640 1 --2- 172.21.15.76:0/2426858492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x563fb0ddf590 0x563fb11a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.339+0000 7fa8b4b6a640 1 --2- 172.21.15.76:0/2426858492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fb0dd73b0 0x563fb1206ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.339+0000 7fa8b536b640 1 --2- 172.21.15.76:0/2426858492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x563fb0ddf590 0x563fb11a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.339+0000 7fa8b536b640 1 -- 172.21.15.76:0/2426858492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fb0dd73b0 msgr2=0x563fb1206ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.339+0000 7fa8b536b640 1 --2- 172.21.15.76:0/2426858492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fb0dd73b0 0x563fb1206ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.339+0000 7fa8b536b640 1 -- 172.21.15.76:0/2426858492 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fb11a1fa0 con 0x563fb0e00270 2024-04-25T19:15:54.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.339+0000 7fa8affff640 1 --2- 172.21.15.76:0/2426858492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x563fb0ddf590 0x563fb11a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:15:54.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.339+0000 7fa8b4b6a640 1 --2- 172.21.15.76:0/2426858492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fb0dd73b0 0x563fb1206ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:15:54.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.339+0000 7fa8b536b640 1 --2- 172.21.15.76:0/2426858492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0e00270 0x563fb11a1230 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa8b008f740 tx=0x7fa8a8006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6548fb75a12e85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:54.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.339+0000 7fa899ffb640 1 -- 172.21.15.76:0/2426858492 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a8017460 con 0x563fb0e00270 2024-04-25T19:15:54.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.339+0000 7fa899ffb640 1 -- 172.21.15.76:0/2426858492 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8a8017600 con 0x563fb0e00270 2024-04-25T19:15:54.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.339+0000 7fa899ffb640 1 -- 172.21.15.76:0/2426858492 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a80179b0 con 0x563fb0e00270 2024-04-25T19:15:54.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.340+0000 7fa8bdc80740 1 -- 172.21.15.76:0/2426858492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0e00270 msgr2=0x563fb11a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.340+0000 7fa8bdc80740 1 --2- 172.21.15.76:0/2426858492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0e00270 0x563fb11a1230 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa8b008f740 tx=0x7fa8a8006040 comp rx=0 tx=0).stop 2024-04-25T19:15:54.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.340+0000 7fa8bdc80740 1 -- 172.21.15.76:0/2426858492 shutdown_connections 2024-04-25T19:15:54.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.340+0000 7fa8bdc80740 1 --2- 172.21.15.76:0/2426858492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0e00270 0x563fb11a1230 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.340+0000 7fa8bdc80740 1 --2- 172.21.15.76:0/2426858492 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x563fb0ddf590 0x563fb11a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.340+0000 7fa8bdc80740 1 --2- 172.21.15.76:0/2426858492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fb0dd73b0 0x563fb1206ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.340+0000 7fa8bdc80740 1 -- 172.21.15.76:0/2426858492 >> 172.21.15.76:0/2426858492 conn(0x563fb0df43c0 msgr2=0x563fb119c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:54.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.340+0000 7fa8bdc80740 1 -- 172.21.15.76:0/2426858492 shutdown_connections 2024-04-25T19:15:54.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.340+0000 7fa8bdc80740 1 -- 172.21.15.76:0/2426858492 wait complete. 2024-04-25T19:15:54.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.341+0000 7fa8bdc80740 1 -- start start 2024-04-25T19:15:54.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.341+0000 7fa8bdc80740 1 Processor -- start 2024-04-25T19:15:54.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.342+0000 7fa8bdc80740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fb0dd73b0 0x563fb11bfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.342+0000 7fa8bdc80740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0ddf590 0x563fb11c9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.342+0000 7fa8bdc80740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x563fb0e00270 0x563fb11bae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.342+0000 7fa8bdc80740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563fb11bdeb0 con 0x563fb0ddf590 2024-04-25T19:15:54.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.342+0000 7fa8bdc80740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x563fb11c3a40 con 0x563fb0dd73b0 2024-04-25T19:15:54.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.342+0000 7fa8bdc80740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x563fb11be550 con 0x563fb0e00270 2024-04-25T19:15:54.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.342+0000 7fa8bdc80740 10 client.0 _ll_register_callbacks cb 0x7fa8a407b9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:15:54.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.342+0000 7fa8affff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0ddf590 0x563fb11c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.342+0000 7fa8affff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0ddf590 0x563fb11c9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37952/0 (socket says 172.21.15.76:37952) 2024-04-25T19:15:54.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.342+0000 7fa8affff640 1 -- 172.21.15.76:0/410036146 learned_addr learned my addr 172.21.15.76:0/410036146 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:54.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.343+0000 7fa8b4b6a640 1 --2- 172.21.15.76:0/410036146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fb0dd73b0 0x563fb11bfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.343+0000 7fa8b536b640 1 --2- 172.21.15.76:0/410036146 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x563fb0e00270 0x563fb11bae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.343+0000 7fa8affff640 1 -- 172.21.15.76:0/410036146 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x563fb0e00270 msgr2=0x563fb11bae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.343+0000 7fa8affff640 1 --2- 172.21.15.76:0/410036146 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x563fb0e00270 0x563fb11bae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.343+0000 7fa8affff640 1 -- 172.21.15.76:0/410036146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fb0dd73b0 msgr2=0x563fb11bfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.343+0000 7fa8affff640 1 --2- 172.21.15.76:0/410036146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fb0dd73b0 0x563fb11bfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.343+0000 7fa8affff640 1 -- 172.21.15.76:0/410036146 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fb1206700 con 0x563fb0ddf590 2024-04-25T19:15:54.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.343+0000 7fa8affff640 1 --2- 172.21.15.76:0/410036146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0ddf590 0x563fb11c9560 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa89c023040 tx=0x7fa89c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f0ef91a03f6f0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:54.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.343+0000 7fa88f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-04-25T19:15:54.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.343+0000 7fa88f7fe640 1 -- 172.21.15.76:0/410036146 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89c034540 con 0x563fb0ddf590 2024-04-25T19:15:54.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.343+0000 7fa88f7fe640 1 -- 172.21.15.76:0/410036146 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa89c026040 con 0x563fb0ddf590 2024-04-25T19:15:54.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.343+0000 7fa8bdc80740 1 -- 172.21.15.76:0/410036146 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8a8017600 con 0x563fb0ddf590 2024-04-25T19:15:54.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.343+0000 7fa88f7fe640 1 -- 172.21.15.76:0/410036146 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fa89c0346e0 con 0x563fb0ddf590 2024-04-25T19:15:54.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.343+0000 7fa8bdc80740 10 client.4767 fetch_fsmap learned FSMap version 16 2024-04-25T19:15:54.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.343+0000 7fa8bdc80740 1 -- 172.21.15.76:0/410036146 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x563fb11c4170 con 0x563fb0ddf590 2024-04-25T19:15:54.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.343+0000 7fa88f7fe640 1 -- 172.21.15.76:0/410036146 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8a8017600 con 0x563fb0ddf590 2024-04-25T19:15:54.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.343+0000 7fa88f7fe640 1 -- 172.21.15.76:0/410036146 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 16) ==== 4784+0+0 (secure 0 0 0) 0x7fa89c034ce0 con 0x563fb0ddf590 2024-04-25T19:15:54.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.343+0000 7fa8bdc80740 10 client.4767 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T19:15:54.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.343+0000 7fa8bdc80740 10 client.4767 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T19:15:54.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.343+0000 7fa8bdc80740 20 client.4767 populate_metadata read hostname 'smithi076' 2024-04-25T19:15:54.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.343+0000 7fa8bdc80740 1 --2- 172.21.15.76:0/410036146 >> [v2:172.21.15.76:6832/2101494685,v1:172.21.15.76:6833/2101494685] conn(0x563fb1259d40 0x563fb119bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.343+0000 7fa8bdc80740 4 client.4767 mds_command: new command op to 4278 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T19:15:54.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.343+0000 7fa8bdc80740 1 -- 172.21.15.76:0/410036146 --> [v2:172.21.15.76:6832/2101494685,v1:172.21.15.76:6833/2101494685] -- command(tid 0: {"prefix": "status"}) -- 0x563fb11e0b50 con 0x563fb1259d40 2024-04-25T19:15:54.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.344+0000 7fa8b4b6a640 1 --2- 172.21.15.76:0/410036146 >> [v2:172.21.15.76:6832/2101494685,v1:172.21.15.76:6833/2101494685] conn(0x563fb1259d40 0x563fb119bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.344+0000 7fa88f7fe640 10 client.4767 ms_handle_connect on v2:172.21.15.76:6832/2101494685 2024-04-25T19:15:54.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.344+0000 7fa8b4b6a640 1 --2- 172.21.15.76:0/410036146 >> [v2:172.21.15.76:6832/2101494685,v1:172.21.15.76:6833/2101494685] conn(0x563fb1259d40 0x563fb119bad0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5c7b30ec45a68f53 server_cookie=a7ea637e7aad377f in_seq=0 out_seq=0 2024-04-25T19:15:54.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.344+0000 7fa88f7fe640 1 -- 172.21.15.76:0/410036146 <== mds.1 v2:172.21.15.76:6832/2101494685 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x563fb11e0b50 con 0x563fb1259d40 2024-04-25T19:15:54.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.344+0000 7fa88f7fe640 10 client.4767 handle_command_reply: tid=0 2024-04-25T19:15:54.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.344+0000 7fa8bdc80740 1 client.4767 shutdown 2024-04-25T19:15:54.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.344+0000 7fa8bdc80740 1 -- 172.21.15.76:0/410036146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0ddf590 msgr2=0x563fb11c9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.344+0000 7fa8bdc80740 1 --2- 172.21.15.76:0/410036146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0ddf590 0x563fb11c9560 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa89c023040 tx=0x7fa89c017000 comp rx=0 tx=0).stop 2024-04-25T19:15:54.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.344+0000 7fa8bdc80740 1 -- 172.21.15.76:0/410036146 shutdown_connections 2024-04-25T19:15:54.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.344+0000 7fa8bdc80740 1 --2- 172.21.15.76:0/410036146 >> [v2:172.21.15.76:6832/2101494685,v1:172.21.15.76:6833/2101494685] conn(0x563fb1259d40 0x563fb119bad0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.344+0000 7fa8bdc80740 1 --2- 172.21.15.76:0/410036146 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x563fb0e00270 0x563fb11bae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.344+0000 7fa8bdc80740 1 --2- 172.21.15.76:0/410036146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563fb0ddf590 0x563fb11c9560 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.344+0000 7fa88f7fe640 0 client.4767 ms_handle_reset on v2:172.21.15.76:6832/2101494685 2024-04-25T19:15:54.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.344+0000 7fa8bdc80740 1 --2- 172.21.15.76:0/410036146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563fb0dd73b0 0x563fb11bfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.344+0000 7fa8bdc80740 1 -- 172.21.15.76:0/410036146 >> 172.21.15.76:0/410036146 conn(0x563fb0df43c0 msgr2=0x563fb119c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:54.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.344+0000 7fa8bdc80740 1 -- 172.21.15.76:0/410036146 shutdown_connections 2024-04-25T19:15:54.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.344+0000 7fa8bdc80740 1 -- 172.21.15.76:0/410036146 wait complete. 2024-04-25T19:15:54.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.345+0000 7fa8bdc80740 20 client.4767 trim_cache size 0 max 16384 2024-04-25T19:15:54.355 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:15:54.355 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:15:54.355 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 1, 2024-04-25T19:15:54.355 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4278, 2024-04-25T19:15:54.355 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-04-25T19:15:54.355 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-04-25T19:15:54.355 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-04-25T19:15:54.355 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 29.903628643000001, 2024-04-25T19:15:54.355 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 16, 2024-04-25T19:15:54.355 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 21, 2024-04-25T19:15:54.355 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 21, 2024-04-25T19:15:54.356 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 41.985293329000001 2024-04-25T19:15:54.356 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:15:54.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.345+0000 7fa8b6dce640 1 -- 172.21.15.76:0/3429043810 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa8940744e0 msgr2=0x7fa8940769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.345+0000 7fa8b6dce640 1 --2- 172.21.15.76:0/3429043810 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa8940744e0 0x7fa8940769f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa8b01f38e0 tx=0x7fa89c009040 comp rx=0 tx=0).stop 2024-04-25T19:15:54.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.345+0000 7fa8b6dce640 1 -- 172.21.15.76:0/3429043810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a00039e0 msgr2=0x7fa8b0159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.345+0000 7fa8b6dce640 1 --2- 172.21.15.76:0/3429043810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a00039e0 0x7fa8b0159540 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4009ac0 tx=0x7fa8a4004730 comp rx=0 tx=0).stop 2024-04-25T19:15:54.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.345+0000 7fa8b6dce640 1 -- 172.21.15.76:0/3429043810 shutdown_connections 2024-04-25T19:15:54.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.345+0000 7fa8b6dce640 1 --2- 172.21.15.76:0/3429043810 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa8940744e0 0x7fa8940769f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.345+0000 7fa8b6dce640 1 --2- 172.21.15.76:0/3429043810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8b0160bd0 0x7fa8b01500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.345+0000 7fa8b6dce640 1 --2- 172.21.15.76:0/3429043810 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa8b01f27b0 0x7fa8b0153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.345+0000 7fa8b6dce640 1 --2- 172.21.15.76:0/3429043810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a00039e0 0x7fa8b0159540 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.345+0000 7fa8b6dce640 1 -- 172.21.15.76:0/3429043810 >> 172.21.15.76:0/3429043810 conn(0x7fa8b00b3c00 msgr2=0x7fa8b015b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:54.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.345+0000 7fa8b6dce640 1 -- 172.21.15.76:0/3429043810 shutdown_connections 2024-04-25T19:15:54.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.345+0000 7fa8b6dce640 1 -- 172.21.15.76:0/3429043810 wait complete. 2024-04-25T19:15:54.359 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-25T19:15:54.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.465+0000 7f95e0ae7640 1 Processor -- start 2024-04-25T19:15:54.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.465+0000 7f95e0ae7640 1 -- start start 2024-04-25T19:15:54.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.467+0000 7f95e0ae7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f95dc0cb880 0x7f95dc15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.467+0000 7f95e0ae7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f95dc072ba0 0x7f95dc072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.467+0000 7f95e0ae7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95dc0cbd50 con 0x7f95dc0cad30 2024-04-25T19:15:54.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.467+0000 7f95e0ae7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f95dc161340 con 0x7f95dc072ba0 2024-04-25T19:15:54.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.467+0000 7f95e0ae7640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f95dc1607a0 con 0x7f95dc0cb880 2024-04-25T19:15:54.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.467+0000 7f95dad9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f95dc072ba0 0x7f95dc072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.467+0000 7f95dad9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f95dc072ba0 0x7f95dc072fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37968/0 (socket says 172.21.15.76:37968) 2024-04-25T19:15:54.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.467+0000 7f95dad9d640 1 -- 172.21.15.76:0/3663775607 learned_addr learned my addr 172.21.15.76:0/3663775607 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:54.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.467+0000 7f95da59c640 1 --2- 172.21.15.76:0/3663775607 >> v2:172.21.15.105:3300/0 conn(0x7f95dc0cb880 0x7f95dc15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.467+0000 7f95dad9d640 1 -- 172.21.15.76:0/3663775607 >> v2:172.21.15.105:3300/0 conn(0x7f95dc0cb880 msgr2=0x7f95dc15d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.467+0000 7f95dad9d640 1 --2- 172.21.15.76:0/3663775607 >> v2:172.21.15.105:3300/0 conn(0x7f95dc0cb880 0x7f95dc15d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.467+0000 7f95dad9d640 1 -- 172.21.15.76:0/3663775607 >> v1:172.21.15.76:6789/0 conn(0x7f95dc0cad30 legacy=0x7f95dc160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.467+0000 7f95dad9d640 1 -- 172.21.15.76:0/3663775607 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95dc0cc010 con 0x7f95dc072ba0 2024-04-25T19:15:54.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.468+0000 7f95dad9d640 1 --2- 172.21.15.76:0/3663775607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95dc072ba0 0x7f95dc072fb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f95dc0737c0 tx=0x7f95d402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965c9176d534572d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:54.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.468+0000 7f95d959a640 1 -- 172.21.15.76:0/3663775607 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d4037070 con 0x7f95dc072ba0 2024-04-25T19:15:54.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.468+0000 7f95d959a640 1 -- 172.21.15.76:0/3663775607 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95d402ae50 con 0x7f95dc072ba0 2024-04-25T19:15:54.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.468+0000 7f95d959a640 1 -- 172.21.15.76:0/3663775607 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d4032860 con 0x7f95dc072ba0 2024-04-25T19:15:54.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.470+0000 7f95e0ae7640 1 -- 172.21.15.76:0/3663775607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95dc072ba0 msgr2=0x7f95dc072fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.470+0000 7f95e0ae7640 1 --2- 172.21.15.76:0/3663775607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95dc072ba0 0x7f95dc072fb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f95dc0737c0 tx=0x7f95d402a300 comp rx=0 tx=0).stop 2024-04-25T19:15:54.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.470+0000 7f95e0ae7640 1 -- 172.21.15.76:0/3663775607 shutdown_connections 2024-04-25T19:15:54.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.470+0000 7f95e0ae7640 1 --2- 172.21.15.76:0/3663775607 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95dc072ba0 0x7f95dc072fb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.470+0000 7f95e0ae7640 1 --2- 172.21.15.76:0/3663775607 >> v2:172.21.15.105:3300/0 conn(0x7f95dc0cb880 0x7f95dc15d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.470+0000 7f95e0ae7640 1 -- 172.21.15.76:0/3663775607 >> 172.21.15.76:0/3663775607 conn(0x7f95dc070b00 msgr2=0x7f95dc070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:54.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.470+0000 7f95e0ae7640 1 -- 172.21.15.76:0/3663775607 shutdown_connections 2024-04-25T19:15:54.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.470+0000 7f95e0ae7640 1 -- 172.21.15.76:0/3663775607 wait complete. 2024-04-25T19:15:54.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.471+0000 7f95e0ae7640 1 Processor -- start 2024-04-25T19:15:54.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.471+0000 7f95e0ae7640 1 -- start start 2024-04-25T19:15:54.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.472+0000 7f95e0ae7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95dc072ba0 0x7f95dc1e7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.472+0000 7f95e0ae7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f95dc0cad30 0x7f95dc0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.472+0000 7f95e0ae7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc0cb880 0x7f95dc1f6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.472+0000 7f95e0ae7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc06e7f0 con 0x7f95dc072ba0 2024-04-25T19:15:54.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.472+0000 7f95e0ae7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc160430 con 0x7f95dc0cb880 2024-04-25T19:15:54.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.472+0000 7f95e0ae7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f95dc0cc5d0 con 0x7f95dc0cad30 2024-04-25T19:15:54.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.472+0000 7f95da59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95dc072ba0 0x7f95dc1e7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.472+0000 7f95da59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95dc072ba0 0x7f95dc1e7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37978/0 (socket says 172.21.15.76:37978) 2024-04-25T19:15:54.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.472+0000 7f95da59c640 1 -- 172.21.15.76:0/500839835 learned_addr learned my addr 172.21.15.76:0/500839835 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:54.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.472+0000 7f95dad9d640 1 --2- 172.21.15.76:0/500839835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc0cb880 0x7f95dc1f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.472+0000 7f95d9d9b640 1 --2- 172.21.15.76:0/500839835 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f95dc0cad30 0x7f95dc0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.472+0000 7f95da59c640 1 -- 172.21.15.76:0/500839835 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f95dc0cad30 msgr2=0x7f95dc0c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.472+0000 7f95da59c640 1 --2- 172.21.15.76:0/500839835 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f95dc0cad30 0x7f95dc0c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.472+0000 7f95da59c640 1 -- 172.21.15.76:0/500839835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc0cb880 msgr2=0x7f95dc1f6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.472+0000 7f95da59c640 1 --2- 172.21.15.76:0/500839835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc0cb880 0x7f95dc1f6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.472+0000 7f95da59c640 1 -- 172.21.15.76:0/500839835 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95dc1e7b50 con 0x7f95dc072ba0 2024-04-25T19:15:54.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.472+0000 7f95da59c640 1 --2- 172.21.15.76:0/500839835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95dc072ba0 0x7f95dc1e7610 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009d10 tx=0x7f95cc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:54.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.473+0000 7f95caffd640 1 -- 172.21.15.76:0/500839835 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc018020 con 0x7f95dc072ba0 2024-04-25T19:15:54.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.473+0000 7f95caffd640 1 -- 172.21.15.76:0/500839835 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95cc014480 con 0x7f95dc072ba0 2024-04-25T19:15:54.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.473+0000 7f95e0ae7640 1 -- 172.21.15.76:0/500839835 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95dc1f32d0 con 0x7f95dc072ba0 2024-04-25T19:15:54.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.473+0000 7f95caffd640 1 -- 172.21.15.76:0/500839835 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc014830 con 0x7f95dc072ba0 2024-04-25T19:15:54.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.473+0000 7f95e0ae7640 1 -- 172.21.15.76:0/500839835 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95dc1f3890 con 0x7f95dc072ba0 2024-04-25T19:15:54.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.475+0000 7f95caffd640 1 -- 172.21.15.76:0/500839835 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f95cc005d10 con 0x7f95dc072ba0 2024-04-25T19:15:54.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.475+0000 7f95caffd640 1 --2- 172.21.15.76:0/500839835 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f95bc0744b0 0x7f95bc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.475+0000 7f95caffd640 1 -- 172.21.15.76:0/500839835 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f95cc095b80 con 0x7f95dc072ba0 2024-04-25T19:15:54.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.475+0000 7f95d9d9b640 1 --2- 172.21.15.76:0/500839835 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f95bc0744b0 0x7f95bc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.476+0000 7f95d9d9b640 1 --2- 172.21.15.76:0/500839835 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f95bc0744b0 0x7f95bc0769c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f95dc1e8950 tx=0x7f95d0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:54.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.476+0000 7f95e7999740 1 Processor -- start 2024-04-25T19:15:54.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.476+0000 7f95e7999740 1 -- start start 2024-04-25T19:15:54.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.477+0000 7f95e7999740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x565506a74270 0x565506e165e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.477+0000 7f95e7999740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565506a4b3b0 0x565506e10880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.477+0000 7f95e7999740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a53590 0x565506e10dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.477+0000 7f95e7999740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x565506d7a740 con 0x565506a53590 2024-04-25T19:15:54.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.477+0000 7f95e7999740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x565506d163a0 con 0x565506a4b3b0 2024-04-25T19:15:54.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.477+0000 7f95e7999740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x565506d76700 con 0x565506a74270 2024-04-25T19:15:54.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.479+0000 7f95dad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a53590 0x565506e10dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.479+0000 7f95dad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a53590 0x565506e10dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37984/0 (socket says 172.21.15.76:37984) 2024-04-25T19:15:54.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.479+0000 7f95dad9d640 1 -- 172.21.15.76:0/1296265186 learned_addr learned my addr 172.21.15.76:0/1296265186 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:54.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.479+0000 7f95d9d9b640 1 --2- 172.21.15.76:0/1296265186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565506a4b3b0 0x565506e10880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.479+0000 7f95da59c640 1 --2- 172.21.15.76:0/1296265186 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x565506a74270 0x565506e165e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.479+0000 7f95dad9d640 1 -- 172.21.15.76:0/1296265186 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x565506a74270 msgr2=0x565506e165e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.479+0000 7f95dad9d640 1 --2- 172.21.15.76:0/1296265186 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x565506a74270 0x565506e165e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.479+0000 7f95dad9d640 1 -- 172.21.15.76:0/1296265186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565506a4b3b0 msgr2=0x565506e10880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.479+0000 7f95dad9d640 1 --2- 172.21.15.76:0/1296265186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565506a4b3b0 0x565506e10880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.479+0000 7f95dad9d640 1 -- 172.21.15.76:0/1296265186 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565506d32750 con 0x565506a53590 2024-04-25T19:15:54.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.479+0000 7f95d9d9b640 1 --2- 172.21.15.76:0/1296265186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565506a4b3b0 0x565506e10880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:54.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.479+0000 7f95dad9d640 1 --2- 172.21.15.76:0/1296265186 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a53590 0x565506e10dc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009960 tx=0x7f95d402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c204d7680fa3d391 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:54.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.479+0000 7f95bbfff640 1 -- 172.21.15.76:0/1296265186 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d4042070 con 0x565506a53590 2024-04-25T19:15:54.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.479+0000 7f95bbfff640 1 -- 172.21.15.76:0/1296265186 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95d402ae50 con 0x565506a53590 2024-04-25T19:15:54.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.479+0000 7f95bbfff640 1 -- 172.21.15.76:0/1296265186 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d4037040 con 0x565506a53590 2024-04-25T19:15:54.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.480+0000 7f95e7999740 1 -- 172.21.15.76:0/1296265186 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a53590 msgr2=0x565506e10dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.480+0000 7f95e7999740 1 --2- 172.21.15.76:0/1296265186 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a53590 0x565506e10dc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009960 tx=0x7f95d402c000 comp rx=0 tx=0).stop 2024-04-25T19:15:54.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.480+0000 7f95e7999740 1 -- 172.21.15.76:0/1296265186 shutdown_connections 2024-04-25T19:15:54.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.480+0000 7f95e7999740 1 --2- 172.21.15.76:0/1296265186 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a53590 0x565506e10dc0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.480+0000 7f95e7999740 1 --2- 172.21.15.76:0/1296265186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565506a4b3b0 0x565506e10880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.480+0000 7f95e7999740 1 --2- 172.21.15.76:0/1296265186 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x565506a74270 0x565506e165e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.480+0000 7f95e7999740 1 -- 172.21.15.76:0/1296265186 >> 172.21.15.76:0/1296265186 conn(0x565506a683c0 msgr2=0x565506e14d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:54.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.480+0000 7f95e7999740 1 -- 172.21.15.76:0/1296265186 shutdown_connections 2024-04-25T19:15:54.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.480+0000 7f95e7999740 1 -- 172.21.15.76:0/1296265186 wait complete. 2024-04-25T19:15:54.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.481+0000 7f95e7999740 1 -- start start 2024-04-25T19:15:54.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.481+0000 7f95e7999740 1 Processor -- start 2024-04-25T19:15:54.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.482+0000 7f95e7999740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x565506a4b3b0 0x565506e399f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.482+0000 7f95e7999740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565506a53590 0x565506e33c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.482+0000 7f95e7999740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a74270 0x565506e67fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.482+0000 7f95e7999740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x565506e54d00 con 0x565506a74270 2024-04-25T19:15:54.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.482+0000 7f95e7999740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x565506d01fe0 con 0x565506a53590 2024-04-25T19:15:54.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.482+0000 7f95e7999740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x565506e14be0 con 0x565506a4b3b0 2024-04-25T19:15:54.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.482+0000 7f95e7999740 10 client.0 _ll_register_callbacks cb 0x565506d18150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:15:54.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.482+0000 7f95dad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a74270 0x565506e67fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.482+0000 7f95dad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a74270 0x565506e67fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37998/0 (socket says 172.21.15.76:37998) 2024-04-25T19:15:54.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.482+0000 7f95dad9d640 1 -- 172.21.15.76:0/1655966137 learned_addr learned my addr 172.21.15.76:0/1655966137 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:54.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.482+0000 7f95da59c640 1 --2- 172.21.15.76:0/1655966137 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x565506a4b3b0 0x565506e399f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.482+0000 7f95d9d9b640 1 --2- 172.21.15.76:0/1655966137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565506a53590 0x565506e33c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.482+0000 7f95dad9d640 1 -- 172.21.15.76:0/1655966137 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x565506a4b3b0 msgr2=0x565506e399f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.482+0000 7f95dad9d640 1 --2- 172.21.15.76:0/1655966137 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x565506a4b3b0 0x565506e399f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.482+0000 7f95dad9d640 1 -- 172.21.15.76:0/1655966137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565506a53590 msgr2=0x565506e33c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.482+0000 7f95dad9d640 1 --2- 172.21.15.76:0/1655966137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565506a53590 0x565506e33c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.482+0000 7f95dad9d640 1 -- 172.21.15.76:0/1655966137 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565506a8c2d0 con 0x565506a74270 2024-04-25T19:15:54.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.482+0000 7f95dad9d640 1 --2- 172.21.15.76:0/1655966137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a74270 0x565506e67fa0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f95d4004460 tx=0x7f95d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c297ffac87c88575 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:54.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.482+0000 7f95b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-04-25T19:15:54.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.482+0000 7f95b97fa640 1 -- 172.21.15.76:0/1655966137 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d4042070 con 0x565506a74270 2024-04-25T19:15:54.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.482+0000 7f95b97fa640 1 -- 172.21.15.76:0/1655966137 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f95d4005990 con 0x565506a74270 2024-04-25T19:15:54.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.482+0000 7f95e7999740 1 -- 172.21.15.76:0/1655966137 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95d402ae50 con 0x565506a74270 2024-04-25T19:15:54.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.482+0000 7f95b97fa640 1 -- 172.21.15.76:0/1655966137 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f95d4005b90 con 0x565506a74270 2024-04-25T19:15:54.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.482+0000 7f95e7999740 10 client.4782 fetch_fsmap learned FSMap version 16 2024-04-25T19:15:54.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.482+0000 7f95e7999740 1 -- 172.21.15.76:0/1655966137 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x565506b566b0 con 0x565506a74270 2024-04-25T19:15:54.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.482+0000 7f95b97fa640 1 -- 172.21.15.76:0/1655966137 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95d402ae50 con 0x565506a74270 2024-04-25T19:15:54.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.483+0000 7f95b97fa640 1 -- 172.21.15.76:0/1655966137 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 16) ==== 4784+0+0 (secure 0 0 0) 0x7f95d4055080 con 0x565506a74270 2024-04-25T19:15:54.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.483+0000 7f95e7999740 10 client.4782 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T19:15:54.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.483+0000 7f95e7999740 10 client.4782 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-25T19:15:54.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.483+0000 7f95e7999740 20 client.4782 populate_metadata read hostname 'smithi076' 2024-04-25T19:15:54.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.483+0000 7f95e7999740 1 --2- 172.21.15.76:0/1655966137 >> [v2:172.21.15.76:6835/4226277412,v1:172.21.15.76:6837/4226277412] conn(0x565506a32e40 0x565506e32b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.483+0000 7f95e7999740 4 client.4782 mds_command: new command op to 4284 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:15:54.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.483+0000 7f95e7999740 1 -- 172.21.15.76:0/1655966137 --> [v2:172.21.15.76:6835/4226277412,v1:172.21.15.76:6837/4226277412] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565506d88350 con 0x565506a32e40 2024-04-25T19:15:54.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.483+0000 7f95da59c640 1 --2- 172.21.15.76:0/1655966137 >> [v2:172.21.15.76:6835/4226277412,v1:172.21.15.76:6837/4226277412] conn(0x565506a32e40 0x565506e32b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.483+0000 7f95b97fa640 10 client.4782 ms_handle_connect on v2:172.21.15.76:6835/4226277412 2024-04-25T19:15:54.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.483+0000 7f95da59c640 1 --2- 172.21.15.76:0/1655966137 >> [v2:172.21.15.76:6835/4226277412,v1:172.21.15.76:6837/4226277412] conn(0x565506a32e40 0x565506e32b40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef27e5698bf11472 server_cookie=196ead1cfc943d1f in_seq=0 out_seq=0 2024-04-25T19:15:54.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.485+0000 7f95b97fa640 1 -- 172.21.15.76:0/1655966137 <== mds.0 v2:172.21.15.76:6835/4226277412 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x565506d88350 con 0x565506a32e40 2024-04-25T19:15:54.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.485+0000 7f95b97fa640 10 client.4782 handle_command_reply: tid=0 2024-04-25T19:15:54.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.485+0000 7f95e7999740 1 client.4782 shutdown 2024-04-25T19:15:54.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.485+0000 7f95e7999740 1 -- 172.21.15.76:0/1655966137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a74270 msgr2=0x565506e67fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.485+0000 7f95e7999740 1 --2- 172.21.15.76:0/1655966137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a74270 0x565506e67fa0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f95d4004460 tx=0x7f95d4004730 comp rx=0 tx=0).stop 2024-04-25T19:15:54.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.485+0000 7f95e7999740 1 -- 172.21.15.76:0/1655966137 shutdown_connections 2024-04-25T19:15:54.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.485+0000 7f95e7999740 1 --2- 172.21.15.76:0/1655966137 >> [v2:172.21.15.76:6835/4226277412,v1:172.21.15.76:6837/4226277412] conn(0x565506a32e40 0x565506e32b40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.485+0000 7f95e7999740 1 --2- 172.21.15.76:0/1655966137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a74270 0x565506e67fa0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.485+0000 7f95e7999740 1 --2- 172.21.15.76:0/1655966137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565506a53590 0x565506e33c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.485+0000 7f95b97fa640 0 client.4782 ms_handle_reset on v2:172.21.15.76:6835/4226277412 2024-04-25T19:15:54.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.485+0000 7f95e7999740 1 --2- 172.21.15.76:0/1655966137 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x565506a4b3b0 0x565506e399f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.485+0000 7f95e7999740 1 -- 172.21.15.76:0/1655966137 >> 172.21.15.76:0/1655966137 conn(0x565506a683c0 msgr2=0x565506e0f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:54.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.485+0000 7f95e7999740 1 -- 172.21.15.76:0/1655966137 shutdown_connections 2024-04-25T19:15:54.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.485+0000 7f95e7999740 1 -- 172.21.15.76:0/1655966137 wait complete. 2024-04-25T19:15:54.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.485+0000 7f95e7999740 20 client.4782 trim_cache size 0 max 16384 2024-04-25T19:15:54.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.504+0000 7f95e7999740 1 Processor -- start 2024-04-25T19:15:54.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.504+0000 7f95e7999740 1 -- start start 2024-04-25T19:15:54.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.505+0000 7f95e7999740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a32e40 0x565506e32b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.505+0000 7f95e7999740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565506a4b3b0 0x565506e33080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.505+0000 7f95e7999740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x565506a53590 0x565506e7a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.505+0000 7f95e7999740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc15ca40 con 0x565506a32e40 2024-04-25T19:15:54.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.505+0000 7f95e7999740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x565506e344e0 con 0x565506a4b3b0 2024-04-25T19:15:54.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.505+0000 7f95e7999740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x565506e56ee0 con 0x565506a53590 2024-04-25T19:15:54.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.505+0000 7f95da59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a32e40 0x565506e32b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.505+0000 7f95da59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a32e40 0x565506e32b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38012/0 (socket says 172.21.15.76:38012) 2024-04-25T19:15:54.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.505+0000 7f95da59c640 1 -- 172.21.15.76:0/564115188 learned_addr learned my addr 172.21.15.76:0/564115188 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:54.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.506+0000 7f95dad9d640 1 --2- 172.21.15.76:0/564115188 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x565506a53590 0x565506e7a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.506+0000 7f95d9d9b640 1 --2- 172.21.15.76:0/564115188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565506a4b3b0 0x565506e33080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.506+0000 7f95da59c640 1 -- 172.21.15.76:0/564115188 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x565506a53590 msgr2=0x565506e7a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.506+0000 7f95da59c640 1 --2- 172.21.15.76:0/564115188 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x565506a53590 0x565506e7a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.506+0000 7f95da59c640 1 -- 172.21.15.76:0/564115188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565506a4b3b0 msgr2=0x565506e33080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.506+0000 7f95da59c640 1 --2- 172.21.15.76:0/564115188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565506a4b3b0 0x565506e33080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.506+0000 7f95da59c640 1 -- 172.21.15.76:0/564115188 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565506e11d60 con 0x565506a32e40 2024-04-25T19:15:54.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.506+0000 7f95d9d9b640 1 --2- 172.21.15.76:0/564115188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565506a4b3b0 0x565506e33080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:15:54.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.506+0000 7f95da59c640 1 --2- 172.21.15.76:0/564115188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a32e40 0x565506e32b40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f95cc05fc50 tx=0x7f95cc093f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89834d8870a39b8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:54.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.506+0000 7f95bb7fe640 1 -- 172.21.15.76:0/564115188 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc07e070 con 0x565506a32e40 2024-04-25T19:15:54.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.506+0000 7f95bb7fe640 1 -- 172.21.15.76:0/564115188 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95cc0791c0 con 0x565506a32e40 2024-04-25T19:15:54.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.506+0000 7f95bb7fe640 1 -- 172.21.15.76:0/564115188 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc05fcc0 con 0x565506a32e40 2024-04-25T19:15:54.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.507+0000 7f95e7999740 1 -- 172.21.15.76:0/564115188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a32e40 msgr2=0x565506e32b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.507+0000 7f95e7999740 1 --2- 172.21.15.76:0/564115188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a32e40 0x565506e32b40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f95cc05fc50 tx=0x7f95cc093f80 comp rx=0 tx=0).stop 2024-04-25T19:15:54.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.507+0000 7f95e7999740 1 -- 172.21.15.76:0/564115188 shutdown_connections 2024-04-25T19:15:54.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.507+0000 7f95e7999740 1 --2- 172.21.15.76:0/564115188 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x565506a53590 0x565506e7a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.507+0000 7f95e7999740 1 --2- 172.21.15.76:0/564115188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565506a4b3b0 0x565506e33080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.507+0000 7f95e7999740 1 --2- 172.21.15.76:0/564115188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a32e40 0x565506e32b40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.507+0000 7f95e7999740 1 -- 172.21.15.76:0/564115188 >> 172.21.15.76:0/564115188 conn(0x565506a683c0 msgr2=0x565506e53ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:54.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.507+0000 7f95e7999740 1 -- 172.21.15.76:0/564115188 shutdown_connections 2024-04-25T19:15:54.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.507+0000 7f95e7999740 1 -- 172.21.15.76:0/564115188 wait complete. 2024-04-25T19:15:54.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.508+0000 7f95e7999740 1 -- start start 2024-04-25T19:15:54.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.508+0000 7f95e7999740 1 Processor -- start 2024-04-25T19:15:54.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.509+0000 7f95e7999740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a32e40 0x565506ed6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.509+0000 7f95e7999740 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x565506a4b3b0 0x565506ecdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.509+0000 7f95e7999740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565506a53590 0x565506e0fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.509+0000 7f95e7999740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x565506e329c0 con 0x565506a32e40 2024-04-25T19:15:54.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.509+0000 7f95e7999740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x565506e4d120 con 0x565506a53590 2024-04-25T19:15:54.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.509+0000 7f95e7999740 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x565506e69fa0 con 0x565506a4b3b0 2024-04-25T19:15:54.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.509+0000 7f95e7999740 10 client.0 _ll_register_callbacks cb 0x7f95cc009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:15:54.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.509+0000 7f95da59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a32e40 0x565506ed6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.509+0000 7f95da59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a32e40 0x565506ed6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38022/0 (socket says 172.21.15.76:38022) 2024-04-25T19:15:54.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.509+0000 7f95da59c640 1 -- 172.21.15.76:0/3163625832 learned_addr learned my addr 172.21.15.76:0/3163625832 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:54.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.509+0000 7f95d9d9b640 1 --2- 172.21.15.76:0/3163625832 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x565506a4b3b0 0x565506ecdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.509+0000 7f95dad9d640 1 --2- 172.21.15.76:0/3163625832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565506a53590 0x565506e0fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.509+0000 7f95da59c640 1 -- 172.21.15.76:0/3163625832 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x565506a4b3b0 msgr2=0x565506ecdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.509+0000 7f95da59c640 1 --2- 172.21.15.76:0/3163625832 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x565506a4b3b0 0x565506ecdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.509+0000 7f95da59c640 1 -- 172.21.15.76:0/3163625832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565506a53590 msgr2=0x565506e0fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.509+0000 7f95da59c640 1 --2- 172.21.15.76:0/3163625832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565506a53590 0x565506e0fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.509+0000 7f95da59c640 1 -- 172.21.15.76:0/3163625832 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565506e54190 con 0x565506a32e40 2024-04-25T19:15:54.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.509+0000 7f95dad9d640 1 --2- 172.21.15.76:0/3163625832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565506a53590 0x565506e0fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:15:54.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.509+0000 7f95da59c640 1 --2- 172.21.15.76:0/3163625832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a32e40 0x565506ed6f80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f95cc05c2b0 tx=0x7f95d4008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e01e38763fade92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:54.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.509+0000 7f95b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-04-25T19:15:54.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.510+0000 7f95b97fa640 1 -- 172.21.15.76:0/3163625832 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc086060 con 0x565506a32e40 2024-04-25T19:15:54.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.510+0000 7f95b97fa640 1 -- 172.21.15.76:0/3163625832 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f95cc05f120 con 0x565506a32e40 2024-04-25T19:15:54.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.510+0000 7f95e7999740 1 -- 172.21.15.76:0/3163625832 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95cc0791c0 con 0x565506a32e40 2024-04-25T19:15:54.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.510+0000 7f95b97fa640 1 -- 172.21.15.76:0/3163625832 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f95cc08c050 con 0x565506a32e40 2024-04-25T19:15:54.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.510+0000 7f95e7999740 10 client.4788 fetch_fsmap learned FSMap version 16 2024-04-25T19:15:54.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.510+0000 7f95e7999740 1 -- 172.21.15.76:0/3163625832 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x565506e37d10 con 0x565506a32e40 2024-04-25T19:15:54.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.510+0000 7f95b97fa640 1 -- 172.21.15.76:0/3163625832 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95cc0791c0 con 0x565506a32e40 2024-04-25T19:15:54.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.510+0000 7f95b97fa640 1 -- 172.21.15.76:0/3163625832 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 16) ==== 4784+0+0 (secure 0 0 0) 0x7f95cc05b630 con 0x565506a32e40 2024-04-25T19:15:54.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.510+0000 7f95e7999740 10 client.4788 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T19:15:54.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.510+0000 7f95e7999740 10 client.4788 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-25T19:15:54.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.510+0000 7f95e7999740 20 client.4788 populate_metadata read hostname 'smithi076' 2024-04-25T19:15:54.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.510+0000 7f95e7999740 1 --2- 172.21.15.76:0/3163625832 >> [v2:172.21.15.76:6835/4226277412,v1:172.21.15.76:6837/4226277412] conn(0x565506a08280 0x565506e4c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.510+0000 7f95e7999740 4 client.4788 mds_command: new command op to 4284 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T19:15:54.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.510+0000 7f95e7999740 1 -- 172.21.15.76:0/3163625832 --> [v2:172.21.15.76:6835/4226277412,v1:172.21.15.76:6837/4226277412] -- command(tid 0: {"prefix": "status"}) -- 0x565506e54b50 con 0x565506a08280 2024-04-25T19:15:54.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.510+0000 7f95d9d9b640 1 --2- 172.21.15.76:0/3163625832 >> [v2:172.21.15.76:6835/4226277412,v1:172.21.15.76:6837/4226277412] conn(0x565506a08280 0x565506e4c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.511+0000 7f95b97fa640 10 client.4788 ms_handle_connect on v2:172.21.15.76:6835/4226277412 2024-04-25T19:15:54.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.511+0000 7f95d9d9b640 1 --2- 172.21.15.76:0/3163625832 >> [v2:172.21.15.76:6835/4226277412,v1:172.21.15.76:6837/4226277412] conn(0x565506a08280 0x565506e4c5e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d79b33c4be83210 server_cookie=4e6e9068bfdc8ec1 in_seq=0 out_seq=0 2024-04-25T19:15:54.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.511+0000 7f95b97fa640 1 -- 172.21.15.76:0/3163625832 <== mds.0 v2:172.21.15.76:6835/4226277412 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x565506e54b50 con 0x565506a08280 2024-04-25T19:15:54.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.511+0000 7f95b97fa640 10 client.4788 handle_command_reply: tid=0 2024-04-25T19:15:54.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.511+0000 7f95e7999740 1 client.4788 shutdown 2024-04-25T19:15:54.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.511+0000 7f95e7999740 1 -- 172.21.15.76:0/3163625832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a32e40 msgr2=0x565506ed6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.511+0000 7f95e7999740 1 --2- 172.21.15.76:0/3163625832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a32e40 0x565506ed6f80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f95cc05c2b0 tx=0x7f95d4008030 comp rx=0 tx=0).stop 2024-04-25T19:15:54.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.511+0000 7f95e7999740 1 -- 172.21.15.76:0/3163625832 shutdown_connections 2024-04-25T19:15:54.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.511+0000 7f95e7999740 1 --2- 172.21.15.76:0/3163625832 >> [v2:172.21.15.76:6835/4226277412,v1:172.21.15.76:6837/4226277412] conn(0x565506a08280 0x565506e4c5e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.511+0000 7f95e7999740 1 --2- 172.21.15.76:0/3163625832 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x565506a53590 0x565506e0fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.511+0000 7f95b97fa640 0 client.4788 ms_handle_reset on v2:172.21.15.76:6835/4226277412 2024-04-25T19:15:54.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.511+0000 7f95e7999740 1 --2- 172.21.15.76:0/3163625832 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x565506a4b3b0 0x565506ecdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.511+0000 7f95e7999740 1 --2- 172.21.15.76:0/3163625832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565506a32e40 0x565506ed6f80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.511+0000 7f95e7999740 1 -- 172.21.15.76:0/3163625832 >> 172.21.15.76:0/3163625832 conn(0x565506a683c0 msgr2=0x565506e53160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:54.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.511+0000 7f95e7999740 1 -- 172.21.15.76:0/3163625832 shutdown_connections 2024-04-25T19:15:54.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.511+0000 7f95e7999740 1 -- 172.21.15.76:0/3163625832 wait complete. 2024-04-25T19:15:54.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.511+0000 7f95e7999740 20 client.4788 trim_cache size 0 max 16384 2024-04-25T19:15:54.520 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:15:54.520 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:15:54.520 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2024-04-25T19:15:54.520 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4284, 2024-04-25T19:15:54.520 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-04-25T19:15:54.520 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-04-25T19:15:54.520 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-04-25T19:15:54.520 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 35.110175808999998, 2024-04-25T19:15:54.520 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 16, 2024-04-25T19:15:54.520 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 21, 2024-04-25T19:15:54.520 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 21, 2024-04-25T19:15:54.520 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 42.145999879999998 2024-04-25T19:15:54.520 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:15:54.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.512+0000 7f95e0ae7640 1 -- 172.21.15.76:0/500839835 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f95bc0744b0 msgr2=0x7f95bc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.512+0000 7f95e0ae7640 1 --2- 172.21.15.76:0/500839835 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f95bc0744b0 0x7f95bc0769c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f95dc1e8950 tx=0x7f95d0009040 comp rx=0 tx=0).stop 2024-04-25T19:15:54.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.512+0000 7f95e0ae7640 1 -- 172.21.15.76:0/500839835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95dc072ba0 msgr2=0x7f95dc1e7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.512+0000 7f95e0ae7640 1 --2- 172.21.15.76:0/500839835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95dc072ba0 0x7f95dc1e7610 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009d10 tx=0x7f95cc00c490 comp rx=0 tx=0).stop 2024-04-25T19:15:54.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.512+0000 7f95e0ae7640 1 -- 172.21.15.76:0/500839835 shutdown_connections 2024-04-25T19:15:54.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.512+0000 7f95e0ae7640 1 --2- 172.21.15.76:0/500839835 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f95bc0744b0 0x7f95bc0769c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.512+0000 7f95e0ae7640 1 --2- 172.21.15.76:0/500839835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95dc0cb880 0x7f95dc1f6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.512+0000 7f95e0ae7640 1 --2- 172.21.15.76:0/500839835 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f95dc0cad30 0x7f95dc0c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.512+0000 7f95e0ae7640 1 --2- 172.21.15.76:0/500839835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95dc072ba0 0x7f95dc1e7610 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.512+0000 7f95e0ae7640 1 -- 172.21.15.76:0/500839835 >> 172.21.15.76:0/500839835 conn(0x7f95dc070b00 msgr2=0x7f95dc155610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:54.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.512+0000 7f95e0ae7640 1 -- 172.21.15.76:0/500839835 shutdown_connections 2024-04-25T19:15:54.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.512+0000 7f95e0ae7640 1 -- 172.21.15.76:0/500839835 wait complete. 2024-04-25T19:15:54.525 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T19:15:54.535 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T19:15:54.535 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T19:15:54.535 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T19:15:54.535 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-25T19:15:54.570 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-25T19:15:54.570 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-04-25T19:15:54.571 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-25T19:15:54.571 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T19:15:54.571 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T19:15:54.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:15:54.571 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-04-25T19:15:54.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:15:54.587 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link delete ceph-brx 2024-04-25T19:15:54.652 INFO:teuthology.orchestra.run.smithi076.stderr:Cannot find device "ceph-brx" 2024-04-25T19:15:54.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:15:54.654 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-25T19:15:54.654 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T19:15:54.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.799+0000 7fa93e8aa640 1 Processor -- start 2024-04-25T19:15:54.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.799+0000 7fa93e8aa640 1 -- start start 2024-04-25T19:15:54.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.801+0000 7fa93e8aa640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa93815b270 0x7fa938154a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.801+0000 7fa93e8aa640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa9380caaf0 0x7fa9380caf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.801+0000 7fa93e8aa640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa938072bc0 con 0x7fa9380c7d10 2024-04-25T19:15:54.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.801+0000 7fa93e8aa640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa9380ca3d0 con 0x7fa93815b270 2024-04-25T19:15:54.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.801+0000 7fa93e8aa640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fa938154fa0 con 0x7fa9380caaf0 2024-04-25T19:15:54.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.801+0000 7fa9377fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa93815b270 0x7fa938154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.801+0000 7fa9377fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa93815b270 0x7fa938154a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38030/0 (socket says 172.21.15.76:38030) 2024-04-25T19:15:54.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.801+0000 7fa937fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa9380c7d10 0x7fa9380cdd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56756/0 (socket says 172.21.15.76:56756) 2024-04-25T19:15:54.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.801+0000 7fa9377fe640 1 -- 172.21.15.76:0/3448064660 learned_addr learned my addr 172.21.15.76:0/3448064660 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:54.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.801+0000 7fa93ce47640 1 --2- 172.21.15.76:0/3448064660 >> v2:172.21.15.105:3300/0 conn(0x7fa9380caaf0 0x7fa9380caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.801+0000 7fa9377fe640 1 -- 172.21.15.76:0/3448064660 >> v2:172.21.15.105:3300/0 conn(0x7fa9380caaf0 msgr2=0x7fa9380caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:54.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.801+0000 7fa9377fe640 1 --2- 172.21.15.76:0/3448064660 >> v2:172.21.15.105:3300/0 conn(0x7fa9380caaf0 0x7fa9380caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.801+0000 7fa9377fe640 1 -- 172.21.15.76:0/3448064660 >> v1:172.21.15.76:6789/0 conn(0x7fa9380c7d10 legacy=0x7fa9380cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.801+0000 7fa9377fe640 1 -- 172.21.15.76:0/3448064660 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa93815b680 con 0x7fa93815b270 2024-04-25T19:15:54.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.802+0000 7fa9377fe640 1 --2- 172.21.15.76:0/3448064660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa93815b270 0x7fa938154a60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa93806f630 tx=0x7fa92c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ef410c7b74143 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:54.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.802+0000 7fa936ffd640 1 -- 172.21.15.76:0/3448064660 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c037070 con 0x7fa93815b270 2024-04-25T19:15:54.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.802+0000 7fa936ffd640 1 -- 172.21.15.76:0/3448064660 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa92c032440 con 0x7fa93815b270 2024-04-25T19:15:54.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.802+0000 7fa936ffd640 1 -- 172.21.15.76:0/3448064660 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c032810 con 0x7fa93815b270 2024-04-25T19:15:54.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.803+0000 7fa93e8aa640 1 -- 172.21.15.76:0/3448064660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa93815b270 msgr2=0x7fa938154a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.803+0000 7fa93e8aa640 1 --2- 172.21.15.76:0/3448064660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa93815b270 0x7fa938154a60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa93806f630 tx=0x7fa92c02a270 comp rx=0 tx=0).stop 2024-04-25T19:15:54.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.803+0000 7fa93e8aa640 1 -- 172.21.15.76:0/3448064660 shutdown_connections 2024-04-25T19:15:54.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.803+0000 7fa93e8aa640 1 --2- 172.21.15.76:0/3448064660 >> v2:172.21.15.105:3300/0 conn(0x7fa9380caaf0 0x7fa9380caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.803+0000 7fa93e8aa640 1 --2- 172.21.15.76:0/3448064660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa93815b270 0x7fa938154a60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.803+0000 7fa93e8aa640 1 -- 172.21.15.76:0/3448064660 >> 172.21.15.76:0/3448064660 conn(0x7fa938070800 msgr2=0x7fa938070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:54.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.803+0000 7fa93e8aa640 1 -- 172.21.15.76:0/3448064660 shutdown_connections 2024-04-25T19:15:54.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.803+0000 7fa93e8aa640 1 -- 172.21.15.76:0/3448064660 wait complete. 2024-04-25T19:15:54.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.804+0000 7fa93e8aa640 1 Processor -- start 2024-04-25T19:15:54.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.805+0000 7fa93e8aa640 1 -- start start 2024-04-25T19:15:54.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.806+0000 7fa93e8aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9380c7d10 0x7fa9381f1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.806+0000 7fa93e8aa640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa9380caaf0 0x7fa9381f4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.806+0000 7fa93e8aa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa93815b270 0x7fa938148400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.806+0000 7fa93e8aa640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa9380c8c40 con 0x7fa93815b270 2024-04-25T19:15:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.806+0000 7fa93e8aa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa9380ca5f0 con 0x7fa9380c7d10 2024-04-25T19:15:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.806+0000 7fa93e8aa640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa9380c8f20 con 0x7fa9380caaf0 2024-04-25T19:15:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.806+0000 7fa93ce47640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa93815b270 0x7fa938148400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.806+0000 7fa93ce47640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa93815b270 0x7fa938148400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38036/0 (socket says 172.21.15.76:38036) 2024-04-25T19:15:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.806+0000 7fa93ce47640 1 -- 172.21.15.76:0/74148039 learned_addr learned my addr 172.21.15.76:0/74148039 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.806+0000 7fa937fff640 1 --2- 172.21.15.76:0/74148039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9380c7d10 0x7fa9381f1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.806+0000 7fa9377fe640 1 --2- 172.21.15.76:0/74148039 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa9380caaf0 0x7fa9381f4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.806+0000 7fa93ce47640 1 -- 172.21.15.76:0/74148039 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa9380caaf0 msgr2=0x7fa9381f4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.806+0000 7fa93ce47640 1 --2- 172.21.15.76:0/74148039 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa9380caaf0 0x7fa9381f4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.806+0000 7fa93ce47640 1 -- 172.21.15.76:0/74148039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9380c7d10 msgr2=0x7fa9381f1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.806+0000 7fa93ce47640 1 --2- 172.21.15.76:0/74148039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9380c7d10 0x7fa9381f1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.806+0000 7fa93ce47640 1 -- 172.21.15.76:0/74148039 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9381f4950 con 0x7fa93815b270 2024-04-25T19:15:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.806+0000 7fa93ce47640 1 --2- 172.21.15.76:0/74148039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa93815b270 0x7fa938148400 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa930009cb0 tx=0x7fa93000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.807+0000 7fa934ff9640 1 -- 172.21.15.76:0/74148039 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa930018020 con 0x7fa93815b270 2024-04-25T19:15:54.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.807+0000 7fa934ff9640 1 -- 172.21.15.76:0/74148039 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa930014480 con 0x7fa93815b270 2024-04-25T19:15:54.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.807+0000 7fa93e8aa640 1 -- 172.21.15.76:0/74148039 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa93814e3f0 con 0x7fa93815b270 2024-04-25T19:15:54.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.807+0000 7fa934ff9640 1 -- 172.21.15.76:0/74148039 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa930014870 con 0x7fa93815b270 2024-04-25T19:15:54.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.807+0000 7fa93e8aa640 1 -- 172.21.15.76:0/74148039 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa93814ba60 con 0x7fa93815b270 2024-04-25T19:15:54.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.808+0000 7fa93e8aa640 1 -- 172.21.15.76:0/74148039 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa93806a390 con 0x7fa93815b270 2024-04-25T19:15:54.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.809+0000 7fa934ff9640 1 -- 172.21.15.76:0/74148039 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa930005d10 con 0x7fa93815b270 2024-04-25T19:15:54.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.809+0000 7fa934ff9640 1 --2- 172.21.15.76:0/74148039 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa9240744e0 0x7fa9240769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:54.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.809+0000 7fa934ff9640 1 -- 172.21.15.76:0/74148039 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa930095df0 con 0x7fa93815b270 2024-04-25T19:15:54.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.809+0000 7fa937fff640 1 --2- 172.21.15.76:0/74148039 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa9240744e0 0x7fa9240769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:54.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.810+0000 7fa937fff640 1 --2- 172.21.15.76:0/74148039 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa9240744e0 0x7fa9240769f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa928007fd0 tx=0x7fa928007e80 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:54.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:54.814+0000 7fa934ff9640 1 -- 172.21.15.76:0/74148039 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa930022030 con 0x7fa93815b270 2024-04-25T19:15:55.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:55.024+0000 7fa93e8aa640 1 -- 172.21.15.76:0/74148039 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa9380742e0 con 0x7fa93815b270 2024-04-25T19:15:55.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:55.024+0000 7fa934ff9640 1 -- 172.21.15.76:0/74148039 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7fa930010040 con 0x7fa93815b270 2024-04-25T19:15:55.026 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T19:15:55.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:55.026+0000 7fa93e8aa640 1 -- 172.21.15.76:0/74148039 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa9240744e0 msgr2=0x7fa9240769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:55.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:55.026+0000 7fa93e8aa640 1 --2- 172.21.15.76:0/74148039 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa9240744e0 0x7fa9240769f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa928007fd0 tx=0x7fa928007e80 comp rx=0 tx=0).stop 2024-04-25T19:15:55.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:55.026+0000 7fa93e8aa640 1 -- 172.21.15.76:0/74148039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa93815b270 msgr2=0x7fa938148400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:55.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:55.026+0000 7fa93e8aa640 1 --2- 172.21.15.76:0/74148039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa93815b270 0x7fa938148400 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa930009cb0 tx=0x7fa93000c490 comp rx=0 tx=0).stop 2024-04-25T19:15:55.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:55.026+0000 7fa93e8aa640 1 -- 172.21.15.76:0/74148039 shutdown_connections 2024-04-25T19:15:55.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:55.026+0000 7fa93e8aa640 1 --2- 172.21.15.76:0/74148039 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa9240744e0 0x7fa9240769f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:55.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:55.026+0000 7fa93e8aa640 1 --2- 172.21.15.76:0/74148039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa93815b270 0x7fa938148400 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:55.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:55.026+0000 7fa93e8aa640 1 --2- 172.21.15.76:0/74148039 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa9380caaf0 0x7fa9381f4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:55.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:55.026+0000 7fa93e8aa640 1 --2- 172.21.15.76:0/74148039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9380c7d10 0x7fa9381f1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:55.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:55.026+0000 7fa93e8aa640 1 -- 172.21.15.76:0/74148039 >> 172.21.15.76:0/74148039 conn(0x7fa938070800 msgr2=0x7fa93814a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:55.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:55.026+0000 7fa93e8aa640 1 -- 172.21.15.76:0/74148039 shutdown_connections 2024-04-25T19:15:55.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:55.026+0000 7fa93e8aa640 1 -- 172.21.15.76:0/74148039 wait complete. 2024-04-25T19:15:55.039 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T19:15:55.039 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T19:15:55.039 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2024-04-25T19:15:55.039 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T19:15:55.039 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T19:15:55.039 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T19:15:55.039 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T19:15:55.039 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T19:15:55.040 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T19:15:55.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:15:55.040 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2024-04-25T19:15:55.054 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T19:15:55.054 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T19:15:55.055 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T19:15:55.055 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:15:55.055 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2024-04-25T19:15:55.055 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:15:55.055 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T19:15:55.055 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2024-04-25T19:15:55.055 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2024-04-25T19:15:55.055 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T19:15:55.055 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2024-04-25T19:15:55.055 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2024-04-25T19:15:55.055 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T19:15:55.055 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2024-04-25T19:15:55.055 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T19:15:55.055 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2024-04-25T19:15:55.056 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T19:15:55.056 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42554sec preferred_lft 42554sec 2024-04-25T19:15:55.056 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::5a59:ab51:4212:ba54/64 scope link noprefixroute 2024-04-25T19:15:55.056 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-04-25T19:15:55.056 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T19:15:55.056 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:15:55.056 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-04-25T19:15:55.057 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link add name ceph-brx type bridge 2024-04-25T19:15:55.057 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip addr flush dev ceph-brx 2024-04-25T19:15:55.057 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set ceph-brx up 2024-04-25T19:15:55.057 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T19:15:55.057 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-04-25T19:15:55.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:15:55.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:15:55.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:15:55.231 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T19:15:55.254 INFO:teuthology.orchestra.run.smithi076.stdout:1 2024-04-25T19:15:55.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:15:55.256 DEBUG:teuthology.orchestra.run.smithi076:> route 2024-04-25T19:15:55.323 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2024-04-25T19:15:55.323 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T19:15:55.324 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T19:15:55.324 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T19:15:55.324 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T19:15:55.324 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:15:55.324 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-04-25T19:15:55.324 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T19:15:55.324 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T19:15:55.324 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T19:15:55.324 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-04-25T19:15:55.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:15:55.590 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:15:55.593 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:15:55.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:15:55.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:15:55.618 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-04-25T19:15:55.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:15:55.697 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list-id 2024-04-25T19:15:55.752 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:15:55.752 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-04-25T19:15:55.752 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T19:15:55.752 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T19:15:55.752 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-04-25T19:15:55.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:15:55.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:15:55.871 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T19:15:55.871 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:15:55.872 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-04-25T19:15:55.872 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T19:15:55.872 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T19:15:55.872 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T19:15:55.872 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T19:15:55.872 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T19:15:55.872 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-04-25T19:15:55.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:15:55.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:15:56.004 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T19:15:56.004 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-04-25T19:15:56.005 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2024-04-25T19:15:56.005 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T19:15:56.005 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-04-25T19:15:56.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T19:15:56.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:15:56.081 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T19:15:56.081 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T19:15:56.081 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T19:15:56.106 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T19:15:56.106 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T19:15:56.106 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T19:15:56.162 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-04-25T19:15:56.228 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T19:15:56.284 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-04-25T19:15:56.284 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-04-25T19:15:56.284 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-04-25T19:15:56.284 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-04-25T19:15:56.284 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-04-25T19:15:56.284 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-04-25T19:15:56.284 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-04-25T19:15:56.284 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-04-25T19:15:56.284 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-04-25T19:15:56.284 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-04-25T19:15:56.285 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-04-25T19:15:56.285 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2024-04-25T19:15:56.285 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T19:15:56.285 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-04-25T19:15:56.285 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-04-25T19:15:56.285 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-04-25T19:15:56.285 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-04-25T19:15:56.285 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-04-25T19:15:56.285 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-04-25T19:15:56.285 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T19:15:56.285 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T19:15:56.285 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T19:15:56.285 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T19:15:56.285 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T19:15:56.286 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-04-25T19:15:56.286 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-04-25T19:15:56.286 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T19:15:56.286 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T19:15:56.286 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T19:15:56.286 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T19:15:56.286 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-04-25T19:15:56.286 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T19:15:56.286 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T19:15:56.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:15:56.287 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-04-25T19:15:56.342 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T19:15:56.342 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T19:15:56.385 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-04-25T19:15:56.409 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T19:15:56.465 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-04-25T19:15:56.465 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-04-25T19:15:56.465 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-04-25T19:15:56.465 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-04-25T19:15:56.465 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-04-25T19:15:56.465 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-04-25T19:15:56.465 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-04-25T19:15:56.465 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-04-25T19:15:56.465 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-04-25T19:15:56.465 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-04-25T19:15:56.465 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-04-25T19:15:56.466 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/selinux 2024-04-25T19:15:56.466 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T19:15:56.466 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-04-25T19:15:56.466 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-04-25T19:15:56.466 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-04-25T19:15:56.466 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-04-25T19:15:56.466 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-04-25T19:15:56.466 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-04-25T19:15:56.466 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T19:15:56.466 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T19:15:56.466 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T19:15:56.466 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T19:15:56.466 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T19:15:56.466 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-04-25T19:15:56.467 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-04-25T19:15:56.467 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T19:15:56.467 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T19:15:56.467 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T19:15:56.467 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T19:15:56.467 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-04-25T19:15:56.467 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T19:15:56.467 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T19:15:56.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:15:56.468 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-04-25T19:15:56.493 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2024-04-25T19:15:56.491+0000 7f17f531c480 -1 init, newargv = 0x7f17e4052070 newargc=15 2024-04-25T19:15:56.493 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[42247]: starting ceph client 2024-04-25T19:15:56.507 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[42247]: starting fuse 2024-04-25T19:15:56.508 INFO:teuthology.orchestra.run.smithi076.stdout:43 2024-04-25T19:15:56.508 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T19:15:56.508 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2024-04-25T19:15:56.508 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2024-04-25T19:15:56.508 DEBUG:teuthology.orchestra.run.smithi076:> import re 2024-04-25T19:15:56.508 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-04-25T19:15:56.508 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2024-04-25T19:15:56.508 DEBUG:teuthology.orchestra.run.smithi076:> 2024-04-25T19:15:56.509 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2024-04-25T19:15:56.509 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T19:15:56.509 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2024-04-25T19:15:56.509 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T19:15:56.509 DEBUG:teuthology.orchestra.run.smithi076:> 2024-04-25T19:15:56.509 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2024-04-25T19:15:56.509 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2024-04-25T19:15:56.509 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2024-04-25T19:15:56.509 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2024-04-25T19:15:56.509 DEBUG:teuthology.orchestra.run.smithi076:> 2024-04-25T19:15:56.509 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2024-04-25T19:15:56.509 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T19:15:56.509 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T19:15:56.509 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T19:15:56.510 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2024-04-25T19:15:56.510 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2024-04-25T19:15:56.510 DEBUG:teuthology.orchestra.run.smithi076:> return f 2024-04-25T19:15:56.510 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T19:15:56.510 DEBUG:teuthology.orchestra.run.smithi076:> 2024-04-25T19:15:56.510 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.0")) 2024-04-25T19:15:56.510 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-04-25T19:15:56.626 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.0.42247.asok 2024-04-25T19:15:56.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T19:15:56.637 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42247.asok 2024-04-25T19:15:56.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:15:56.637 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42247.asok status 2024-04-25T19:15:56.779 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:15:56.779 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2024-04-25T19:15:56.779 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T19:15:56.779 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T19:15:56.779 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "0", 2024-04-25T19:15:56.779 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2024-04-25T19:15:56.779 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T19:15:56.779 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "42247", 2024-04-25T19:15:56.780 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2024-04-25T19:15:56.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:15:56.780 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2024-04-25T19:15:56.780 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2024-04-25T19:15:56.780 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4806, 2024-04-25T19:15:56.780 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2024-04-25T19:15:56.780 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2024-04-25T19:15:56.780 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2024-04-25T19:15:56.780 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 4806 2024-04-25T19:15:56.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:15:56.780 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2024-04-25T19:15:56.780 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:15:56.780 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2024-04-25T19:15:56.780 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2151666172 2024-04-25T19:15:56.781 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:15:56.781 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:15:56.781 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2024-04-25T19:15:56.781 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:15:56.781 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2024-04-25T19:15:56.781 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2151666172 2024-04-25T19:15:56.781 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:15:56.781 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.4806 192.168.0.1:0/2151666172", 2024-04-25T19:15:56.781 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.1:0/2151666172", 2024-04-25T19:15:56.781 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2024-04-25T19:15:56.781 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 16, 2024-04-25T19:15:56.781 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 21, 2024-04-25T19:15:56.781 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2024-04-25T19:15:56.781 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2024-04-25T19:15:56.781 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2024-04-25T19:15:56.782 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:15:56.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:15:56.788 DEBUG:teuthology.orchestra.run.smithi076:> stat --file-system '--printf=%T 2024-04-25T19:15:56.788 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T19:15:56.803 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2024-04-25T19:15:56.803 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T19:15:56.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:15:56.807 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T19:15:56.869 INFO:teuthology.run_tasks:Running task mon_thrash... 2024-04-25T19:15:56.905 INFO:tasks.mon_thrash:Beginning mon_thrash... 2024-04-25T19:15:56.906 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:15:57.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.016+0000 7f8d1deca640 1 Processor -- start 2024-04-25T19:15:57.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.016+0000 7f8d1deca640 1 -- start start 2024-04-25T19:15:57.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.017+0000 7f8d1deca640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8d1806de20 0x7f8d1806d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:57.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.017+0000 7f8d1deca640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8d180cc9e0 0x7f8d18156f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:57.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.017+0000 7f8d1deca640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d1806e310 con 0x7f8d1815d780 2024-04-25T19:15:57.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.017+0000 7f8d1deca640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f8d18157450 con 0x7f8d1806de20 2024-04-25T19:15:57.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.017+0000 7f8d1deca640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f8d1815d2f0 con 0x7f8d180cc9e0 2024-04-25T19:15:57.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.017+0000 7f8d177fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8d1806de20 0x7f8d1806d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:57.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.017+0000 7f8d177fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8d1806de20 0x7f8d1806d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38048/0 (socket says 172.21.15.76:38048) 2024-04-25T19:15:57.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.017+0000 7f8d177fe640 1 -- 172.21.15.76:0/3287599653 learned_addr learned my addr 172.21.15.76:0/3287599653 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:57.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.017+0000 7f8d16ffd640 1 --2- 172.21.15.76:0/3287599653 >> v2:172.21.15.105:3300/0 conn(0x7f8d180cc9e0 0x7f8d18156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:57.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.017+0000 7f8d177fe640 1 -- 172.21.15.76:0/3287599653 >> v2:172.21.15.105:3300/0 conn(0x7f8d180cc9e0 msgr2=0x7f8d18156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:57.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.017+0000 7f8d177fe640 1 --2- 172.21.15.76:0/3287599653 >> v2:172.21.15.105:3300/0 conn(0x7f8d180cc9e0 0x7f8d18156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.017+0000 7f8d177fe640 1 -- 172.21.15.76:0/3287599653 >> v1:172.21.15.105:6789/0 conn(0x7f8d1815d780 legacy=0x7f8d180cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:57.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.017+0000 7f8d177fe640 1 -- 172.21.15.76:0/3287599653 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d1815ccc0 con 0x7f8d1806de20 2024-04-25T19:15:57.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.018+0000 7f8d177fe640 1 --2- 172.21.15.76:0/3287599653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d1806de20 0x7f8d1806d6d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8d180d0bc0 tx=0x7f8d0802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4410627a8f227353 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:57.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.018+0000 7f8d167fc640 1 -- 172.21.15.76:0/3287599653 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d08037070 con 0x7f8d1806de20 2024-04-25T19:15:57.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.018+0000 7f8d167fc640 1 -- 172.21.15.76:0/3287599653 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d08032440 con 0x7f8d1806de20 2024-04-25T19:15:57.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.018+0000 7f8d167fc640 1 -- 172.21.15.76:0/3287599653 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d08032810 con 0x7f8d1806de20 2024-04-25T19:15:57.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.019+0000 7f8d1deca640 1 -- 172.21.15.76:0/3287599653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d1806de20 msgr2=0x7f8d1806d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:57.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.019+0000 7f8d1deca640 1 --2- 172.21.15.76:0/3287599653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d1806de20 0x7f8d1806d6d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8d180d0bc0 tx=0x7f8d0802a1a0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.019+0000 7f8d1deca640 1 -- 172.21.15.76:0/3287599653 shutdown_connections 2024-04-25T19:15:57.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.019+0000 7f8d1deca640 1 --2- 172.21.15.76:0/3287599653 >> v2:172.21.15.105:3300/0 conn(0x7f8d180cc9e0 0x7f8d18156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.019+0000 7f8d1deca640 1 --2- 172.21.15.76:0/3287599653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d1806de20 0x7f8d1806d6d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.019+0000 7f8d1deca640 1 -- 172.21.15.76:0/3287599653 >> 172.21.15.76:0/3287599653 conn(0x7f8d180684e0 msgr2=0x7f8d180688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:57.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.019+0000 7f8d1deca640 1 -- 172.21.15.76:0/3287599653 shutdown_connections 2024-04-25T19:15:57.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.019+0000 7f8d1deca640 1 -- 172.21.15.76:0/3287599653 wait complete. 2024-04-25T19:15:57.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.021+0000 7f8d1deca640 1 Processor -- start 2024-04-25T19:15:57.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.021+0000 7f8d1deca640 1 -- start start 2024-04-25T19:15:57.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.022+0000 7f8d1deca640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d1806de20 0x7f8d181f6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:57.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.022+0000 7f8d1deca640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d180cc9e0 0x7f8d18155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:57.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.022+0000 7f8d1deca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d1815d780 0x7f8d181f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:57.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.022+0000 7f8d1deca640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1815cf80 con 0x7f8d1815d780 2024-04-25T19:15:57.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.022+0000 7f8d1deca640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8d180cb980 con 0x7f8d180cc9e0 2024-04-25T19:15:57.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.022+0000 7f8d1deca640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8d180cbda0 con 0x7f8d1806de20 2024-04-25T19:15:57.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.022+0000 7f8d17fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d1815d780 0x7f8d181f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:57.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.022+0000 7f8d17fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d1815d780 0x7f8d181f59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38054/0 (socket says 172.21.15.76:38054) 2024-04-25T19:15:57.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.022+0000 7f8d17fff640 1 -- 172.21.15.76:0/3692092634 learned_addr learned my addr 172.21.15.76:0/3692092634 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:57.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.022+0000 7f8d177fe640 1 --2- 172.21.15.76:0/3692092634 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d1806de20 0x7f8d181f6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:57.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.022+0000 7f8d16ffd640 1 --2- 172.21.15.76:0/3692092634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d180cc9e0 0x7f8d18155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:57.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.022+0000 7f8d17fff640 1 -- 172.21.15.76:0/3692092634 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d1806de20 msgr2=0x7f8d181f6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:57.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.022+0000 7f8d17fff640 1 --2- 172.21.15.76:0/3692092634 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d1806de20 0x7f8d181f6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.022+0000 7f8d17fff640 1 -- 172.21.15.76:0/3692092634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d180cc9e0 msgr2=0x7f8d18155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:57.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.022+0000 7f8d17fff640 1 --2- 172.21.15.76:0/3692092634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d180cc9e0 0x7f8d18155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.022+0000 7f8d17fff640 1 -- 172.21.15.76:0/3692092634 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d181f4360 con 0x7f8d1815d780 2024-04-25T19:15:57.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.023+0000 7f8d17fff640 1 --2- 172.21.15.76:0/3692092634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d1815d780 0x7f8d181f59d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8d1000b9a0 tx=0x7f8d1000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:57.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.023+0000 7f8cfffff640 1 -- 172.21.15.76:0/3692092634 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d10019070 con 0x7f8d1815d780 2024-04-25T19:15:57.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.023+0000 7f8cfffff640 1 -- 172.21.15.76:0/3692092634 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d1000cba0 con 0x7f8d1815d780 2024-04-25T19:15:57.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.023+0000 7f8d1deca640 1 -- 172.21.15.76:0/3692092634 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d181f7320 con 0x7f8d1815d780 2024-04-25T19:15:57.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.023+0000 7f8cfffff640 1 -- 172.21.15.76:0/3692092634 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d10014440 con 0x7f8d1815d780 2024-04-25T19:15:57.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.023+0000 7f8d1deca640 1 -- 172.21.15.76:0/3692092634 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d18160b50 con 0x7f8d1815d780 2024-04-25T19:15:57.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.024+0000 7f8d1deca640 1 -- 172.21.15.76:0/3692092634 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d181f4530 con 0x7f8d1815d780 2024-04-25T19:15:57.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.024+0000 7f8cfffff640 1 -- 172.21.15.76:0/3692092634 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8d1001c070 con 0x7f8d1815d780 2024-04-25T19:15:57.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.025+0000 7f8cfffff640 1 --2- 172.21.15.76:0/3692092634 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8d040744b0 0x7f8d040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:57.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.025+0000 7f8cfffff640 1 -- 172.21.15.76:0/3692092634 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8d10094e60 con 0x7f8d1815d780 2024-04-25T19:15:57.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.025+0000 7f8d177fe640 1 --2- 172.21.15.76:0/3692092634 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8d040744b0 0x7f8d040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:57.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.025+0000 7f8d177fe640 1 --2- 172.21.15.76:0/3692092634 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8d040744b0 0x7f8d040769c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8d08034000 tx=0x7f8d0802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:57.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.029+0000 7f8cfffff640 1 -- 172.21.15.76:0/3692092634 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8d1005f710 con 0x7f8d1815d780 2024-04-25T19:15:57.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.206+0000 7f8d1deca640 1 -- 172.21.15.76:0/3692092634 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8d181571f0 con 0x7f8d1815d780 2024-04-25T19:15:57.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.207+0000 7f8cfffff640 1 -- 172.21.15.76:0/3692092634 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7f8d181571f0 con 0x7f8d1815d780 2024-04-25T19:15:57.209 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:57.210 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":21,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","created":"2024-04-25T19:15:01.693137+0000","modified":"2024-04-25T19:15:20.401211+0000","last_up_change":"2024-04-25T19:15:11.891681+0000","last_in_change":"2024-04-25T19:15:03.898065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:15:13.846514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:15:16.606533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:15:17.719197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72a16c02-48f9-45aa-9447-fcf755ebf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409574998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409574998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409574998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409574998}]},"public_addr":"172.21.15.76:6817/1409574998","cluster_addr":"172.21.15.76:6819/1409574998","heartbeat_back_addr":"172.21.15.76:6823/1409574998","heartbeat_front_addr":"172.21.15.76:6821/1409574998","state":["exists","up"]},{"osd":1,"uuid":"818d50f0-76c6-45f8-bf79-5e304f14c5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6801","nonce":1302663821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6803","nonce":1302663821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6807","nonce":1302663821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6805","nonce":1302663821}]},"public_addr":"172.21.15.76:6801/1302663821","cluster_addr":"172.21.15.76:6803/1302663821","heartbeat_back_addr":"172.21.15.76:6807/1302663821","heartbeat_front_addr":"172.21.15.76:6805/1302663821","state":["exists","up"]},{"osd":2,"uuid":"5714c8f5-ae23-473c-8cf1-db73811acded","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6809","nonce":1992938181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6811","nonce":1992938181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6815","nonce":1992938181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6813","nonce":1992938181}]},"public_addr":"172.21.15.76:6809/1992938181","cluster_addr":"172.21.15.76:6811/1992938181","heartbeat_back_addr":"172.21.15.76:6815/1992938181","heartbeat_front_addr":"172.21.15.76:6813/1992938181","state":["exists","up"]},{"osd":3,"uuid":"13204cdb-d8e6-4f97-bc6e-9561f442decf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6825","nonce":3418257163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6827","nonce":3418257163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6831","nonce":3418257163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6829","nonce":3418257163}]},"public_addr":"172.21.15.76:6825/3418257163","cluster_addr":"172.21.15.76:6827/3418257163","heartbeat_back_addr":"172.21.15.76:6831/3418257163","heartbeat_front_addr":"172.21.15.76:6829/3418257163","state":["exists","up"]},{"osd":4,"uuid":"5bea5166-1986-4d56-b5f5-ecc8cbe7f3ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6801","nonce":2125002468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6803","nonce":2125002468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6807","nonce":2125002468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6805","nonce":2125002468}]},"public_addr":"172.21.15.105:6801/2125002468","cluster_addr":"172.21.15.105:6803/2125002468","heartbeat_back_addr":"172.21.15.105:6807/2125002468","heartbeat_front_addr":"172.21.15.105:6805/2125002468","state":["exists","up"]},{"osd":5,"uuid":"96f89dcc-270d-4421-b8e8-3b92ba5f9844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6819","nonce":3413103121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6821","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6823","nonce":3413103121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6831","nonce":3413103121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6828","nonce":3413103121}]},"public_addr":"172.21.15.105:6819/3413103121","cluster_addr":"172.21.15.105:6823/3413103121","heartbeat_back_addr":"172.21.15.105:6831/3413103121","heartbeat_front_addr":"172.21.15.105:6828/3413103121","state":["exists","up"]},{"osd":6,"uuid":"93eb087a-d9ba-4096-b85d-b74d96e819a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6809","nonce":2721832954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6811","nonce":2721832954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6815","nonce":2721832954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6813","nonce":2721832954}]},"public_addr":"172.21.15.105:6809/2721832954","cluster_addr":"172.21.15.105:6811/2721832954","heartbeat_back_addr":"172.21.15.105:6815/2721832954","heartbeat_front_addr":"172.21.15.105:6813/2721832954","state":["exists","up"]},{"osd":7,"uuid":"2f80f790-ad2d-460e-a37d-47bf1771ef43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6818","nonce":548350430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6822","nonce":548350430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6829","nonce":548350430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6825","nonce":548350430}]},"public_addr":"172.21.15.105:6818/548350430","cluster_addr":"172.21.15.105:6822/548350430","heartbeat_back_addr":"172.21.15.105:6829/548350430","heartbeat_front_addr":"172.21.15.105:6825/548350430","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.106901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.072819+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.126865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.053464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:08.982614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.104825+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.112714+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.107387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:15:57.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.209+0000 7f8d1deca640 1 -- 172.21.15.76:0/3692092634 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8d040744b0 msgr2=0x7f8d040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:57.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.209+0000 7f8d1deca640 1 --2- 172.21.15.76:0/3692092634 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8d040744b0 0x7f8d040769c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8d08034000 tx=0x7f8d0802e000 comp rx=0 tx=0).stop 2024-04-25T19:15:57.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.209+0000 7f8d1deca640 1 -- 172.21.15.76:0/3692092634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d1815d780 msgr2=0x7f8d181f59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:57.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.209+0000 7f8d1deca640 1 --2- 172.21.15.76:0/3692092634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d1815d780 0x7f8d181f59d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8d1000b9a0 tx=0x7f8d1000be60 comp rx=0 tx=0).stop 2024-04-25T19:15:57.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.209+0000 7f8d1deca640 1 -- 172.21.15.76:0/3692092634 shutdown_connections 2024-04-25T19:15:57.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.209+0000 7f8d1deca640 1 --2- 172.21.15.76:0/3692092634 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8d040744b0 0x7f8d040769c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.209+0000 7f8d1deca640 1 --2- 172.21.15.76:0/3692092634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d1815d780 0x7f8d181f59d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.209+0000 7f8d1deca640 1 --2- 172.21.15.76:0/3692092634 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8d180cc9e0 0x7f8d18155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.209+0000 7f8d1deca640 1 --2- 172.21.15.76:0/3692092634 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8d1806de20 0x7f8d181f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.209+0000 7f8d1deca640 1 -- 172.21.15.76:0/3692092634 >> 172.21.15.76:0/3692092634 conn(0x7f8d180684e0 msgr2=0x7f8d18075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:57.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.210+0000 7f8d1deca640 1 -- 172.21.15.76:0/3692092634 shutdown_connections 2024-04-25T19:15:57.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.210+0000 7f8d1deca640 1 -- 172.21.15.76:0/3692092634 wait complete. 2024-04-25T19:15:57.222 INFO:tasks.mon_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T19:15:13.846514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T19:15:16.606533+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T19:15:17.719197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T19:15:57.223 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:15:57.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.332+0000 7fa63cf5e640 1 Processor -- start 2024-04-25T19:15:57.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.332+0000 7fa63cf5e640 1 -- start start 2024-04-25T19:15:57.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.333+0000 7fa63cf5e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa6380ca750 0x7fa6380cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:57.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.333+0000 7fa63cf5e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6380cab60 0x7fa6380ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:57.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.333+0000 7fa63cf5e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6380c56a0 con 0x7fa638158060 2024-04-25T19:15:57.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.333+0000 7fa63cf5e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fa6380c5960 con 0x7fa6380ca750 2024-04-25T19:15:57.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.333+0000 7fa63cf5e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa6380c5c40 con 0x7fa6380cab60 2024-04-25T19:15:57.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.333+0000 7fa636d9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6380cab60 0x7fa6380ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:57.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.333+0000 7fa635d9b640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fa6380ca750 0x7fa6380cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:57.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.333+0000 7fa636d9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6380cab60 0x7fa6380ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:44596/0 (socket says 172.21.15.76:44596) 2024-04-25T19:15:57.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.333+0000 7fa636d9d640 1 -- 172.21.15.76:0/2856272536 learned_addr learned my addr 172.21.15.76:0/2856272536 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:57.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.334+0000 7fa636d9d640 1 -- 172.21.15.76:0/2856272536 >> v2:172.21.15.105:3300/0 conn(0x7fa6380ca750 msgr2=0x7fa6380cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:57.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.334+0000 7fa636d9d640 1 --2- 172.21.15.76:0/2856272536 >> v2:172.21.15.105:3300/0 conn(0x7fa6380ca750 0x7fa6380cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.334+0000 7fa636d9d640 1 -- 172.21.15.76:0/2856272536 >> v1:172.21.15.105:6789/0 conn(0x7fa638158060 legacy=0x7fa6380c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:57.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.334+0000 7fa636d9d640 1 -- 172.21.15.76:0/2856272536 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6380c5f20 con 0x7fa6380cab60 2024-04-25T19:15:57.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.334+0000 7fa635d9b640 1 --2- 172.21.15.76:0/2856272536 >> v2:172.21.15.105:3300/0 conn(0x7fa6380ca750 0x7fa6380cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:15:57.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.334+0000 7fa636d9d640 1 --2- 172.21.15.76:0/2856272536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6380cab60 0x7fa6380ccd20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa630009870 tx=0x7fa63002a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5266524e2dda0a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:57.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.335+0000 7fa63559a640 1 -- 172.21.15.76:0/2856272536 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa630037070 con 0x7fa6380cab60 2024-04-25T19:15:57.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.335+0000 7fa63559a640 1 -- 172.21.15.76:0/2856272536 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa63002ae50 con 0x7fa6380cab60 2024-04-25T19:15:57.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.335+0000 7fa63559a640 1 -- 172.21.15.76:0/2856272536 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa630032860 con 0x7fa6380cab60 2024-04-25T19:15:57.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.335+0000 7fa63cf5e640 1 -- 172.21.15.76:0/2856272536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6380cab60 msgr2=0x7fa6380ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:57.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.335+0000 7fa63cf5e640 1 --2- 172.21.15.76:0/2856272536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6380cab60 0x7fa6380ccd20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa630009870 tx=0x7fa63002a300 comp rx=0 tx=0).stop 2024-04-25T19:15:57.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.335+0000 7fa63cf5e640 1 -- 172.21.15.76:0/2856272536 shutdown_connections 2024-04-25T19:15:57.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.335+0000 7fa63cf5e640 1 --2- 172.21.15.76:0/2856272536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6380cab60 0x7fa6380ccd20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.335+0000 7fa63cf5e640 1 --2- 172.21.15.76:0/2856272536 >> v2:172.21.15.105:3300/0 conn(0x7fa6380ca750 0x7fa6380cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.335+0000 7fa63cf5e640 1 -- 172.21.15.76:0/2856272536 >> 172.21.15.76:0/2856272536 conn(0x7fa638068b10 msgr2=0x7fa638072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:57.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.336+0000 7fa63cf5e640 1 -- 172.21.15.76:0/2856272536 shutdown_connections 2024-04-25T19:15:57.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.336+0000 7fa63cf5e640 1 -- 172.21.15.76:0/2856272536 wait complete. 2024-04-25T19:15:57.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.337+0000 7fa63cf5e640 1 Processor -- start 2024-04-25T19:15:57.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.337+0000 7fa63cf5e640 1 -- start start 2024-04-25T19:15:57.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.339+0000 7fa63cf5e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6380ca750 0x7fa638159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:57.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.339+0000 7fa63cf5e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6380cab60 0x7fa638148610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:57.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.339+0000 7fa63cf5e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa638158060 0x7fa638148b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:57.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.339+0000 7fa63cf5e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa63815af80 con 0x7fa638158060 2024-04-25T19:15:57.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.339+0000 7fa63cf5e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa6380c6200 con 0x7fa6380ca750 2024-04-25T19:15:57.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.339+0000 7fa63cf5e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa63815b100 con 0x7fa6380cab60 2024-04-25T19:15:57.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.339+0000 7fa636d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa638158060 0x7fa638148b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:57.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.339+0000 7fa636d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa638158060 0x7fa638148b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38070/0 (socket says 172.21.15.76:38070) 2024-04-25T19:15:57.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.339+0000 7fa636d9d640 1 -- 172.21.15.76:0/1233275542 learned_addr learned my addr 172.21.15.76:0/1233275542 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:57.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.339+0000 7fa63659c640 1 --2- 172.21.15.76:0/1233275542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6380ca750 0x7fa638159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:57.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.339+0000 7fa635d9b640 1 --2- 172.21.15.76:0/1233275542 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6380cab60 0x7fa638148610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:57.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.339+0000 7fa636d9d640 1 -- 172.21.15.76:0/1233275542 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6380cab60 msgr2=0x7fa638148610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:57.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.339+0000 7fa636d9d640 1 --2- 172.21.15.76:0/1233275542 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6380cab60 0x7fa638148610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.339+0000 7fa636d9d640 1 -- 172.21.15.76:0/1233275542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6380ca750 msgr2=0x7fa638159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:57.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.339+0000 7fa636d9d640 1 --2- 172.21.15.76:0/1233275542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6380ca750 0x7fa638159b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.339+0000 7fa636d9d640 1 -- 172.21.15.76:0/1233275542 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6381598a0 con 0x7fa638158060 2024-04-25T19:15:57.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.339+0000 7fa636d9d640 1 --2- 172.21.15.76:0/1233275542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa638158060 0x7fa638148b50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa63002a340 tx=0x7fa6300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:57.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.339+0000 7fa626ffd640 1 -- 172.21.15.76:0/1233275542 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa630041020 con 0x7fa638158060 2024-04-25T19:15:57.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.339+0000 7fa626ffd640 1 -- 172.21.15.76:0/1233275542 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa63003c890 con 0x7fa638158060 2024-04-25T19:15:57.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.340+0000 7fa63cf5e640 1 -- 172.21.15.76:0/1233275542 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa638150c80 con 0x7fa638158060 2024-04-25T19:15:57.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.340+0000 7fa626ffd640 1 -- 172.21.15.76:0/1233275542 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa630037040 con 0x7fa638158060 2024-04-25T19:15:57.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.340+0000 7fa63cf5e640 1 -- 172.21.15.76:0/1233275542 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa638151240 con 0x7fa638158060 2024-04-25T19:15:57.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.340+0000 7fa63cf5e640 1 -- 172.21.15.76:0/1233275542 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa638077e30 con 0x7fa638158060 2024-04-25T19:15:57.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.341+0000 7fa626ffd640 1 -- 172.21.15.76:0/1233275542 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa630044020 con 0x7fa638158060 2024-04-25T19:15:57.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.341+0000 7fa626ffd640 1 --2- 172.21.15.76:0/1233275542 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa6180744b0 0x7fa6180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:57.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.342+0000 7fa626ffd640 1 -- 172.21.15.76:0/1233275542 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa6300b5a90 con 0x7fa638158060 2024-04-25T19:15:57.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.342+0000 7fa63659c640 1 --2- 172.21.15.76:0/1233275542 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa6180744b0 0x7fa6180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:57.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.342+0000 7fa63659c640 1 --2- 172.21.15.76:0/1233275542 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa6180744b0 0x7fa6180769c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa6280012e0 tx=0x7fa628009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:57.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.346+0000 7fa626ffd640 1 -- 172.21.15.76:0/1233275542 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa638077e30 con 0x7fa638158060 2024-04-25T19:15:57.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.524+0000 7fa63cf5e640 1 -- 172.21.15.76:0/1233275542 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa6381553f0 con 0x7fa638158060 2024-04-25T19:15:57.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.525+0000 7fa626ffd640 1 -- 172.21.15.76:0/1233275542 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fa6381553f0 con 0x7fa638158060 2024-04-25T19:15:57.527 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-04-25T19:15:57.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.527+0000 7fa63cf5e640 1 -- 172.21.15.76:0/1233275542 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa6180744b0 msgr2=0x7fa6180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:57.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.527+0000 7fa63cf5e640 1 --2- 172.21.15.76:0/1233275542 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa6180744b0 0x7fa6180769c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa6280012e0 tx=0x7fa628009040 comp rx=0 tx=0).stop 2024-04-25T19:15:57.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.527+0000 7fa63cf5e640 1 -- 172.21.15.76:0/1233275542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa638158060 msgr2=0x7fa638148b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:57.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.527+0000 7fa63cf5e640 1 --2- 172.21.15.76:0/1233275542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa638158060 0x7fa638148b50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa63002a340 tx=0x7fa6300046c0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.527+0000 7fa63cf5e640 1 -- 172.21.15.76:0/1233275542 shutdown_connections 2024-04-25T19:15:57.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.527+0000 7fa63cf5e640 1 --2- 172.21.15.76:0/1233275542 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa6180744b0 0x7fa6180769c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.527+0000 7fa63cf5e640 1 --2- 172.21.15.76:0/1233275542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa638158060 0x7fa638148b50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.527+0000 7fa63cf5e640 1 --2- 172.21.15.76:0/1233275542 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa6380cab60 0x7fa638148610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.527+0000 7fa63cf5e640 1 --2- 172.21.15.76:0/1233275542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa6380ca750 0x7fa638159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.527+0000 7fa63cf5e640 1 -- 172.21.15.76:0/1233275542 >> 172.21.15.76:0/1233275542 conn(0x7fa638068b10 msgr2=0x7fa638152790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:57.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.527+0000 7fa63cf5e640 1 -- 172.21.15.76:0/1233275542 shutdown_connections 2024-04-25T19:15:57.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.527+0000 7fa63cf5e640 1 -- 172.21.15.76:0/1233275542 wait complete. 2024-04-25T19:15:57.539 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T19:15:57.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.647+0000 7fefeb8b6640 1 Processor -- start 2024-04-25T19:15:57.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.648+0000 7fefeb8b6640 1 -- start start 2024-04-25T19:15:57.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.649+0000 7fefeb8b6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fefe40ca750 0x7fefe40cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:57.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.649+0000 7fefeb8b6640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefe41606f0 con 0x7fefe41600b0 2024-04-25T19:15:57.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.649+0000 7fefeb8b6640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefe41609d0 con 0x7fefe40cc6f0 2024-04-25T19:15:57.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.649+0000 7fefeb8b6640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fefe41604c0 con 0x7fefe40ca750 2024-04-25T19:15:57.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.649+0000 7fefe8e51640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fefe40ca750 0x7fefe40cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:57.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.649+0000 7fefe9652640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fefe41600b0 0x7fefe415d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56760/0 (socket says 172.21.15.76:56760) 2024-04-25T19:15:57.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.649+0000 7fefe9652640 1 -- 172.21.15.76:0/2625623745 learned_addr learned my addr 172.21.15.76:0/2625623745 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:57.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.649+0000 7fefe8e51640 1 -- 172.21.15.76:0/2625623745 >> v1:172.21.15.105:6789/0 conn(0x7fefe40cc6f0 legacy=0x7fefe40ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:57.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.649+0000 7fefe8e51640 1 -- 172.21.15.76:0/2625623745 >> v1:172.21.15.76:6789/0 conn(0x7fefe41600b0 legacy=0x7fefe415d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:57.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.649+0000 7fefe8e51640 1 -- 172.21.15.76:0/2625623745 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe4160cb0 con 0x7fefe40ca750 2024-04-25T19:15:57.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.650+0000 7fefe8e51640 1 --2- 172.21.15.76:0/2625623745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefe40ca750 0x7fefe40cab60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fefd40097a0 tx=0x7fefd402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29fa880356058839 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:57.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.650+0000 7fefdbfff640 1 -- 172.21.15.76:0/2625623745 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4037070 con 0x7fefe40ca750 2024-04-25T19:15:57.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.650+0000 7fefdbfff640 1 -- 172.21.15.76:0/2625623745 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefd4032440 con 0x7fefe40ca750 2024-04-25T19:15:57.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.650+0000 7fefdbfff640 1 -- 172.21.15.76:0/2625623745 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4032810 con 0x7fefe40ca750 2024-04-25T19:15:57.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.651+0000 7fefeb8b6640 1 -- 172.21.15.76:0/2625623745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefe40ca750 msgr2=0x7fefe40cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:57.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.651+0000 7fefeb8b6640 1 --2- 172.21.15.76:0/2625623745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefe40ca750 0x7fefe40cab60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fefd40097a0 tx=0x7fefd402a270 comp rx=0 tx=0).stop 2024-04-25T19:15:57.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.651+0000 7fefeb8b6640 1 -- 172.21.15.76:0/2625623745 shutdown_connections 2024-04-25T19:15:57.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.651+0000 7fefeb8b6640 1 --2- 172.21.15.76:0/2625623745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefe40ca750 0x7fefe40cab60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.651+0000 7fefeb8b6640 1 -- 172.21.15.76:0/2625623745 >> 172.21.15.76:0/2625623745 conn(0x7fefe4068b10 msgr2=0x7fefe4072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:57.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.651+0000 7fefeb8b6640 1 -- 172.21.15.76:0/2625623745 shutdown_connections 2024-04-25T19:15:57.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.651+0000 7fefeb8b6640 1 -- 172.21.15.76:0/2625623745 wait complete. 2024-04-25T19:15:57.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.653+0000 7fefeb8b6640 1 Processor -- start 2024-04-25T19:15:57.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.653+0000 7fefeb8b6640 1 -- start start 2024-04-25T19:15:57.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.654+0000 7fefeb8b6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefe40ca750 0x7fefe41f3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:57.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.654+0000 7fefeb8b6640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefe40cc6f0 0x7fefe415c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:57.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.654+0000 7fefeb8b6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefe41600b0 0x7fefe414e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:57.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.654+0000 7fefeb8b6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fefe415ceb0 con 0x7fefe40ca750 2024-04-25T19:15:57.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.654+0000 7fefeb8b6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fefe415dae0 con 0x7fefe41600b0 2024-04-25T19:15:57.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.654+0000 7fefeb8b6640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fefe4160f90 con 0x7fefe40cc6f0 2024-04-25T19:15:57.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.654+0000 7fefe9652640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefe40ca750 0x7fefe41f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:57.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.654+0000 7fefe9652640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefe40ca750 0x7fefe41f3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38090/0 (socket says 172.21.15.76:38090) 2024-04-25T19:15:57.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.654+0000 7fefe9652640 1 -- 172.21.15.76:0/3300336188 learned_addr learned my addr 172.21.15.76:0/3300336188 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:57.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.654+0000 7fefe8e51640 1 --2- 172.21.15.76:0/3300336188 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefe40cc6f0 0x7fefe415c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:57.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.654+0000 7fefe9e53640 1 --2- 172.21.15.76:0/3300336188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefe41600b0 0x7fefe414e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:57.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.654+0000 7fefe9652640 1 -- 172.21.15.76:0/3300336188 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefe40cc6f0 msgr2=0x7fefe415c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:57.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.654+0000 7fefe9652640 1 --2- 172.21.15.76:0/3300336188 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefe40cc6f0 0x7fefe415c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.654+0000 7fefe9652640 1 -- 172.21.15.76:0/3300336188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefe41600b0 msgr2=0x7fefe414e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:57.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.654+0000 7fefe9652640 1 --2- 172.21.15.76:0/3300336188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefe41600b0 0x7fefe414e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.654+0000 7fefe9652640 1 -- 172.21.15.76:0/3300336188 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe41f3c40 con 0x7fefe40ca750 2024-04-25T19:15:57.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.654+0000 7fefe9e53640 1 --2- 172.21.15.76:0/3300336188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefe41600b0 0x7fefe414e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:15:57.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.654+0000 7fefe9652640 1 --2- 172.21.15.76:0/3300336188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefe40ca750 0x7fefe41f3700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fefe000caa0 tx=0x7fefe000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:57.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.655+0000 7fefd9ffb640 1 -- 172.21.15.76:0/3300336188 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe0017070 con 0x7fefe40ca750 2024-04-25T19:15:57.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.655+0000 7fefd9ffb640 1 -- 172.21.15.76:0/3300336188 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe000ac00 con 0x7fefe40ca750 2024-04-25T19:15:57.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.655+0000 7fefd9ffb640 1 -- 172.21.15.76:0/3300336188 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe0012400 con 0x7fefe40ca750 2024-04-25T19:15:57.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.655+0000 7fefeb8b6640 1 -- 172.21.15.76:0/3300336188 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe41f6c50 con 0x7fefe40ca750 2024-04-25T19:15:57.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.655+0000 7fefeb8b6640 1 -- 172.21.15.76:0/3300336188 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe41f7210 con 0x7fefe40ca750 2024-04-25T19:15:57.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.656+0000 7fefeb8b6640 1 -- 172.21.15.76:0/3300336188 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefe406a390 con 0x7fefe40ca750 2024-04-25T19:15:57.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.656+0000 7fefd9ffb640 1 -- 172.21.15.76:0/3300336188 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fefe001a070 con 0x7fefe40ca750 2024-04-25T19:15:57.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.657+0000 7fefd9ffb640 1 --2- 172.21.15.76:0/3300336188 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fefc80744b0 0x7fefc80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:57.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.657+0000 7fefd9ffb640 1 -- 172.21.15.76:0/3300336188 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fefe0094f70 con 0x7fefe40ca750 2024-04-25T19:15:57.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.657+0000 7fefe8e51640 1 --2- 172.21.15.76:0/3300336188 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fefc80744b0 0x7fefc80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:57.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.657+0000 7fefe8e51640 1 --2- 172.21.15.76:0/3300336188 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fefc80744b0 0x7fefc80769c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fefd4034000 tx=0x7fefd402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:57.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.661+0000 7fefd9ffb640 1 -- 172.21.15.76:0/3300336188 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fefe005f700 con 0x7fefe40ca750 2024-04-25T19:15:57.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.837+0000 7fefeb8b6640 1 -- 172.21.15.76:0/3300336188 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fefe40cb1d0 con 0x7fefe40ca750 2024-04-25T19:15:57.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.838+0000 7fefd9ffb640 1 -- 172.21.15.76:0/3300336188 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fefe005f000 con 0x7fefe40ca750 2024-04-25T19:15:57.840 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-04-25T19:15:57.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.840+0000 7fefeb8b6640 1 -- 172.21.15.76:0/3300336188 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fefc80744b0 msgr2=0x7fefc80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:57.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.840+0000 7fefeb8b6640 1 --2- 172.21.15.76:0/3300336188 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fefc80744b0 0x7fefc80769c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fefd4034000 tx=0x7fefd402e000 comp rx=0 tx=0).stop 2024-04-25T19:15:57.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.840+0000 7fefeb8b6640 1 -- 172.21.15.76:0/3300336188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefe40ca750 msgr2=0x7fefe41f3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:57.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.840+0000 7fefeb8b6640 1 --2- 172.21.15.76:0/3300336188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefe40ca750 0x7fefe41f3700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fefe000caa0 tx=0x7fefe000cf60 comp rx=0 tx=0).stop 2024-04-25T19:15:57.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.840+0000 7fefeb8b6640 1 -- 172.21.15.76:0/3300336188 shutdown_connections 2024-04-25T19:15:57.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.840+0000 7fefeb8b6640 1 --2- 172.21.15.76:0/3300336188 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fefc80744b0 0x7fefc80769c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.840+0000 7fefeb8b6640 1 --2- 172.21.15.76:0/3300336188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fefe41600b0 0x7fefe414e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.840+0000 7fefeb8b6640 1 --2- 172.21.15.76:0/3300336188 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fefe40cc6f0 0x7fefe415c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.840+0000 7fefeb8b6640 1 --2- 172.21.15.76:0/3300336188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefe40ca750 0x7fefe41f3700 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.840+0000 7fefeb8b6640 1 -- 172.21.15.76:0/3300336188 >> 172.21.15.76:0/3300336188 conn(0x7fefe4068b10 msgr2=0x7fefe4151ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:57.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.840+0000 7fefeb8b6640 1 -- 172.21.15.76:0/3300336188 shutdown_connections 2024-04-25T19:15:57.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.840+0000 7fefeb8b6640 1 -- 172.21.15.76:0/3300336188 wait complete. 2024-04-25T19:15:57.852 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T19:15:57.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.961+0000 7fb7ebbc7640 1 Processor -- start 2024-04-25T19:15:57.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.961+0000 7fb7ebbc7640 1 -- start start 2024-04-25T19:15:57.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.962+0000 7fb7ebbc7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb7e4157200 0x7fb7e40d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:57.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.962+0000 7fb7ebbc7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb7e40d0160 0x7fb7e40ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:57.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.962+0000 7fb7ebbc7640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7e415dd00 con 0x7fb7e40cfd50 2024-04-25T19:15:57.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.962+0000 7fb7ebbc7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb7e415cea0 con 0x7fb7e40d0160 2024-04-25T19:15:57.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.962+0000 7fb7ebbc7640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fb7e415d180 con 0x7fb7e4157200 2024-04-25T19:15:57.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.963+0000 7fb7ea164640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb7e40d0160 0x7fb7e40ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:57.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.963+0000 7fb7ea164640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb7e40d0160 0x7fb7e40ccb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38104/0 (socket says 172.21.15.76:38104) 2024-04-25T19:15:57.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.963+0000 7fb7ea164640 1 -- 172.21.15.76:0/4181598987 learned_addr learned my addr 172.21.15.76:0/4181598987 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:57.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.963+0000 7fb7e9963640 1 --2- 172.21.15.76:0/4181598987 >> v2:172.21.15.105:3300/0 conn(0x7fb7e4157200 0x7fb7e40d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:57.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.963+0000 7fb7ea164640 1 -- 172.21.15.76:0/4181598987 >> v2:172.21.15.105:3300/0 conn(0x7fb7e4157200 msgr2=0x7fb7e40d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:57.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.963+0000 7fb7ea164640 1 --2- 172.21.15.76:0/4181598987 >> v2:172.21.15.105:3300/0 conn(0x7fb7e4157200 0x7fb7e40d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.963+0000 7fb7ea164640 1 -- 172.21.15.76:0/4181598987 >> v1:172.21.15.105:6789/0 conn(0x7fb7e40cfd50 legacy=0x7fb7e40cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:57.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.963+0000 7fb7ea164640 1 -- 172.21.15.76:0/4181598987 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e415d460 con 0x7fb7e40d0160 2024-04-25T19:15:57.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.964+0000 7fb7ea164640 1 --2- 172.21.15.76:0/4181598987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7e40d0160 0x7fb7e40ccb70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb7e40cfad0 tx=0x7fb7dc02a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182c88d5c6e62f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:57.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.964+0000 7fb7e8961640 1 -- 172.21.15.76:0/4181598987 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc036070 con 0x7fb7e40d0160 2024-04-25T19:15:57.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.964+0000 7fb7e8961640 1 -- 172.21.15.76:0/4181598987 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7dc031600 con 0x7fb7e40d0160 2024-04-25T19:15:57.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.964+0000 7fb7e8961640 1 -- 172.21.15.76:0/4181598987 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc0319d0 con 0x7fb7e40d0160 2024-04-25T19:15:57.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.964+0000 7fb7ebbc7640 1 -- 172.21.15.76:0/4181598987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7e40d0160 msgr2=0x7fb7e40ccb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:57.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.964+0000 7fb7ebbc7640 1 --2- 172.21.15.76:0/4181598987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7e40d0160 0x7fb7e40ccb70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb7e40cfad0 tx=0x7fb7dc02a780 comp rx=0 tx=0).stop 2024-04-25T19:15:57.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.964+0000 7fb7ebbc7640 1 -- 172.21.15.76:0/4181598987 shutdown_connections 2024-04-25T19:15:57.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.964+0000 7fb7ebbc7640 1 --2- 172.21.15.76:0/4181598987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7e40d0160 0x7fb7e40ccb70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.964+0000 7fb7ebbc7640 1 --2- 172.21.15.76:0/4181598987 >> v2:172.21.15.105:3300/0 conn(0x7fb7e4157200 0x7fb7e40d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.964+0000 7fb7ebbc7640 1 -- 172.21.15.76:0/4181598987 >> 172.21.15.76:0/4181598987 conn(0x7fb7e40d2370 msgr2=0x7fb7e40d2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:57.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.965+0000 7fb7ebbc7640 1 -- 172.21.15.76:0/4181598987 shutdown_connections 2024-04-25T19:15:57.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.965+0000 7fb7ebbc7640 1 -- 172.21.15.76:0/4181598987 wait complete. 2024-04-25T19:15:57.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.966+0000 7fb7ebbc7640 1 Processor -- start 2024-04-25T19:15:57.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.966+0000 7fb7ebbc7640 1 -- start start 2024-04-25T19:15:57.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.967+0000 7fb7ebbc7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7e40cfd50 0x7fb7e4157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:57.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.967+0000 7fb7ebbc7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7e40d0160 0x7fb7e41f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:57.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.967+0000 7fb7ebbc7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7e4157200 0x7fb7e4153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:57.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.967+0000 7fb7ebbc7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e40cf5a0 con 0x7fb7e40cfd50 2024-04-25T19:15:57.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.967+0000 7fb7ebbc7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e415da20 con 0x7fb7e40d0160 2024-04-25T19:15:57.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.967+0000 7fb7ebbc7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb7e40cf720 con 0x7fb7e4157200 2024-04-25T19:15:57.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.968+0000 7fb7e9963640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7e40cfd50 0x7fb7e4157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:57.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.968+0000 7fb7e9963640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7e40cfd50 0x7fb7e4157700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38112/0 (socket says 172.21.15.76:38112) 2024-04-25T19:15:57.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.968+0000 7fb7e9963640 1 -- 172.21.15.76:0/3951666852 learned_addr learned my addr 172.21.15.76:0/3951666852 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:57.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.968+0000 7fb7e9162640 1 --2- 172.21.15.76:0/3951666852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7e40d0160 0x7fb7e41f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:57.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.968+0000 7fb7ea164640 1 --2- 172.21.15.76:0/3951666852 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7e4157200 0x7fb7e4153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:57.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.968+0000 7fb7e9963640 1 -- 172.21.15.76:0/3951666852 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7e4157200 msgr2=0x7fb7e4153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:57.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.968+0000 7fb7e9963640 1 --2- 172.21.15.76:0/3951666852 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7e4157200 0x7fb7e4153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.968+0000 7fb7e9963640 1 -- 172.21.15.76:0/3951666852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7e40d0160 msgr2=0x7fb7e41f3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:57.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.968+0000 7fb7e9963640 1 --2- 172.21.15.76:0/3951666852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7e40d0160 0x7fb7e41f3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:57.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.968+0000 7fb7e9963640 1 -- 172.21.15.76:0/3951666852 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e4157c40 con 0x7fb7e40cfd50 2024-04-25T19:15:57.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.968+0000 7fb7ea164640 1 --2- 172.21.15.76:0/3951666852 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7e4157200 0x7fb7e4153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:57.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.968+0000 7fb7e9162640 1 --2- 172.21.15.76:0/3951666852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7e40d0160 0x7fb7e41f3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:57.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.968+0000 7fb7e9963640 1 --2- 172.21.15.76:0/3951666852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7e40cfd50 0x7fb7e4157700 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000d620 tx=0x7fb7e000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:57.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.968+0000 7fb7da7fc640 1 -- 172.21.15.76:0/3951666852 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e000b620 con 0x7fb7e40cfd50 2024-04-25T19:15:57.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.968+0000 7fb7da7fc640 1 -- 172.21.15.76:0/3951666852 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7e0019070 con 0x7fb7e40cfd50 2024-04-25T19:15:57.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.968+0000 7fb7ebbc7640 1 -- 172.21.15.76:0/3951666852 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7e41f41e0 con 0x7fb7e40cfd50 2024-04-25T19:15:57.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.968+0000 7fb7da7fc640 1 -- 172.21.15.76:0/3951666852 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e000ba00 con 0x7fb7e40cfd50 2024-04-25T19:15:57.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.968+0000 7fb7ebbc7640 1 -- 172.21.15.76:0/3951666852 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7e41598b0 con 0x7fb7e40cfd50 2024-04-25T19:15:57.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.969+0000 7fb7ebbc7640 1 -- 172.21.15.76:0/3951666852 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7e41f43b0 con 0x7fb7e40cfd50 2024-04-25T19:15:57.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.970+0000 7fb7da7fc640 1 -- 172.21.15.76:0/3951666852 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb7e000bba0 con 0x7fb7e40cfd50 2024-04-25T19:15:57.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.970+0000 7fb7da7fc640 1 --2- 172.21.15.76:0/3951666852 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb7c80744b0 0x7fb7c80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:57.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.970+0000 7fb7da7fc640 1 -- 172.21.15.76:0/3951666852 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb7e00955e0 con 0x7fb7e40cfd50 2024-04-25T19:15:57.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.971+0000 7fb7e9162640 1 --2- 172.21.15.76:0/3951666852 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb7c80744b0 0x7fb7c80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:57.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.971+0000 7fb7e9162640 1 --2- 172.21.15.76:0/3951666852 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb7c80744b0 0x7fb7c80769c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4006fd0 tx=0x7fb7d4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:57.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:57.975+0000 7fb7da7fc640 1 -- 172.21.15.76:0/3951666852 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb7e005fe00 con 0x7fb7e40cfd50 2024-04-25T19:15:58.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.152+0000 7fb7ebbc7640 1 -- 172.21.15.76:0/3951666852 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb7e40caf00 con 0x7fb7e40cfd50 2024-04-25T19:15:58.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.153+0000 7fb7da7fc640 1 -- 172.21.15.76:0/3951666852 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fb7e40caf00 con 0x7fb7e40cfd50 2024-04-25T19:15:58.154 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-04-25T19:15:58.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.155+0000 7fb7ebbc7640 1 -- 172.21.15.76:0/3951666852 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb7c80744b0 msgr2=0x7fb7c80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:58.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.155+0000 7fb7ebbc7640 1 --2- 172.21.15.76:0/3951666852 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb7c80744b0 0x7fb7c80769c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4006fd0 tx=0x7fb7d4008040 comp rx=0 tx=0).stop 2024-04-25T19:15:58.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.155+0000 7fb7ebbc7640 1 -- 172.21.15.76:0/3951666852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7e40cfd50 msgr2=0x7fb7e4157700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:58.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.155+0000 7fb7ebbc7640 1 --2- 172.21.15.76:0/3951666852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7e40cfd50 0x7fb7e4157700 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000d620 tx=0x7fb7e000dae0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.155+0000 7fb7ebbc7640 1 -- 172.21.15.76:0/3951666852 shutdown_connections 2024-04-25T19:15:58.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.155+0000 7fb7ebbc7640 1 --2- 172.21.15.76:0/3951666852 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb7c80744b0 0x7fb7c80769c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.155+0000 7fb7ebbc7640 1 --2- 172.21.15.76:0/3951666852 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7e4157200 0x7fb7e4153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.155+0000 7fb7ebbc7640 1 --2- 172.21.15.76:0/3951666852 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7e40d0160 0x7fb7e41f3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.155+0000 7fb7ebbc7640 1 --2- 172.21.15.76:0/3951666852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7e40cfd50 0x7fb7e4157700 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.155+0000 7fb7ebbc7640 1 -- 172.21.15.76:0/3951666852 >> 172.21.15.76:0/3951666852 conn(0x7fb7e40d2370 msgr2=0x7fb7e415c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:58.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.155+0000 7fb7ebbc7640 1 -- 172.21.15.76:0/3951666852 shutdown_connections 2024-04-25T19:15:58.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.155+0000 7fb7ebbc7640 1 -- 172.21.15.76:0/3951666852 wait complete. 2024-04-25T19:15:58.167 INFO:tasks.mon_thrash.ceph_manager:waiting for clean 2024-04-25T19:15:58.168 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T19:15:58.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.274+0000 7f0555c40640 1 Processor -- start 2024-04-25T19:15:58.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.274+0000 7f0555c40640 1 -- start start 2024-04-25T19:15:58.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.275+0000 7f0555c40640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f055006e0c0 0x7f055006e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.275+0000 7f0555c40640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05501609c0 0x7f0550160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.276+0000 7f0555c40640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f055015d250 con 0x7f055015cd10 2024-04-25T19:15:58.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.276+0000 7f0555c40640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f055015d510 con 0x7f055006e0c0 2024-04-25T19:15:58.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.276+0000 7f0555c40640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f055015d7f0 con 0x7f05501609c0 2024-04-25T19:15:58.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.276+0000 7f054f7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f055015cd10 0x7f0550156c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56776/0 (socket says 172.21.15.76:56776) 2024-04-25T19:15:58.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.276+0000 7f054f7fe640 1 -- 172.21.15.76:0/76224567 learned_addr learned my addr 172.21.15.76:0/76224567 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:58.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.276+0000 7f054ffff640 1 --2- 172.21.15.76:0/76224567 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05501609c0 0x7f0550160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.276+0000 7f054effd640 1 --2- 172.21.15.76:0/76224567 >> v2:172.21.15.105:3300/0 conn(0x7f055006e0c0 0x7f055006e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.276+0000 7f054e7fc640 1 -- 172.21.15.76:0/76224567 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3314907068 0 0) 0x7f055015d250 con 0x7f055015cd10 2024-04-25T19:15:58.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.276+0000 7f054e7fc640 1 -- 172.21.15.76:0/76224567 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0538003520 con 0x7f055015cd10 2024-04-25T19:15:58.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.276+0000 7f054e7fc640 1 -- 172.21.15.76:0/76224567 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2126704911 0 0) 0x7f0538003520 con 0x7f055015cd10 2024-04-25T19:15:58.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.276+0000 7f054e7fc640 1 -- 172.21.15.76:0/76224567 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05501609c0 msgr2=0x7f0550160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:58.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.276+0000 7f054e7fc640 1 --2- 172.21.15.76:0/76224567 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05501609c0 0x7f0550160dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.276+0000 7f054e7fc640 1 -- 172.21.15.76:0/76224567 >> v2:172.21.15.105:3300/0 conn(0x7f055006e0c0 msgr2=0x7f055006e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:58.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.276+0000 7f054e7fc640 1 --2- 172.21.15.76:0/76224567 >> v2:172.21.15.105:3300/0 conn(0x7f055006e0c0 0x7f055006e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.276+0000 7f054e7fc640 1 -- 172.21.15.76:0/76224567 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055015dad0 con 0x7f055015cd10 2024-04-25T19:15:58.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.276+0000 7f054e7fc640 1 -- 172.21.15.76:0/76224567 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f0540001670 con 0x7f055015cd10 2024-04-25T19:15:58.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.276+0000 7f054e7fc640 1 -- 172.21.15.76:0/76224567 >> v1:172.21.15.76:6789/0 conn(0x7f055015cd10 legacy=0x7f0550156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:58.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.276+0000 7f054e7fc640 1 --2- 172.21.15.76:0/76224567 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05501609c0 0x7f05380039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.276+0000 7f054e7fc640 1 --2- 172.21.15.76:0/76224567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0538003f30 0x7f0538004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.276+0000 7f054e7fc640 1 --2- 172.21.15.76:0/76224567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0538004d40 0x7f0538005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.276+0000 7f054e7fc640 1 -- 172.21.15.76:0/76224567 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f055015d510 con 0x7f0538003f30 2024-04-25T19:15:58.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.276+0000 7f054e7fc640 1 -- 172.21.15.76:0/76224567 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f055015d7f0 con 0x7f0538004d40 2024-04-25T19:15:58.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.276+0000 7f054e7fc640 1 -- 172.21.15.76:0/76224567 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f0538014220 con 0x7f05501609c0 2024-04-25T19:15:58.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.276+0000 7f054effd640 1 --2- 172.21.15.76:0/76224567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0538003f30 0x7f0538004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.276+0000 7f054f7fe640 1 --2- 172.21.15.76:0/76224567 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05501609c0 0x7f05380039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.277+0000 7f054ffff640 1 --2- 172.21.15.76:0/76224567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0538004d40 0x7f0538005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.277+0000 7f054effd640 1 -- 172.21.15.76:0/76224567 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05501609c0 msgr2=0x7f05380039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:58.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.277+0000 7f054effd640 1 --2- 172.21.15.76:0/76224567 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05501609c0 0x7f05380039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.277+0000 7f054effd640 1 -- 172.21.15.76:0/76224567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0538004d40 msgr2=0x7f0538005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:58.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.277+0000 7f054effd640 1 --2- 172.21.15.76:0/76224567 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0538004d40 0x7f0538005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.277+0000 7f054effd640 1 -- 172.21.15.76:0/76224567 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0538003520 con 0x7f0538003f30 2024-04-25T19:15:58.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.277+0000 7f054effd640 1 --2- 172.21.15.76:0/76224567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0538003f30 0x7f0538004800 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f05500cdea0 tx=0x7f0538005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60da8a3653e74133 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:58.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.278+0000 7f054e7fc640 1 -- 172.21.15.76:0/76224567 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0544036070 con 0x7f0538003f30 2024-04-25T19:15:58.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.278+0000 7f054e7fc640 1 -- 172.21.15.76:0/76224567 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0544031400 con 0x7f0538003f30 2024-04-25T19:15:58.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.278+0000 7f054e7fc640 1 -- 172.21.15.76:0/76224567 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0544009690 con 0x7f0538003f30 2024-04-25T19:15:58.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.278+0000 7f0555c40640 1 -- 172.21.15.76:0/76224567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0538003f30 msgr2=0x7f0538004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:58.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.278+0000 7f0555c40640 1 --2- 172.21.15.76:0/76224567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0538003f30 0x7f0538004800 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f05500cdea0 tx=0x7f0538005830 comp rx=0 tx=0).stop 2024-04-25T19:15:58.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.278+0000 7f054ffff640 1 -- 172.21.15.76:0/76224567 reap_dead start 2024-04-25T19:15:58.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.278+0000 7f0555c40640 1 -- 172.21.15.76:0/76224567 shutdown_connections 2024-04-25T19:15:58.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.278+0000 7f0555c40640 1 -- 172.21.15.76:0/76224567 >> 172.21.15.76:0/76224567 conn(0x7f0550070b00 msgr2=0x7f0550070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:58.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.278+0000 7f0555c40640 1 -- 172.21.15.76:0/76224567 shutdown_connections 2024-04-25T19:15:58.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.278+0000 7f0555c40640 1 -- 172.21.15.76:0/76224567 wait complete. 2024-04-25T19:15:58.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.280+0000 7f0555c40640 1 Processor -- start 2024-04-25T19:15:58.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.280+0000 7f0555c40640 1 -- start start 2024-04-25T19:15:58.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.281+0000 7f0555c40640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0538003f30 0x7f05501ef550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.281+0000 7f0555c40640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05500b42d0 0x7f0550159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.281+0000 7f0555c40640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f055015a210 0x7f05501f2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.281+0000 7f0555c40640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f05500c5390 con 0x7f0538003f30 2024-04-25T19:15:58.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.281+0000 7f0555c40640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f05500c5210 con 0x7f055015a210 2024-04-25T19:15:58.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.281+0000 7f0555c40640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f0550160dd0 con 0x7f05500b42d0 2024-04-25T19:15:58.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.281+0000 7f054f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0538003f30 0x7f05501ef550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.282+0000 7f054f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0538003f30 0x7f05501ef550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38138/0 (socket says 172.21.15.76:38138) 2024-04-25T19:15:58.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.282+0000 7f054f7fe640 1 -- 172.21.15.76:0/3761546480 learned_addr learned my addr 172.21.15.76:0/3761546480 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:58.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.282+0000 7f054effd640 1 --2- 172.21.15.76:0/3761546480 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05500b42d0 0x7f0550159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.282+0000 7f054ffff640 1 --2- 172.21.15.76:0/3761546480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f055015a210 0x7f05501f2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.282+0000 7f054f7fe640 1 -- 172.21.15.76:0/3761546480 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05500b42d0 msgr2=0x7f0550159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:58.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.282+0000 7f054f7fe640 1 --2- 172.21.15.76:0/3761546480 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05500b42d0 0x7f0550159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.282+0000 7f054f7fe640 1 -- 172.21.15.76:0/3761546480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f055015a210 msgr2=0x7f05501f2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:58.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.282+0000 7f054f7fe640 1 --2- 172.21.15.76:0/3761546480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f055015a210 0x7f05501f2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.282+0000 7f054f7fe640 1 -- 172.21.15.76:0/3761546480 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05500cea90 con 0x7f0538003f30 2024-04-25T19:15:58.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.282+0000 7f054f7fe640 1 --2- 172.21.15.76:0/3761546480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0538003f30 0x7f05501ef550 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0540007340 tx=0x7f0540007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:58.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.282+0000 7f0537fff640 1 -- 172.21.15.76:0/3761546480 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054000f910 con 0x7f0538003f30 2024-04-25T19:15:58.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.282+0000 7f0537fff640 1 -- 172.21.15.76:0/3761546480 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0540018070 con 0x7f0538003f30 2024-04-25T19:15:58.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.282+0000 7f0555c40640 1 -- 172.21.15.76:0/3761546480 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05501efa90 con 0x7f0538003f30 2024-04-25T19:15:58.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.282+0000 7f0537fff640 1 -- 172.21.15.76:0/3761546480 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054000fcf0 con 0x7f0538003f30 2024-04-25T19:15:58.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.282+0000 7f0555c40640 1 -- 172.21.15.76:0/3761546480 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0550159620 con 0x7f0538003f30 2024-04-25T19:15:58.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.283+0000 7f0555c40640 1 -- 172.21.15.76:0/3761546480 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05500cd330 con 0x7f0538003f30 2024-04-25T19:15:58.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.284+0000 7f0537fff640 1 -- 172.21.15.76:0/3761546480 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f054001d460 con 0x7f0538003f30 2024-04-25T19:15:58.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.284+0000 7f0537fff640 1 --2- 172.21.15.76:0/3761546480 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f053c0744b0 0x7f053c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.284+0000 7f0537fff640 1 -- 172.21.15.76:0/3761546480 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0540095420 con 0x7f0538003f30 2024-04-25T19:15:58.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.284+0000 7f054effd640 1 --2- 172.21.15.76:0/3761546480 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f053c0744b0 0x7f053c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.285+0000 7f054effd640 1 --2- 172.21.15.76:0/3761546480 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f053c0744b0 0x7f053c0769c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0544033000 tx=0x7f0544008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:58.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.289+0000 7f0537fff640 1 -- 172.21.15.76:0/3761546480 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f05500cd330 con 0x7f0538003f30 2024-04-25T19:15:58.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.465+0000 7f0555c40640 1 -- 172.21.15.76:0/3761546480 --> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f055006a390 con 0x7f053c0744b0 2024-04-25T19:15:58.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.472+0000 7f0537fff640 1 -- 172.21.15.76:0/3761546480 <== mgr.4109 v2:172.21.15.105:6832/2585775531 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251992 (secure 0 0 0) 0x7f055006a390 con 0x7f053c0744b0 2024-04-25T19:15:58.477 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:58.478 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-04-25T19:15:58.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.475+0000 7f0555c40640 1 -- 172.21.15.76:0/3761546480 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f053c0744b0 msgr2=0x7f053c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:58.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.475+0000 7f0555c40640 1 --2- 172.21.15.76:0/3761546480 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f053c0744b0 0x7f053c0769c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0544033000 tx=0x7f0544008030 comp rx=0 tx=0).stop 2024-04-25T19:15:58.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.475+0000 7f0555c40640 1 -- 172.21.15.76:0/3761546480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0538003f30 msgr2=0x7f05501ef550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:58.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.475+0000 7f0555c40640 1 --2- 172.21.15.76:0/3761546480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0538003f30 0x7f05501ef550 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0540007340 tx=0x7f0540007800 comp rx=0 tx=0).stop 2024-04-25T19:15:58.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.475+0000 7f0555c40640 1 -- 172.21.15.76:0/3761546480 shutdown_connections 2024-04-25T19:15:58.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.475+0000 7f0555c40640 1 --2- 172.21.15.76:0/3761546480 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f053c0744b0 0x7f053c0769c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.475+0000 7f0555c40640 1 --2- 172.21.15.76:0/3761546480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f055015a210 0x7f05501f2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.475+0000 7f0555c40640 1 --2- 172.21.15.76:0/3761546480 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f05500b42d0 0x7f0550159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.475+0000 7f0555c40640 1 --2- 172.21.15.76:0/3761546480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0538003f30 0x7f05501ef550 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.475+0000 7f0555c40640 1 -- 172.21.15.76:0/3761546480 >> 172.21.15.76:0/3761546480 conn(0x7f0550070b00 msgr2=0x7f05500cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:58.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.476+0000 7f0555c40640 1 -- 172.21.15.76:0/3761546480 shutdown_connections 2024-04-25T19:15:58.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.476+0000 7f0555c40640 1 -- 172.21.15.76:0/3761546480 wait complete. 2024-04-25T19:15:58.488 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-04-25T19:15:57.838727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598402,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":305,"num_write_kb":1690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":250,"ondisk_log_size":250,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218208,"kb_used_data":2080,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756512,"statfs":{"total":773094113280,"available":772870668288,"internally_reserved":0,"allocated":2129920,"data_stored":1801644,"data_compressed":64838,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1406,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003101"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425589+0000","last_change":"2024-04-25T19:15:19.433569+0000","last_active":"2024-04-25T19:15:20.425589+0000","last_peered":"2024-04-25T19:15:20.425589+0000","last_clean":"2024-04-25T19:15:20.425589+0000","last_became_active":"2024-04-25T19:15:19.431332+0000","last_became_peered":"2024-04-25T19:15:19.431332+0000","last_unstale":"2024-04-25T19:15:20.425589+0000","last_undegraded":"2024-04-25T19:15:20.425589+0000","last_fullsized":"2024-04-25T19:15:20.425589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:57:01.213834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404926+0000","last_change":"2024-04-25T19:15:19.418695+0000","last_active":"2024-04-25T19:15:20.404926+0000","last_peered":"2024-04-25T19:15:20.404926+0000","last_clean":"2024-04-25T19:15:20.404926+0000","last_became_active":"2024-04-25T19:15:19.418235+0000","last_became_peered":"2024-04-25T19:15:19.418235+0000","last_unstale":"2024-04-25T19:15:20.404926+0000","last_undegraded":"2024-04-25T19:15:20.404926+0000","last_fullsized":"2024-04-25T19:15:20.404926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:30.516472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804491+0000","last_change":"2024-04-25T19:15:19.416983+0000","last_active":"2024-04-25T19:15:20.804491+0000","last_peered":"2024-04-25T19:15:20.804491+0000","last_clean":"2024-04-25T19:15:20.804491+0000","last_became_active":"2024-04-25T19:15:19.416443+0000","last_became_peered":"2024-04-25T19:15:19.416443+0000","last_unstale":"2024-04-25T19:15:20.804491+0000","last_undegraded":"2024-04-25T19:15:20.804491+0000","last_fullsized":"2024-04-25T19:15:20.804491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:02.794164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848308+0000","last_change":"2024-04-25T19:15:19.429058+0000","last_active":"2024-04-25T19:15:20.848308+0000","last_peered":"2024-04-25T19:15:20.848308+0000","last_clean":"2024-04-25T19:15:20.848308+0000","last_became_active":"2024-04-25T19:15:19.428419+0000","last_became_peered":"2024-04-25T19:15:19.428419+0000","last_unstale":"2024-04-25T19:15:20.848308+0000","last_undegraded":"2024-04-25T19:15:20.848308+0000","last_fullsized":"2024-04-25T19:15:20.848308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:23:56.817616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425402+0000","last_change":"2024-04-25T19:15:19.427615+0000","last_active":"2024-04-25T19:15:20.425402+0000","last_peered":"2024-04-25T19:15:20.425402+0000","last_clean":"2024-04-25T19:15:20.425402+0000","last_became_active":"2024-04-25T19:15:19.426968+0000","last_became_peered":"2024-04-25T19:15:19.426968+0000","last_unstale":"2024-04-25T19:15:20.425402+0000","last_undegraded":"2024-04-25T19:15:20.425402+0000","last_fullsized":"2024-04-25T19:15:20.425402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:42:00.686712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847989+0000","last_change":"2024-04-25T19:15:19.417602+0000","last_active":"2024-04-25T19:15:20.847989+0000","last_peered":"2024-04-25T19:15:20.847989+0000","last_clean":"2024-04-25T19:15:20.847989+0000","last_became_active":"2024-04-25T19:15:19.417292+0000","last_became_peered":"2024-04-25T19:15:19.417292+0000","last_unstale":"2024-04-25T19:15:20.847989+0000","last_undegraded":"2024-04-25T19:15:20.847989+0000","last_fullsized":"2024-04-25T19:15:20.847989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:37:43.583809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425114+0000","last_change":"2024-04-25T19:15:19.428180+0000","last_active":"2024-04-25T19:15:20.425114+0000","last_peered":"2024-04-25T19:15:20.425114+0000","last_clean":"2024-04-25T19:15:20.425114+0000","last_became_active":"2024-04-25T19:15:19.427852+0000","last_became_peered":"2024-04-25T19:15:19.427852+0000","last_unstale":"2024-04-25T19:15:20.425114+0000","last_undegraded":"2024-04-25T19:15:20.425114+0000","last_fullsized":"2024-04-25T19:15:20.425114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:51.300880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405446+0000","last_change":"2024-04-25T19:15:19.418253+0000","last_active":"2024-04-25T19:15:20.405446+0000","last_peered":"2024-04-25T19:15:20.405446+0000","last_clean":"2024-04-25T19:15:20.405446+0000","last_became_active":"2024-04-25T19:15:19.417183+0000","last_became_peered":"2024-04-25T19:15:19.417183+0000","last_unstale":"2024-04-25T19:15:20.405446+0000","last_undegraded":"2024-04-25T19:15:20.405446+0000","last_fullsized":"2024-04-25T19:15:20.405446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:05:13.181646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425578+0000","last_change":"2024-04-25T19:15:19.431671+0000","last_active":"2024-04-25T19:15:20.425578+0000","last_peered":"2024-04-25T19:15:20.425578+0000","last_clean":"2024-04-25T19:15:20.425578+0000","last_became_active":"2024-04-25T19:15:19.431240+0000","last_became_peered":"2024-04-25T19:15:19.431240+0000","last_unstale":"2024-04-25T19:15:20.425578+0000","last_undegraded":"2024-04-25T19:15:20.425578+0000","last_fullsized":"2024-04-25T19:15:20.425578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:22.759462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.406080+0000","last_change":"2024-04-25T19:15:19.416304+0000","last_active":"2024-04-25T19:15:20.406080+0000","last_peered":"2024-04-25T19:15:20.406080+0000","last_clean":"2024-04-25T19:15:20.406080+0000","last_became_active":"2024-04-25T19:15:19.415967+0000","last_became_peered":"2024-04-25T19:15:19.415967+0000","last_unstale":"2024-04-25T19:15:20.406080+0000","last_undegraded":"2024-04-25T19:15:20.406080+0000","last_fullsized":"2024-04-25T19:15:20.406080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:30:52.958731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405773+0000","last_change":"2024-04-25T19:15:19.417596+0000","last_active":"2024-04-25T19:15:20.405773+0000","last_peered":"2024-04-25T19:15:20.405773+0000","last_clean":"2024-04-25T19:15:20.405773+0000","last_became_active":"2024-04-25T19:15:19.417230+0000","last_became_peered":"2024-04-25T19:15:19.417230+0000","last_unstale":"2024-04-25T19:15:20.405773+0000","last_undegraded":"2024-04-25T19:15:20.405773+0000","last_fullsized":"2024-04-25T19:15:20.405773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:28:12.387651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822843+0000","last_change":"2024-04-25T19:15:19.420564+0000","last_active":"2024-04-25T19:15:20.822843+0000","last_peered":"2024-04-25T19:15:20.822843+0000","last_clean":"2024-04-25T19:15:20.822843+0000","last_became_active":"2024-04-25T19:15:19.420044+0000","last_became_peered":"2024-04-25T19:15:19.420044+0000","last_unstale":"2024-04-25T19:15:20.822843+0000","last_undegraded":"2024-04-25T19:15:20.822843+0000","last_fullsized":"2024-04-25T19:15:20.822843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:04.066030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405269+0000","last_change":"2024-04-25T19:15:19.415500+0000","last_active":"2024-04-25T19:15:20.405269+0000","last_peered":"2024-04-25T19:15:20.405269+0000","last_clean":"2024-04-25T19:15:20.405269+0000","last_became_active":"2024-04-25T19:15:19.414007+0000","last_became_peered":"2024-04-25T19:15:19.414007+0000","last_unstale":"2024-04-25T19:15:20.405269+0000","last_undegraded":"2024-04-25T19:15:20.405269+0000","last_fullsized":"2024-04-25T19:15:20.405269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:56:12.320555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804235+0000","last_change":"2024-04-25T19:15:19.423884+0000","last_active":"2024-04-25T19:15:20.804235+0000","last_peered":"2024-04-25T19:15:20.804235+0000","last_clean":"2024-04-25T19:15:20.804235+0000","last_became_active":"2024-04-25T19:15:19.422575+0000","last_became_peered":"2024-04-25T19:15:19.422575+0000","last_unstale":"2024-04-25T19:15:20.804235+0000","last_undegraded":"2024-04-25T19:15:20.804235+0000","last_fullsized":"2024-04-25T19:15:20.804235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:30.988950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404396+0000","last_change":"2024-04-25T19:15:19.424267+0000","last_active":"2024-04-25T19:15:20.404396+0000","last_peered":"2024-04-25T19:15:20.404396+0000","last_clean":"2024-04-25T19:15:20.404396+0000","last_became_active":"2024-04-25T19:15:19.422953+0000","last_became_peered":"2024-04-25T19:15:19.422953+0000","last_unstale":"2024-04-25T19:15:20.404396+0000","last_undegraded":"2024-04-25T19:15:20.404396+0000","last_fullsized":"2024-04-25T19:15:20.404396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:42.823233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847983+0000","last_change":"2024-04-25T19:15:19.420374+0000","last_active":"2024-04-25T19:15:20.847983+0000","last_peered":"2024-04-25T19:15:20.847983+0000","last_clean":"2024-04-25T19:15:20.847983+0000","last_became_active":"2024-04-25T19:15:19.420015+0000","last_became_peered":"2024-04-25T19:15:19.420015+0000","last_unstale":"2024-04-25T19:15:20.847983+0000","last_undegraded":"2024-04-25T19:15:20.847983+0000","last_fullsized":"2024-04-25T19:15:20.847983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:04:26.719359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848705+0000","last_change":"2024-04-25T19:15:19.434476+0000","last_active":"2024-04-25T19:15:20.848705+0000","last_peered":"2024-04-25T19:15:20.848705+0000","last_clean":"2024-04-25T19:15:20.848705+0000","last_became_active":"2024-04-25T19:15:19.433836+0000","last_became_peered":"2024-04-25T19:15:19.433836+0000","last_unstale":"2024-04-25T19:15:20.848705+0000","last_undegraded":"2024-04-25T19:15:20.848705+0000","last_fullsized":"2024-04-25T19:15:20.848705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:13.234829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847359+0000","last_change":"2024-04-25T19:15:19.415521+0000","last_active":"2024-04-25T19:15:20.847359+0000","last_peered":"2024-04-25T19:15:20.847359+0000","last_clean":"2024-04-25T19:15:20.847359+0000","last_became_active":"2024-04-25T19:15:19.415177+0000","last_became_peered":"2024-04-25T19:15:19.415177+0000","last_unstale":"2024-04-25T19:15:20.847359+0000","last_undegraded":"2024-04-25T19:15:20.847359+0000","last_fullsized":"2024-04-25T19:15:20.847359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:24.355619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425590+0000","last_change":"2024-04-25T19:15:19.430291+0000","last_active":"2024-04-25T19:15:20.425590+0000","last_peered":"2024-04-25T19:15:20.425590+0000","last_clean":"2024-04-25T19:15:20.425590+0000","last_became_active":"2024-04-25T19:15:19.427226+0000","last_became_peered":"2024-04-25T19:15:19.427226+0000","last_unstale":"2024-04-25T19:15:20.425590+0000","last_undegraded":"2024-04-25T19:15:20.425590+0000","last_fullsized":"2024-04-25T19:15:20.425590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:38.464211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405750+0000","last_change":"2024-04-25T19:15:19.421696+0000","last_active":"2024-04-25T19:15:20.405750+0000","last_peered":"2024-04-25T19:15:20.405750+0000","last_clean":"2024-04-25T19:15:20.405750+0000","last_became_active":"2024-04-25T19:15:19.421300+0000","last_became_peered":"2024-04-25T19:15:19.421300+0000","last_unstale":"2024-04-25T19:15:20.405750+0000","last_undegraded":"2024-04-25T19:15:20.405750+0000","last_fullsized":"2024-04-25T19:15:20.405750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:15.226398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804410+0000","last_change":"2024-04-25T19:15:19.427880+0000","last_active":"2024-04-25T19:15:20.804410+0000","last_peered":"2024-04-25T19:15:20.804410+0000","last_clean":"2024-04-25T19:15:20.804410+0000","last_became_active":"2024-04-25T19:15:19.427579+0000","last_became_peered":"2024-04-25T19:15:19.427579+0000","last_unstale":"2024-04-25T19:15:20.804410+0000","last_undegraded":"2024-04-25T19:15:20.804410+0000","last_fullsized":"2024-04-25T19:15:20.804410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:47:10.657715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848145+0000","last_change":"2024-04-25T19:15:19.430409+0000","last_active":"2024-04-25T19:15:20.848145+0000","last_peered":"2024-04-25T19:15:20.848145+0000","last_clean":"2024-04-25T19:15:20.848145+0000","last_became_active":"2024-04-25T19:15:19.428305+0000","last_became_peered":"2024-04-25T19:15:19.428305+0000","last_unstale":"2024-04-25T19:15:20.848145+0000","last_undegraded":"2024-04-25T19:15:20.848145+0000","last_fullsized":"2024-04-25T19:15:20.848145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:10:01.268963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405396+0000","last_change":"2024-04-25T19:15:19.424435+0000","last_active":"2024-04-25T19:15:20.405396+0000","last_peered":"2024-04-25T19:15:20.405396+0000","last_clean":"2024-04-25T19:15:20.405396+0000","last_became_active":"2024-04-25T19:15:19.422763+0000","last_became_peered":"2024-04-25T19:15:19.422763+0000","last_unstale":"2024-04-25T19:15:20.405396+0000","last_undegraded":"2024-04-25T19:15:20.405396+0000","last_fullsized":"2024-04-25T19:15:20.405396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:58.092412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848072+0000","last_change":"2024-04-25T19:15:19.420275+0000","last_active":"2024-04-25T19:15:20.848072+0000","last_peered":"2024-04-25T19:15:20.848072+0000","last_clean":"2024-04-25T19:15:20.848072+0000","last_became_active":"2024-04-25T19:15:19.419936+0000","last_became_peered":"2024-04-25T19:15:19.419936+0000","last_unstale":"2024-04-25T19:15:20.848072+0000","last_undegraded":"2024-04-25T19:15:20.848072+0000","last_fullsized":"2024-04-25T19:15:20.848072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:55.710620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405087+0000","last_change":"2024-04-25T19:15:19.423408+0000","last_active":"2024-04-25T19:15:20.405087+0000","last_peered":"2024-04-25T19:15:20.405087+0000","last_clean":"2024-04-25T19:15:20.405087+0000","last_became_active":"2024-04-25T19:15:19.423116+0000","last_became_peered":"2024-04-25T19:15:19.423116+0000","last_unstale":"2024-04-25T19:15:20.405087+0000","last_undegraded":"2024-04-25T19:15:20.405087+0000","last_fullsized":"2024-04-25T19:15:20.405087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:18:57.787013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.406015+0000","last_change":"2024-04-25T19:15:19.428826+0000","last_active":"2024-04-25T19:15:20.406015+0000","last_peered":"2024-04-25T19:15:20.406015+0000","last_clean":"2024-04-25T19:15:20.406015+0000","last_became_active":"2024-04-25T19:15:19.428474+0000","last_became_peered":"2024-04-25T19:15:19.428474+0000","last_unstale":"2024-04-25T19:15:20.406015+0000","last_undegraded":"2024-04-25T19:15:20.406015+0000","last_fullsized":"2024-04-25T19:15:20.406015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:19.974851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404739+0000","last_change":"2024-04-25T19:15:19.423353+0000","last_active":"2024-04-25T19:15:20.404739+0000","last_peered":"2024-04-25T19:15:20.404739+0000","last_clean":"2024-04-25T19:15:20.404739+0000","last_became_active":"2024-04-25T19:15:19.422795+0000","last_became_peered":"2024-04-25T19:15:19.422795+0000","last_unstale":"2024-04-25T19:15:20.404739+0000","last_undegraded":"2024-04-25T19:15:20.404739+0000","last_fullsized":"2024-04-25T19:15:20.404739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:34.119976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405076+0000","last_change":"2024-04-25T19:15:19.421809+0000","last_active":"2024-04-25T19:15:20.405076+0000","last_peered":"2024-04-25T19:15:20.405076+0000","last_clean":"2024-04-25T19:15:20.405076+0000","last_became_active":"2024-04-25T19:15:19.421515+0000","last_became_peered":"2024-04-25T19:15:19.421515+0000","last_unstale":"2024-04-25T19:15:20.405076+0000","last_undegraded":"2024-04-25T19:15:20.405076+0000","last_fullsized":"2024-04-25T19:15:20.405076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:22.603291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804477+0000","last_change":"2024-04-25T19:15:19.410183+0000","last_active":"2024-04-25T19:15:20.804477+0000","last_peered":"2024-04-25T19:15:20.804477+0000","last_clean":"2024-04-25T19:15:20.804477+0000","last_became_active":"2024-04-25T19:15:19.409857+0000","last_became_peered":"2024-04-25T19:15:19.409857+0000","last_unstale":"2024-04-25T19:15:20.804477+0000","last_undegraded":"2024-04-25T19:15:20.804477+0000","last_fullsized":"2024-04-25T19:15:20.804477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:30.180786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848239+0000","last_change":"2024-04-25T19:15:19.425524+0000","last_active":"2024-04-25T19:15:20.848239+0000","last_peered":"2024-04-25T19:15:20.848239+0000","last_clean":"2024-04-25T19:15:20.848239+0000","last_became_active":"2024-04-25T19:15:19.424578+0000","last_became_peered":"2024-04-25T19:15:19.424578+0000","last_unstale":"2024-04-25T19:15:20.848239+0000","last_undegraded":"2024-04-25T19:15:20.848239+0000","last_fullsized":"2024-04-25T19:15:20.848239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:02.103879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847243+0000","last_change":"2024-04-25T19:15:19.426720+0000","last_active":"2024-04-25T19:15:20.847243+0000","last_peered":"2024-04-25T19:15:20.847243+0000","last_clean":"2024-04-25T19:15:20.847243+0000","last_became_active":"2024-04-25T19:15:19.426139+0000","last_became_peered":"2024-04-25T19:15:19.426139+0000","last_unstale":"2024-04-25T19:15:20.847243+0000","last_undegraded":"2024-04-25T19:15:20.847243+0000","last_fullsized":"2024-04-25T19:15:20.847243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:31.608993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405390+0000","last_change":"2024-04-25T19:15:19.413805+0000","last_active":"2024-04-25T19:15:20.405390+0000","last_peered":"2024-04-25T19:15:20.405390+0000","last_clean":"2024-04-25T19:15:20.405390+0000","last_became_active":"2024-04-25T19:15:19.413455+0000","last_became_peered":"2024-04-25T19:15:19.413455+0000","last_unstale":"2024-04-25T19:15:20.405390+0000","last_undegraded":"2024-04-25T19:15:20.405390+0000","last_fullsized":"2024-04-25T19:15:20.405390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:05:45.008076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822709+0000","last_change":"2024-04-25T19:15:19.423770+0000","last_active":"2024-04-25T19:15:20.822709+0000","last_peered":"2024-04-25T19:15:20.822709+0000","last_clean":"2024-04-25T19:15:20.822709+0000","last_became_active":"2024-04-25T19:15:19.423406+0000","last_became_peered":"2024-04-25T19:15:19.423406+0000","last_unstale":"2024-04-25T19:15:20.822709+0000","last_undegraded":"2024-04-25T19:15:20.822709+0000","last_fullsized":"2024-04-25T19:15:20.822709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:41:23.669832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:29.482042+0000","last_change":"2024-04-25T19:15:18.422035+0000","last_active":"2024-04-25T19:15:29.482042+0000","last_peered":"2024-04-25T19:15:29.482042+0000","last_clean":"2024-04-25T19:15:29.482042+0000","last_became_active":"2024-04-25T19:15:18.421658+0000","last_became_peered":"2024-04-25T19:15:18.421658+0000","last_unstale":"2024-04-25T19:15:29.482042+0000","last_undegraded":"2024-04-25T19:15:29.482042+0000","last_fullsized":"2024-04-25T19:15:29.482042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:23.349126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804571+0000","last_change":"2024-04-25T19:15:19.421805+0000","last_active":"2024-04-25T19:15:20.804571+0000","last_peered":"2024-04-25T19:15:20.804571+0000","last_clean":"2024-04-25T19:15:20.804571+0000","last_became_active":"2024-04-25T19:15:19.421497+0000","last_became_peered":"2024-04-25T19:15:19.421497+0000","last_unstale":"2024-04-25T19:15:20.804571+0000","last_undegraded":"2024-04-25T19:15:20.804571+0000","last_fullsized":"2024-04-25T19:15:20.804571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:29:29.949508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'7","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:41.495006+0000","last_change":"2024-04-25T19:15:18.416894+0000","last_active":"2024-04-25T19:15:41.495006+0000","last_peered":"2024-04-25T19:15:41.495006+0000","last_clean":"2024-04-25T19:15:41.495006+0000","last_became_active":"2024-04-25T19:15:18.416539+0000","last_became_peered":"2024-04-25T19:15:18.416539+0000","last_unstale":"2024-04-25T19:15:41.495006+0000","last_undegraded":"2024-04-25T19:15:41.495006+0000","last_fullsized":"2024-04-25T19:15:41.495006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:30.388606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:35.471396+0000","last_change":"2024-04-25T19:15:18.420129+0000","last_active":"2024-04-25T19:15:35.471396+0000","last_peered":"2024-04-25T19:15:35.471396+0000","last_clean":"2024-04-25T19:15:35.471396+0000","last_became_active":"2024-04-25T19:15:18.419732+0000","last_became_peered":"2024-04-25T19:15:18.419732+0000","last_unstale":"2024-04-25T19:15:35.471396+0000","last_undegraded":"2024-04-25T19:15:35.471396+0000","last_fullsized":"2024-04-25T19:15:35.471396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:41:48.673989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405615+0000","last_change":"2024-04-25T19:15:19.415655+0000","last_active":"2024-04-25T19:15:20.405615+0000","last_peered":"2024-04-25T19:15:20.405615+0000","last_clean":"2024-04-25T19:15:20.405615+0000","last_became_active":"2024-04-25T19:15:19.415287+0000","last_became_peered":"2024-04-25T19:15:19.415287+0000","last_unstale":"2024-04-25T19:15:20.405615+0000","last_undegraded":"2024-04-25T19:15:20.405615+0000","last_fullsized":"2024-04-25T19:15:20.405615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:49.155289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:41.492883+0000","last_change":"2024-04-25T19:15:18.420711+0000","last_active":"2024-04-25T19:15:41.492883+0000","last_peered":"2024-04-25T19:15:41.492883+0000","last_clean":"2024-04-25T19:15:41.492883+0000","last_became_active":"2024-04-25T19:15:18.420362+0000","last_became_peered":"2024-04-25T19:15:18.420362+0000","last_unstale":"2024-04-25T19:15:41.492883+0000","last_undegraded":"2024-04-25T19:15:41.492883+0000","last_fullsized":"2024-04-25T19:15:41.492883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:48.245193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404375+0000","last_change":"2024-04-25T19:15:19.424356+0000","last_active":"2024-04-25T19:15:20.404375+0000","last_peered":"2024-04-25T19:15:20.404375+0000","last_clean":"2024-04-25T19:15:20.404375+0000","last_became_active":"2024-04-25T19:15:19.423556+0000","last_became_peered":"2024-04-25T19:15:19.423556+0000","last_unstale":"2024-04-25T19:15:20.404375+0000","last_undegraded":"2024-04-25T19:15:20.404375+0000","last_fullsized":"2024-04-25T19:15:20.404375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:19:35.427261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:29.476002+0000","last_change":"2024-04-25T19:15:18.419250+0000","last_active":"2024-04-25T19:15:29.476002+0000","last_peered":"2024-04-25T19:15:29.476002+0000","last_clean":"2024-04-25T19:15:29.476002+0000","last_became_active":"2024-04-25T19:15:18.418927+0000","last_became_peered":"2024-04-25T19:15:18.418927+0000","last_unstale":"2024-04-25T19:15:29.476002+0000","last_undegraded":"2024-04-25T19:15:29.476002+0000","last_fullsized":"2024-04-25T19:15:29.476002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:26.012965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848606+0000","last_change":"2024-04-25T19:15:19.430117+0000","last_active":"2024-04-25T19:15:20.848606+0000","last_peered":"2024-04-25T19:15:20.848606+0000","last_clean":"2024-04-25T19:15:20.848606+0000","last_became_active":"2024-04-25T19:15:19.428985+0000","last_became_peered":"2024-04-25T19:15:19.428985+0000","last_unstale":"2024-04-25T19:15:20.848606+0000","last_undegraded":"2024-04-25T19:15:20.848606+0000","last_fullsized":"2024-04-25T19:15:20.848606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:36:29.069541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:47.499556+0000","last_change":"2024-04-25T19:15:18.422931+0000","last_active":"2024-04-25T19:15:47.499556+0000","last_peered":"2024-04-25T19:15:47.499556+0000","last_clean":"2024-04-25T19:15:47.499556+0000","last_became_active":"2024-04-25T19:15:18.422610+0000","last_became_peered":"2024-04-25T19:15:18.422610+0000","last_unstale":"2024-04-25T19:15:47.499556+0000","last_undegraded":"2024-04-25T19:15:47.499556+0000","last_fullsized":"2024-04-25T19:15:47.499556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:18.489484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804598+0000","last_change":"2024-04-25T19:15:19.416109+0000","last_active":"2024-04-25T19:15:20.804598+0000","last_peered":"2024-04-25T19:15:20.804598+0000","last_clean":"2024-04-25T19:15:20.804598+0000","last_became_active":"2024-04-25T19:15:19.415734+0000","last_became_peered":"2024-04-25T19:15:19.415734+0000","last_unstale":"2024-04-25T19:15:20.804598+0000","last_undegraded":"2024-04-25T19:15:20.804598+0000","last_fullsized":"2024-04-25T19:15:20.804598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:57:31.406303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:29.488292+0000","last_change":"2024-04-25T19:15:18.428677+0000","last_active":"2024-04-25T19:15:29.488292+0000","last_peered":"2024-04-25T19:15:29.488292+0000","last_clean":"2024-04-25T19:15:29.488292+0000","last_became_active":"2024-04-25T19:15:18.428432+0000","last_became_peered":"2024-04-25T19:15:18.428432+0000","last_unstale":"2024-04-25T19:15:29.488292+0000","last_undegraded":"2024-04-25T19:15:29.488292+0000","last_fullsized":"2024-04-25T19:15:29.488292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:38.478595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822814+0000","last_change":"2024-04-25T19:15:19.428245+0000","last_active":"2024-04-25T19:15:20.822814+0000","last_peered":"2024-04-25T19:15:20.822814+0000","last_clean":"2024-04-25T19:15:20.822814+0000","last_became_active":"2024-04-25T19:15:19.427941+0000","last_became_peered":"2024-04-25T19:15:19.427941+0000","last_unstale":"2024-04-25T19:15:20.822814+0000","last_undegraded":"2024-04-25T19:15:20.822814+0000","last_fullsized":"2024-04-25T19:15:20.822814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:09.947923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'10","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:47.500021+0000","last_change":"2024-04-25T19:15:18.420881+0000","last_active":"2024-04-25T19:15:47.500021+0000","last_peered":"2024-04-25T19:15:47.500021+0000","last_clean":"2024-04-25T19:15:47.500021+0000","last_became_active":"2024-04-25T19:15:18.420303+0000","last_became_peered":"2024-04-25T19:15:18.420303+0000","last_unstale":"2024-04-25T19:15:47.500021+0000","last_undegraded":"2024-04-25T19:15:47.500021+0000","last_fullsized":"2024-04-25T19:15:47.500021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:39:24.356298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425356+0000","last_change":"2024-04-25T19:15:19.432735+0000","last_active":"2024-04-25T19:15:20.425356+0000","last_peered":"2024-04-25T19:15:20.425356+0000","last_clean":"2024-04-25T19:15:20.425356+0000","last_became_active":"2024-04-25T19:15:19.431828+0000","last_became_peered":"2024-04-25T19:15:19.431828+0000","last_unstale":"2024-04-25T19:15:20.425356+0000","last_undegraded":"2024-04-25T19:15:20.425356+0000","last_fullsized":"2024-04-25T19:15:20.425356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:20.189230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'9","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:47.496364+0000","last_change":"2024-04-25T19:15:18.421008+0000","last_active":"2024-04-25T19:15:47.496364+0000","last_peered":"2024-04-25T19:15:47.496364+0000","last_clean":"2024-04-25T19:15:47.496364+0000","last_became_active":"2024-04-25T19:15:18.420402+0000","last_became_peered":"2024-04-25T19:15:18.420402+0000","last_unstale":"2024-04-25T19:15:47.496364+0000","last_undegraded":"2024-04-25T19:15:47.496364+0000","last_fullsized":"2024-04-25T19:15:47.496364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:05:28.723958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804304+0000","last_change":"2024-04-25T19:15:19.424805+0000","last_active":"2024-04-25T19:15:20.804304+0000","last_peered":"2024-04-25T19:15:20.804304+0000","last_clean":"2024-04-25T19:15:20.804304+0000","last_became_active":"2024-04-25T19:15:19.424512+0000","last_became_peered":"2024-04-25T19:15:19.424512+0000","last_unstale":"2024-04-25T19:15:20.804304+0000","last_undegraded":"2024-04-25T19:15:20.804304+0000","last_fullsized":"2024-04-25T19:15:20.804304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:50.423186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:41.492047+0000","last_change":"2024-04-25T19:15:18.426805+0000","last_active":"2024-04-25T19:15:41.492047+0000","last_peered":"2024-04-25T19:15:41.492047+0000","last_clean":"2024-04-25T19:15:41.492047+0000","last_became_active":"2024-04-25T19:15:18.426472+0000","last_became_peered":"2024-04-25T19:15:18.426472+0000","last_unstale":"2024-04-25T19:15:41.492047+0000","last_undegraded":"2024-04-25T19:15:41.492047+0000","last_fullsized":"2024-04-25T19:15:41.492047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:29.373363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425459+0000","last_change":"2024-04-25T19:15:19.428780+0000","last_active":"2024-04-25T19:15:20.425459+0000","last_peered":"2024-04-25T19:15:20.425459+0000","last_clean":"2024-04-25T19:15:20.425459+0000","last_became_active":"2024-04-25T19:15:19.428450+0000","last_became_peered":"2024-04-25T19:15:19.428450+0000","last_unstale":"2024-04-25T19:15:20.425459+0000","last_undegraded":"2024-04-25T19:15:20.425459+0000","last_fullsized":"2024-04-25T19:15:20.425459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:01.203880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:41.492432+0000","last_change":"2024-04-25T19:15:18.417740+0000","last_active":"2024-04-25T19:15:41.492432+0000","last_peered":"2024-04-25T19:15:41.492432+0000","last_clean":"2024-04-25T19:15:41.492432+0000","last_became_active":"2024-04-25T19:15:18.417404+0000","last_became_peered":"2024-04-25T19:15:18.417404+0000","last_unstale":"2024-04-25T19:15:41.492432+0000","last_undegraded":"2024-04-25T19:15:41.492432+0000","last_fullsized":"2024-04-25T19:15:41.492432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:21.737283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405391+0000","last_change":"2024-04-25T19:15:19.411214+0000","last_active":"2024-04-25T19:15:20.405391+0000","last_peered":"2024-04-25T19:15:20.405391+0000","last_clean":"2024-04-25T19:15:20.405391+0000","last_became_active":"2024-04-25T19:15:19.410814+0000","last_became_peered":"2024-04-25T19:15:19.410814+0000","last_unstale":"2024-04-25T19:15:20.405391+0000","last_undegraded":"2024-04-25T19:15:20.405391+0000","last_fullsized":"2024-04-25T19:15:20.405391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:42.430277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'17","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:41.489606+0000","last_change":"2024-04-25T19:15:18.422226+0000","last_active":"2024-04-25T19:15:41.489606+0000","last_peered":"2024-04-25T19:15:41.489606+0000","last_clean":"2024-04-25T19:15:41.489606+0000","last_became_active":"2024-04-25T19:15:18.421900+0000","last_became_peered":"2024-04-25T19:15:18.421900+0000","last_unstale":"2024-04-25T19:15:41.489606+0000","last_undegraded":"2024-04-25T19:15:41.489606+0000","last_fullsized":"2024-04-25T19:15:41.489606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:18.040114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425347+0000","last_change":"2024-04-25T19:15:19.428885+0000","last_active":"2024-04-25T19:15:20.425347+0000","last_peered":"2024-04-25T19:15:20.425347+0000","last_clean":"2024-04-25T19:15:20.425347+0000","last_became_active":"2024-04-25T19:15:19.428563+0000","last_became_peered":"2024-04-25T19:15:19.428563+0000","last_unstale":"2024-04-25T19:15:20.425347+0000","last_undegraded":"2024-04-25T19:15:20.425347+0000","last_fullsized":"2024-04-25T19:15:20.425347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:43.469189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:24.414802+0000","last_change":"2024-04-25T19:15:18.422125+0000","last_active":"2024-04-25T19:15:24.414802+0000","last_peered":"2024-04-25T19:15:24.414802+0000","last_clean":"2024-04-25T19:15:24.414802+0000","last_became_active":"2024-04-25T19:15:18.421777+0000","last_became_peered":"2024-04-25T19:15:18.421777+0000","last_unstale":"2024-04-25T19:15:24.414802+0000","last_undegraded":"2024-04-25T19:15:24.414802+0000","last_fullsized":"2024-04-25T19:15:24.414802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:24.289010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425489+0000","last_change":"2024-04-25T19:15:19.429675+0000","last_active":"2024-04-25T19:15:20.425489+0000","last_peered":"2024-04-25T19:15:20.425489+0000","last_clean":"2024-04-25T19:15:20.425489+0000","last_became_active":"2024-04-25T19:15:19.429316+0000","last_became_peered":"2024-04-25T19:15:19.429316+0000","last_unstale":"2024-04-25T19:15:20.425489+0000","last_undegraded":"2024-04-25T19:15:20.425489+0000","last_fullsized":"2024-04-25T19:15:20.425489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:09:55.521944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:47.495217+0000","last_change":"2024-04-25T19:15:18.418661+0000","last_active":"2024-04-25T19:15:47.495217+0000","last_peered":"2024-04-25T19:15:47.495217+0000","last_clean":"2024-04-25T19:15:47.495217+0000","last_became_active":"2024-04-25T19:15:18.418359+0000","last_became_peered":"2024-04-25T19:15:18.418359+0000","last_unstale":"2024-04-25T19:15:47.495217+0000","last_undegraded":"2024-04-25T19:15:47.495217+0000","last_fullsized":"2024-04-25T19:15:47.495217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:49:22.023652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425687+0000","last_change":"2024-04-25T19:15:19.434853+0000","last_active":"2024-04-25T19:15:20.425687+0000","last_peered":"2024-04-25T19:15:20.425687+0000","last_clean":"2024-04-25T19:15:20.425687+0000","last_became_active":"2024-04-25T19:15:19.434508+0000","last_became_peered":"2024-04-25T19:15:19.434508+0000","last_unstale":"2024-04-25T19:15:20.425687+0000","last_undegraded":"2024-04-25T19:15:20.425687+0000","last_fullsized":"2024-04-25T19:15:20.425687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:52.965864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:41.494028+0000","last_change":"2024-04-25T19:15:18.428626+0000","last_active":"2024-04-25T19:15:41.494028+0000","last_peered":"2024-04-25T19:15:41.494028+0000","last_clean":"2024-04-25T19:15:41.494028+0000","last_became_active":"2024-04-25T19:15:18.428332+0000","last_became_peered":"2024-04-25T19:15:18.428332+0000","last_unstale":"2024-04-25T19:15:41.494028+0000","last_undegraded":"2024-04-25T19:15:41.494028+0000","last_fullsized":"2024-04-25T19:15:41.494028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:15.595305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848535+0000","last_change":"2024-04-25T19:15:19.428754+0000","last_active":"2024-04-25T19:15:20.848535+0000","last_peered":"2024-04-25T19:15:20.848535+0000","last_clean":"2024-04-25T19:15:20.848535+0000","last_became_active":"2024-04-25T19:15:19.427992+0000","last_became_peered":"2024-04-25T19:15:19.427992+0000","last_unstale":"2024-04-25T19:15:20.848535+0000","last_undegraded":"2024-04-25T19:15:20.848535+0000","last_fullsized":"2024-04-25T19:15:20.848535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:43:48.443923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:41.491818+0000","last_change":"2024-04-25T19:15:18.419645+0000","last_active":"2024-04-25T19:15:41.491818+0000","last_peered":"2024-04-25T19:15:41.491818+0000","last_clean":"2024-04-25T19:15:41.491818+0000","last_became_active":"2024-04-25T19:15:18.419286+0000","last_became_peered":"2024-04-25T19:15:18.419286+0000","last_unstale":"2024-04-25T19:15:41.491818+0000","last_undegraded":"2024-04-25T19:15:41.491818+0000","last_fullsized":"2024-04-25T19:15:41.491818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:08.295208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425261+0000","last_change":"2024-04-25T19:15:19.432395+0000","last_active":"2024-04-25T19:15:20.425261+0000","last_peered":"2024-04-25T19:15:20.425261+0000","last_clean":"2024-04-25T19:15:20.425261+0000","last_became_active":"2024-04-25T19:15:19.431739+0000","last_became_peered":"2024-04-25T19:15:19.431739+0000","last_unstale":"2024-04-25T19:15:20.425261+0000","last_undegraded":"2024-04-25T19:15:20.425261+0000","last_fullsized":"2024-04-25T19:15:20.425261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:50:03.964030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":110,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425087+0000","last_change":"2024-04-25T19:15:15.604825+0000","last_active":"2024-04-25T19:15:20.425087+0000","last_peered":"2024-04-25T19:15:20.425087+0000","last_clean":"2024-04-25T19:15:20.425087+0000","last_became_active":"2024-04-25T19:15:15.603730+0000","last_became_peered":"2024-04-25T19:15:15.603730+0000","last_unstale":"2024-04-25T19:15:20.425087+0000","last_undegraded":"2024-04-25T19:15:20.425087+0000","last_fullsized":"2024-04-25T19:15:20.425087+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:14.367890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:14.367890+0000","last_clean_scrub_stamp":"2024-04-25T19:15:14.367890+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:02.411899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:41.491282+0000","last_change":"2024-04-25T19:15:18.426331+0000","last_active":"2024-04-25T19:15:41.491282+0000","last_peered":"2024-04-25T19:15:41.491282+0000","last_clean":"2024-04-25T19:15:41.491282+0000","last_became_active":"2024-04-25T19:15:18.425993+0000","last_became_peered":"2024-04-25T19:15:18.425993+0000","last_unstale":"2024-04-25T19:15:41.491282+0000","last_undegraded":"2024-04-25T19:15:41.491282+0000","last_fullsized":"2024-04-25T19:15:41.491282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:17.440484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405021+0000","last_change":"2024-04-25T19:15:19.417337+0000","last_active":"2024-04-25T19:15:20.405021+0000","last_peered":"2024-04-25T19:15:20.405021+0000","last_clean":"2024-04-25T19:15:20.405021+0000","last_became_active":"2024-04-25T19:15:19.416808+0000","last_became_peered":"2024-04-25T19:15:19.416808+0000","last_unstale":"2024-04-25T19:15:20.405021+0000","last_undegraded":"2024-04-25T19:15:20.405021+0000","last_fullsized":"2024-04-25T19:15:20.405021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:57.797904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:35.470254+0000","last_change":"2024-04-25T19:15:18.426358+0000","last_active":"2024-04-25T19:15:35.470254+0000","last_peered":"2024-04-25T19:15:35.470254+0000","last_clean":"2024-04-25T19:15:35.470254+0000","last_became_active":"2024-04-25T19:15:18.426037+0000","last_became_peered":"2024-04-25T19:15:18.426037+0000","last_unstale":"2024-04-25T19:15:35.470254+0000","last_undegraded":"2024-04-25T19:15:35.470254+0000","last_fullsized":"2024-04-25T19:15:35.470254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:26:28.945667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804206+0000","last_change":"2024-04-25T19:15:19.426219+0000","last_active":"2024-04-25T19:15:20.804206+0000","last_peered":"2024-04-25T19:15:20.804206+0000","last_clean":"2024-04-25T19:15:20.804206+0000","last_became_active":"2024-04-25T19:15:19.425626+0000","last_became_peered":"2024-04-25T19:15:19.425626+0000","last_unstale":"2024-04-25T19:15:20.804206+0000","last_undegraded":"2024-04-25T19:15:20.804206+0000","last_fullsized":"2024-04-25T19:15:20.804206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:58.372454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:35.470403+0000","last_change":"2024-04-25T19:15:18.424560+0000","last_active":"2024-04-25T19:15:35.470403+0000","last_peered":"2024-04-25T19:15:35.470403+0000","last_clean":"2024-04-25T19:15:35.470403+0000","last_became_active":"2024-04-25T19:15:18.424179+0000","last_became_peered":"2024-04-25T19:15:18.424179+0000","last_unstale":"2024-04-25T19:15:35.470403+0000","last_undegraded":"2024-04-25T19:15:35.470403+0000","last_fullsized":"2024-04-25T19:15:35.470403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:06.184399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404401+0000","last_change":"2024-04-25T19:15:19.415219+0000","last_active":"2024-04-25T19:15:20.404401+0000","last_peered":"2024-04-25T19:15:20.404401+0000","last_clean":"2024-04-25T19:15:20.404401+0000","last_became_active":"2024-04-25T19:15:19.414877+0000","last_became_peered":"2024-04-25T19:15:19.414877+0000","last_unstale":"2024-04-25T19:15:20.404401+0000","last_undegraded":"2024-04-25T19:15:20.404401+0000","last_fullsized":"2024-04-25T19:15:20.404401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:15.063419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:47.497653+0000","last_change":"2024-04-25T19:15:18.423402+0000","last_active":"2024-04-25T19:15:47.497653+0000","last_peered":"2024-04-25T19:15:47.497653+0000","last_clean":"2024-04-25T19:15:47.497653+0000","last_became_active":"2024-04-25T19:15:18.422330+0000","last_became_peered":"2024-04-25T19:15:18.422330+0000","last_unstale":"2024-04-25T19:15:47.497653+0000","last_undegraded":"2024-04-25T19:15:47.497653+0000","last_fullsized":"2024-04-25T19:15:47.497653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:39:17.512502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822708+0000","last_change":"2024-04-25T19:15:19.421096+0000","last_active":"2024-04-25T19:15:20.822708+0000","last_peered":"2024-04-25T19:15:20.822708+0000","last_clean":"2024-04-25T19:15:20.822708+0000","last_became_active":"2024-04-25T19:15:19.420598+0000","last_became_peered":"2024-04-25T19:15:19.420598+0000","last_unstale":"2024-04-25T19:15:20.822708+0000","last_undegraded":"2024-04-25T19:15:20.822708+0000","last_fullsized":"2024-04-25T19:15:20.822708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:20.733888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404677+0000","last_change":"2024-04-25T19:15:18.422979+0000","last_active":"2024-04-25T19:15:20.404677+0000","last_peered":"2024-04-25T19:15:20.404677+0000","last_clean":"2024-04-25T19:15:20.404677+0000","last_became_active":"2024-04-25T19:15:18.422621+0000","last_became_peered":"2024-04-25T19:15:18.422621+0000","last_unstale":"2024-04-25T19:15:20.404677+0000","last_undegraded":"2024-04-25T19:15:20.404677+0000","last_fullsized":"2024-04-25T19:15:20.404677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:47:56.160615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848468+0000","last_change":"2024-04-25T19:15:19.426379+0000","last_active":"2024-04-25T19:15:20.848468+0000","last_peered":"2024-04-25T19:15:20.848468+0000","last_clean":"2024-04-25T19:15:20.848468+0000","last_became_active":"2024-04-25T19:15:19.425997+0000","last_became_peered":"2024-04-25T19:15:19.425997+0000","last_unstale":"2024-04-25T19:15:20.848468+0000","last_undegraded":"2024-04-25T19:15:20.848468+0000","last_fullsized":"2024-04-25T19:15:20.848468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:32.972274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:41.491934+0000","last_change":"2024-04-25T19:15:18.427748+0000","last_active":"2024-04-25T19:15:41.491934+0000","last_peered":"2024-04-25T19:15:41.491934+0000","last_clean":"2024-04-25T19:15:41.491934+0000","last_became_active":"2024-04-25T19:15:18.427374+0000","last_became_peered":"2024-04-25T19:15:18.427374+0000","last_unstale":"2024-04-25T19:15:41.491934+0000","last_undegraded":"2024-04-25T19:15:41.491934+0000","last_fullsized":"2024-04-25T19:15:41.491934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:47:41.469558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404710+0000","last_change":"2024-04-25T19:15:19.422196+0000","last_active":"2024-04-25T19:15:20.404710+0000","last_peered":"2024-04-25T19:15:20.404710+0000","last_clean":"2024-04-25T19:15:20.404710+0000","last_became_active":"2024-04-25T19:15:19.421835+0000","last_became_peered":"2024-04-25T19:15:19.421835+0000","last_unstale":"2024-04-25T19:15:20.404710+0000","last_undegraded":"2024-04-25T19:15:20.404710+0000","last_fullsized":"2024-04-25T19:15:20.404710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:38.500369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'7","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:41.493792+0000","last_change":"2024-04-25T19:15:18.419086+0000","last_active":"2024-04-25T19:15:41.493792+0000","last_peered":"2024-04-25T19:15:41.493792+0000","last_clean":"2024-04-25T19:15:41.493792+0000","last_became_active":"2024-04-25T19:15:18.418775+0000","last_became_peered":"2024-04-25T19:15:18.418775+0000","last_unstale":"2024-04-25T19:15:41.493792+0000","last_undegraded":"2024-04-25T19:15:41.493792+0000","last_fullsized":"2024-04-25T19:15:41.493792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:20.074753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405244+0000","last_change":"2024-04-25T19:15:19.432155+0000","last_active":"2024-04-25T19:15:20.405244+0000","last_peered":"2024-04-25T19:15:20.405244+0000","last_clean":"2024-04-25T19:15:20.405244+0000","last_became_active":"2024-04-25T19:15:19.431905+0000","last_became_peered":"2024-04-25T19:15:19.431905+0000","last_unstale":"2024-04-25T19:15:20.405244+0000","last_undegraded":"2024-04-25T19:15:20.405244+0000","last_fullsized":"2024-04-25T19:15:20.405244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:52.752469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'7","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:47.501382+0000","last_change":"2024-04-25T19:15:18.423144+0000","last_active":"2024-04-25T19:15:47.501382+0000","last_peered":"2024-04-25T19:15:47.501382+0000","last_clean":"2024-04-25T19:15:47.501382+0000","last_became_active":"2024-04-25T19:15:18.422838+0000","last_became_peered":"2024-04-25T19:15:18.422838+0000","last_unstale":"2024-04-25T19:15:47.501382+0000","last_undegraded":"2024-04-25T19:15:47.501382+0000","last_fullsized":"2024-04-25T19:15:47.501382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:28.026912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804520+0000","last_change":"2024-04-25T19:15:19.420582+0000","last_active":"2024-04-25T19:15:20.804520+0000","last_peered":"2024-04-25T19:15:20.804520+0000","last_clean":"2024-04-25T19:15:20.804520+0000","last_became_active":"2024-04-25T19:15:19.420270+0000","last_became_peered":"2024-04-25T19:15:19.420270+0000","last_unstale":"2024-04-25T19:15:20.804520+0000","last_undegraded":"2024-04-25T19:15:20.804520+0000","last_fullsized":"2024-04-25T19:15:20.804520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:06.827819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:29.474676+0000","last_change":"2024-04-25T19:15:18.425469+0000","last_active":"2024-04-25T19:15:29.474676+0000","last_peered":"2024-04-25T19:15:29.474676+0000","last_clean":"2024-04-25T19:15:29.474676+0000","last_became_active":"2024-04-25T19:15:18.425084+0000","last_became_peered":"2024-04-25T19:15:18.425084+0000","last_unstale":"2024-04-25T19:15:29.474676+0000","last_undegraded":"2024-04-25T19:15:29.474676+0000","last_fullsized":"2024-04-25T19:15:29.474676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:14.363158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822624+0000","last_change":"2024-04-25T19:15:19.425159+0000","last_active":"2024-04-25T19:15:20.822624+0000","last_peered":"2024-04-25T19:15:20.822624+0000","last_clean":"2024-04-25T19:15:20.822624+0000","last_became_active":"2024-04-25T19:15:19.424847+0000","last_became_peered":"2024-04-25T19:15:19.424847+0000","last_unstale":"2024-04-25T19:15:20.822624+0000","last_undegraded":"2024-04-25T19:15:20.822624+0000","last_fullsized":"2024-04-25T19:15:20.822624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:28.715286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"21'7","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:47.500688+0000","last_change":"2024-04-25T19:15:18.424645+0000","last_active":"2024-04-25T19:15:47.500688+0000","last_peered":"2024-04-25T19:15:47.500688+0000","last_clean":"2024-04-25T19:15:47.500688+0000","last_became_active":"2024-04-25T19:15:18.424241+0000","last_became_peered":"2024-04-25T19:15:18.424241+0000","last_unstale":"2024-04-25T19:15:47.500688+0000","last_undegraded":"2024-04-25T19:15:47.500688+0000","last_fullsized":"2024-04-25T19:15:47.500688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:29.373363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405640+0000","last_change":"2024-04-25T19:15:19.421095+0000","last_active":"2024-04-25T19:15:20.405640+0000","last_peered":"2024-04-25T19:15:20.405640+0000","last_clean":"2024-04-25T19:15:20.405640+0000","last_became_active":"2024-04-25T19:15:19.420795+0000","last_became_peered":"2024-04-25T19:15:19.420795+0000","last_unstale":"2024-04-25T19:15:20.405640+0000","last_undegraded":"2024-04-25T19:15:20.405640+0000","last_fullsized":"2024-04-25T19:15:20.405640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:13.061506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:41.490164+0000","last_change":"2024-04-25T19:15:18.423841+0000","last_active":"2024-04-25T19:15:41.490164+0000","last_peered":"2024-04-25T19:15:41.490164+0000","last_clean":"2024-04-25T19:15:41.490164+0000","last_became_active":"2024-04-25T19:15:18.423500+0000","last_became_peered":"2024-04-25T19:15:18.423500+0000","last_unstale":"2024-04-25T19:15:41.490164+0000","last_undegraded":"2024-04-25T19:15:41.490164+0000","last_fullsized":"2024-04-25T19:15:41.490164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:58.933085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405564+0000","last_change":"2024-04-25T19:15:19.416232+0000","last_active":"2024-04-25T19:15:20.405564+0000","last_peered":"2024-04-25T19:15:20.405564+0000","last_clean":"2024-04-25T19:15:20.405564+0000","last_became_active":"2024-04-25T19:15:19.415828+0000","last_became_peered":"2024-04-25T19:15:19.415828+0000","last_unstale":"2024-04-25T19:15:20.405564+0000","last_undegraded":"2024-04-25T19:15:20.405564+0000","last_fullsized":"2024-04-25T19:15:20.405564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:59.751840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:29.474678+0000","last_change":"2024-04-25T19:15:18.423918+0000","last_active":"2024-04-25T19:15:29.474678+0000","last_peered":"2024-04-25T19:15:29.474678+0000","last_clean":"2024-04-25T19:15:29.474678+0000","last_became_active":"2024-04-25T19:15:18.423588+0000","last_became_peered":"2024-04-25T19:15:18.423588+0000","last_unstale":"2024-04-25T19:15:29.474678+0000","last_undegraded":"2024-04-25T19:15:29.474678+0000","last_fullsized":"2024-04-25T19:15:29.474678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:16.378285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405402+0000","last_change":"2024-04-25T19:15:19.422984+0000","last_active":"2024-04-25T19:15:20.405402+0000","last_peered":"2024-04-25T19:15:20.405402+0000","last_clean":"2024-04-25T19:15:20.405402+0000","last_became_active":"2024-04-25T19:15:19.422651+0000","last_became_peered":"2024-04-25T19:15:19.422651+0000","last_unstale":"2024-04-25T19:15:20.405402+0000","last_undegraded":"2024-04-25T19:15:20.405402+0000","last_fullsized":"2024-04-25T19:15:20.405402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:54.984914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848417+0000","last_change":"2024-04-25T19:15:19.423982+0000","last_active":"2024-04-25T19:15:20.848417+0000","last_peered":"2024-04-25T19:15:20.848417+0000","last_clean":"2024-04-25T19:15:20.848417+0000","last_became_active":"2024-04-25T19:15:19.423341+0000","last_became_peered":"2024-04-25T19:15:19.423341+0000","last_unstale":"2024-04-25T19:15:20.848417+0000","last_undegraded":"2024-04-25T19:15:20.848417+0000","last_fullsized":"2024-04-25T19:15:20.848417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:51.808352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:47.500561+0000","last_change":"2024-04-25T19:15:18.424455+0000","last_active":"2024-04-25T19:15:47.500561+0000","last_peered":"2024-04-25T19:15:47.500561+0000","last_clean":"2024-04-25T19:15:47.500561+0000","last_became_active":"2024-04-25T19:15:18.424119+0000","last_became_peered":"2024-04-25T19:15:18.424119+0000","last_unstale":"2024-04-25T19:15:47.500561+0000","last_undegraded":"2024-04-25T19:15:47.500561+0000","last_fullsized":"2024-04-25T19:15:47.500561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:43.135567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848336+0000","last_change":"2024-04-25T19:15:19.424064+0000","last_active":"2024-04-25T19:15:20.848336+0000","last_peered":"2024-04-25T19:15:20.848336+0000","last_clean":"2024-04-25T19:15:20.848336+0000","last_became_active":"2024-04-25T19:15:19.423187+0000","last_became_peered":"2024-04-25T19:15:19.423187+0000","last_unstale":"2024-04-25T19:15:20.848336+0000","last_undegraded":"2024-04-25T19:15:20.848336+0000","last_fullsized":"2024-04-25T19:15:20.848336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:43:31.590700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'11","reported_seq":25,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:47.501382+0000","last_change":"2024-04-25T19:15:18.423151+0000","last_active":"2024-04-25T19:15:47.501382+0000","last_peered":"2024-04-25T19:15:47.501382+0000","last_clean":"2024-04-25T19:15:47.501382+0000","last_became_active":"2024-04-25T19:15:18.422831+0000","last_became_peered":"2024-04-25T19:15:18.422831+0000","last_unstale":"2024-04-25T19:15:47.501382+0000","last_undegraded":"2024-04-25T19:15:47.501382+0000","last_fullsized":"2024-04-25T19:15:47.501382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:43.244963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804534+0000","last_change":"2024-04-25T19:15:19.416717+0000","last_active":"2024-04-25T19:15:20.804534+0000","last_peered":"2024-04-25T19:15:20.804534+0000","last_clean":"2024-04-25T19:15:20.804534+0000","last_became_active":"2024-04-25T19:15:19.416163+0000","last_became_peered":"2024-04-25T19:15:19.416163+0000","last_unstale":"2024-04-25T19:15:20.804534+0000","last_undegraded":"2024-04-25T19:15:20.804534+0000","last_fullsized":"2024-04-25T19:15:20.804534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:46.955159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:24.414530+0000","last_change":"2024-04-25T19:15:18.422875+0000","last_active":"2024-04-25T19:15:24.414530+0000","last_peered":"2024-04-25T19:15:24.414530+0000","last_clean":"2024-04-25T19:15:24.414530+0000","last_became_active":"2024-04-25T19:15:18.422489+0000","last_became_peered":"2024-04-25T19:15:18.422489+0000","last_unstale":"2024-04-25T19:15:24.414530+0000","last_undegraded":"2024-04-25T19:15:24.414530+0000","last_fullsized":"2024-04-25T19:15:24.414530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:30.602500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405477+0000","last_change":"2024-04-25T19:15:19.416987+0000","last_active":"2024-04-25T19:15:20.405477+0000","last_peered":"2024-04-25T19:15:20.405477+0000","last_clean":"2024-04-25T19:15:20.405477+0000","last_became_active":"2024-04-25T19:15:19.414173+0000","last_became_peered":"2024-04-25T19:15:19.414173+0000","last_unstale":"2024-04-25T19:15:20.405477+0000","last_undegraded":"2024-04-25T19:15:20.405477+0000","last_fullsized":"2024-04-25T19:15:20.405477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:32:43.188653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'13","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:47.500644+0000","last_change":"2024-04-25T19:15:18.423867+0000","last_active":"2024-04-25T19:15:47.500644+0000","last_peered":"2024-04-25T19:15:47.500644+0000","last_clean":"2024-04-25T19:15:47.500644+0000","last_became_active":"2024-04-25T19:15:18.423532+0000","last_became_peered":"2024-04-25T19:15:18.423532+0000","last_unstale":"2024-04-25T19:15:47.500644+0000","last_undegraded":"2024-04-25T19:15:47.500644+0000","last_fullsized":"2024-04-25T19:15:47.500644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:04.531338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8034,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":204,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":16068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":178,"ondisk_log_size":178,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64838,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":319488,"data_stored":668073,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":78007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":79537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":241664,"data_stored":76935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":249856,"data_stored":78041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":249856,"data_stored":77067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":77137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":666847,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T19:15:58.493 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T19:15:58.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.634+0000 7f4b0f796640 1 Processor -- start 2024-04-25T19:15:58.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.634+0000 7f4b0f796640 1 -- start start 2024-04-25T19:15:58.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.636+0000 7f4b0f796640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4b0806d6d0 0x7f4b08156f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.636+0000 7f4b0f796640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b0806e310 con 0x7f4b0806de20 2024-04-25T19:15:58.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.636+0000 7f4b0f796640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b0815ccc0 con 0x7f4b080cc9e0 2024-04-25T19:15:58.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.636+0000 7f4b0f796640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4b08157450 con 0x7f4b0806d6d0 2024-04-25T19:15:58.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.636+0000 7f4b0dd33640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4b0806d6d0 0x7f4b08156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.636+0000 7f4b0d532640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f4b0806de20 0x7f4b080cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56786/0 (socket says 172.21.15.76:56786) 2024-04-25T19:15:58.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.636+0000 7f4b0d532640 1 -- 172.21.15.76:0/1635796023 learned_addr learned my addr 172.21.15.76:0/1635796023 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:58.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.636+0000 7f4b0dd33640 1 -- 172.21.15.76:0/1635796023 >> v1:172.21.15.105:6789/0 conn(0x7f4b080cc9e0 legacy=0x7f4b08165630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:58.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.636+0000 7f4afffff640 1 -- 172.21.15.76:0/1635796023 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2819004594 0 0) 0x7f4b0806e310 con 0x7f4b0806de20 2024-04-25T19:15:58.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.636+0000 7f4b0dd33640 1 -- 172.21.15.76:0/1635796023 >> v1:172.21.15.76:6789/0 conn(0x7f4b0806de20 legacy=0x7f4b080cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:58.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.636+0000 7f4b0dd33640 1 -- 172.21.15.76:0/1635796023 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b080cb6a0 con 0x7f4b0806d6d0 2024-04-25T19:15:58.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.637+0000 7f4b0dd33640 1 --2- 172.21.15.76:0/1635796023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b0806d6d0 0x7f4b08156f10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009d90 tx=0x7f4b0002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f0481aa45bc1ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:58.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.637+0000 7f4afffff640 1 -- 172.21.15.76:0/1635796023 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00007740 con 0x7f4b0806d6d0 2024-04-25T19:15:58.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.637+0000 7f4afffff640 1 -- 172.21.15.76:0/1635796023 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b000078e0 con 0x7f4b0806d6d0 2024-04-25T19:15:58.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.637+0000 7f4afffff640 1 -- 172.21.15.76:0/1635796023 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00007cb0 con 0x7f4b0806d6d0 2024-04-25T19:15:58.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.638+0000 7f4b0f796640 1 -- 172.21.15.76:0/1635796023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b0806d6d0 msgr2=0x7f4b08156f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:58.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.638+0000 7f4b0f796640 1 --2- 172.21.15.76:0/1635796023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b0806d6d0 0x7f4b08156f10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009d90 tx=0x7f4b0002a660 comp rx=0 tx=0).stop 2024-04-25T19:15:58.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.638+0000 7f4b0f796640 1 -- 172.21.15.76:0/1635796023 shutdown_connections 2024-04-25T19:15:58.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.638+0000 7f4b0f796640 1 --2- 172.21.15.76:0/1635796023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b0806d6d0 0x7f4b08156f10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.638+0000 7f4b0f796640 1 -- 172.21.15.76:0/1635796023 >> 172.21.15.76:0/1635796023 conn(0x7f4b080684e0 msgr2=0x7f4b080688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:58.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.638+0000 7f4b0f796640 1 -- 172.21.15.76:0/1635796023 shutdown_connections 2024-04-25T19:15:58.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.638+0000 7f4b0f796640 1 -- 172.21.15.76:0/1635796023 wait complete. 2024-04-25T19:15:58.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.639+0000 7f4b0f796640 1 Processor -- start 2024-04-25T19:15:58.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.640+0000 7f4b0f796640 1 -- start start 2024-04-25T19:15:58.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.641+0000 7f4b0f796640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b0806d6d0 0x7f4b08151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.641+0000 7f4b0f796640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b0806de20 0x7f4b0815e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.641+0000 7f4b0f796640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b080cc9e0 0x7f4b0815ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.641+0000 7f4b0f796640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0815cf80 con 0x7f4b080cc9e0 2024-04-25T19:15:58.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.641+0000 7f4b0f796640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0815d2f0 con 0x7f4b0806de20 2024-04-25T19:15:58.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.641+0000 7f4b0f796640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f4b080cbda0 con 0x7f4b0806d6d0 2024-04-25T19:15:58.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.641+0000 7f4b0dd33640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b080cc9e0 0x7f4b0815ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.641+0000 7f4b0dd33640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b080cc9e0 0x7f4b0815ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38154/0 (socket says 172.21.15.76:38154) 2024-04-25T19:15:58.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.641+0000 7f4b0dd33640 1 -- 172.21.15.76:0/914386680 learned_addr learned my addr 172.21.15.76:0/914386680 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:58.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.641+0000 7f4b0d532640 1 --2- 172.21.15.76:0/914386680 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b0806d6d0 0x7f4b08151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.641+0000 7f4b0cd31640 1 --2- 172.21.15.76:0/914386680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b0806de20 0x7f4b0815e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.641+0000 7f4b0dd33640 1 -- 172.21.15.76:0/914386680 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b0806d6d0 msgr2=0x7f4b08151d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:58.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.641+0000 7f4b0dd33640 1 --2- 172.21.15.76:0/914386680 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b0806d6d0 0x7f4b08151d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.641+0000 7f4b0dd33640 1 -- 172.21.15.76:0/914386680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b0806de20 msgr2=0x7f4b0815e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:58.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.641+0000 7f4b0dd33640 1 --2- 172.21.15.76:0/914386680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b0806de20 0x7f4b0815e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.641+0000 7f4b0dd33640 1 -- 172.21.15.76:0/914386680 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b080c9a80 con 0x7f4b080cc9e0 2024-04-25T19:15:58.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.641+0000 7f4b0dd33640 1 --2- 172.21.15.76:0/914386680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b080cc9e0 0x7f4b0815ea40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009d90 tx=0x7f4b00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:58.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.641+0000 7f4afdffb640 1 -- 172.21.15.76:0/914386680 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0003f070 con 0x7f4b080cc9e0 2024-04-25T19:15:58.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.642+0000 7f4afdffb640 1 -- 172.21.15.76:0/914386680 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b00031c90 con 0x7f4b080cc9e0 2024-04-25T19:15:58.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.642+0000 7f4b0f796640 1 -- 172.21.15.76:0/914386680 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b0815f330 con 0x7f4b080cc9e0 2024-04-25T19:15:58.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.642+0000 7f4afdffb640 1 -- 172.21.15.76:0/914386680 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00043630 con 0x7f4b080cc9e0 2024-04-25T19:15:58.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.642+0000 7f4b0f796640 1 -- 172.21.15.76:0/914386680 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b08160200 con 0x7f4b080cc9e0 2024-04-25T19:15:58.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.643+0000 7f4b0f796640 1 -- 172.21.15.76:0/914386680 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b0815f500 con 0x7f4b080cc9e0 2024-04-25T19:15:58.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.643+0000 7f4afdffb640 1 -- 172.21.15.76:0/914386680 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4b0003b410 con 0x7f4b080cc9e0 2024-04-25T19:15:58.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.644+0000 7f4afdffb640 1 --2- 172.21.15.76:0/914386680 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4aec0744b0 0x7f4aec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.644+0000 7f4afdffb640 1 -- 172.21.15.76:0/914386680 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4b000b6700 con 0x7f4b080cc9e0 2024-04-25T19:15:58.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.644+0000 7f4b0d532640 1 --2- 172.21.15.76:0/914386680 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4aec0744b0 0x7f4aec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.644+0000 7f4b0d532640 1 --2- 172.21.15.76:0/914386680 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4aec0744b0 0x7f4aec0769c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4b04000f60 tx=0x7f4b04007e40 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:58.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.648+0000 7f4afdffb640 1 -- 172.21.15.76:0/914386680 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4b00080900 con 0x7f4b080cc9e0 2024-04-25T19:15:58.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.820+0000 7f4b0f796640 1 -- 172.21.15.76:0/914386680 --> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4b0815f0c0 con 0x7f4aec0744b0 2024-04-25T19:15:58.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.827+0000 7f4afdffb640 1 -- 172.21.15.76:0/914386680 <== mgr.4109 v2:172.21.15.105:6832/2585775531 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251992 (secure 0 0 0) 0x7f4b0815f0c0 con 0x7f4aec0744b0 2024-04-25T19:15:58.832 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:58.833 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-04-25T19:15:58.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.830+0000 7f4b0f796640 1 -- 172.21.15.76:0/914386680 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4aec0744b0 msgr2=0x7f4aec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:58.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.830+0000 7f4b0f796640 1 --2- 172.21.15.76:0/914386680 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4aec0744b0 0x7f4aec0769c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4b04000f60 tx=0x7f4b04007e40 comp rx=0 tx=0).stop 2024-04-25T19:15:58.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.830+0000 7f4b0f796640 1 -- 172.21.15.76:0/914386680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b080cc9e0 msgr2=0x7f4b0815ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:58.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.830+0000 7f4b0f796640 1 --2- 172.21.15.76:0/914386680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b080cc9e0 0x7f4b0815ea40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009d90 tx=0x7f4b00004a40 comp rx=0 tx=0).stop 2024-04-25T19:15:58.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.830+0000 7f4b0f796640 1 -- 172.21.15.76:0/914386680 shutdown_connections 2024-04-25T19:15:58.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.830+0000 7f4b0f796640 1 --2- 172.21.15.76:0/914386680 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4aec0744b0 0x7f4aec0769c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.830+0000 7f4b0f796640 1 --2- 172.21.15.76:0/914386680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b080cc9e0 0x7f4b0815ea40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.830+0000 7f4b0f796640 1 --2- 172.21.15.76:0/914386680 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b0806de20 0x7f4b0815e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.830+0000 7f4b0f796640 1 --2- 172.21.15.76:0/914386680 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b0806d6d0 0x7f4b08151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.830+0000 7f4b0f796640 1 -- 172.21.15.76:0/914386680 >> 172.21.15.76:0/914386680 conn(0x7f4b080684e0 msgr2=0x7f4b0815bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:58.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.830+0000 7f4b0f796640 1 -- 172.21.15.76:0/914386680 shutdown_connections 2024-04-25T19:15:58.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.830+0000 7f4b0f796640 1 -- 172.21.15.76:0/914386680 wait complete. 2024-04-25T19:15:58.843 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-04-25T19:15:57.838727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598402,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":305,"num_write_kb":1690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":250,"ondisk_log_size":250,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218208,"kb_used_data":2080,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756512,"statfs":{"total":773094113280,"available":772870668288,"internally_reserved":0,"allocated":2129920,"data_stored":1801644,"data_compressed":64838,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1406,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003101"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425589+0000","last_change":"2024-04-25T19:15:19.433569+0000","last_active":"2024-04-25T19:15:20.425589+0000","last_peered":"2024-04-25T19:15:20.425589+0000","last_clean":"2024-04-25T19:15:20.425589+0000","last_became_active":"2024-04-25T19:15:19.431332+0000","last_became_peered":"2024-04-25T19:15:19.431332+0000","last_unstale":"2024-04-25T19:15:20.425589+0000","last_undegraded":"2024-04-25T19:15:20.425589+0000","last_fullsized":"2024-04-25T19:15:20.425589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:57:01.213834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404926+0000","last_change":"2024-04-25T19:15:19.418695+0000","last_active":"2024-04-25T19:15:20.404926+0000","last_peered":"2024-04-25T19:15:20.404926+0000","last_clean":"2024-04-25T19:15:20.404926+0000","last_became_active":"2024-04-25T19:15:19.418235+0000","last_became_peered":"2024-04-25T19:15:19.418235+0000","last_unstale":"2024-04-25T19:15:20.404926+0000","last_undegraded":"2024-04-25T19:15:20.404926+0000","last_fullsized":"2024-04-25T19:15:20.404926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:30.516472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804491+0000","last_change":"2024-04-25T19:15:19.416983+0000","last_active":"2024-04-25T19:15:20.804491+0000","last_peered":"2024-04-25T19:15:20.804491+0000","last_clean":"2024-04-25T19:15:20.804491+0000","last_became_active":"2024-04-25T19:15:19.416443+0000","last_became_peered":"2024-04-25T19:15:19.416443+0000","last_unstale":"2024-04-25T19:15:20.804491+0000","last_undegraded":"2024-04-25T19:15:20.804491+0000","last_fullsized":"2024-04-25T19:15:20.804491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:02.794164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848308+0000","last_change":"2024-04-25T19:15:19.429058+0000","last_active":"2024-04-25T19:15:20.848308+0000","last_peered":"2024-04-25T19:15:20.848308+0000","last_clean":"2024-04-25T19:15:20.848308+0000","last_became_active":"2024-04-25T19:15:19.428419+0000","last_became_peered":"2024-04-25T19:15:19.428419+0000","last_unstale":"2024-04-25T19:15:20.848308+0000","last_undegraded":"2024-04-25T19:15:20.848308+0000","last_fullsized":"2024-04-25T19:15:20.848308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:23:56.817616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425402+0000","last_change":"2024-04-25T19:15:19.427615+0000","last_active":"2024-04-25T19:15:20.425402+0000","last_peered":"2024-04-25T19:15:20.425402+0000","last_clean":"2024-04-25T19:15:20.425402+0000","last_became_active":"2024-04-25T19:15:19.426968+0000","last_became_peered":"2024-04-25T19:15:19.426968+0000","last_unstale":"2024-04-25T19:15:20.425402+0000","last_undegraded":"2024-04-25T19:15:20.425402+0000","last_fullsized":"2024-04-25T19:15:20.425402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:42:00.686712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847989+0000","last_change":"2024-04-25T19:15:19.417602+0000","last_active":"2024-04-25T19:15:20.847989+0000","last_peered":"2024-04-25T19:15:20.847989+0000","last_clean":"2024-04-25T19:15:20.847989+0000","last_became_active":"2024-04-25T19:15:19.417292+0000","last_became_peered":"2024-04-25T19:15:19.417292+0000","last_unstale":"2024-04-25T19:15:20.847989+0000","last_undegraded":"2024-04-25T19:15:20.847989+0000","last_fullsized":"2024-04-25T19:15:20.847989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:37:43.583809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425114+0000","last_change":"2024-04-25T19:15:19.428180+0000","last_active":"2024-04-25T19:15:20.425114+0000","last_peered":"2024-04-25T19:15:20.425114+0000","last_clean":"2024-04-25T19:15:20.425114+0000","last_became_active":"2024-04-25T19:15:19.427852+0000","last_became_peered":"2024-04-25T19:15:19.427852+0000","last_unstale":"2024-04-25T19:15:20.425114+0000","last_undegraded":"2024-04-25T19:15:20.425114+0000","last_fullsized":"2024-04-25T19:15:20.425114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:51.300880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405446+0000","last_change":"2024-04-25T19:15:19.418253+0000","last_active":"2024-04-25T19:15:20.405446+0000","last_peered":"2024-04-25T19:15:20.405446+0000","last_clean":"2024-04-25T19:15:20.405446+0000","last_became_active":"2024-04-25T19:15:19.417183+0000","last_became_peered":"2024-04-25T19:15:19.417183+0000","last_unstale":"2024-04-25T19:15:20.405446+0000","last_undegraded":"2024-04-25T19:15:20.405446+0000","last_fullsized":"2024-04-25T19:15:20.405446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:05:13.181646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425578+0000","last_change":"2024-04-25T19:15:19.431671+0000","last_active":"2024-04-25T19:15:20.425578+0000","last_peered":"2024-04-25T19:15:20.425578+0000","last_clean":"2024-04-25T19:15:20.425578+0000","last_became_active":"2024-04-25T19:15:19.431240+0000","last_became_peered":"2024-04-25T19:15:19.431240+0000","last_unstale":"2024-04-25T19:15:20.425578+0000","last_undegraded":"2024-04-25T19:15:20.425578+0000","last_fullsized":"2024-04-25T19:15:20.425578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:22.759462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.406080+0000","last_change":"2024-04-25T19:15:19.416304+0000","last_active":"2024-04-25T19:15:20.406080+0000","last_peered":"2024-04-25T19:15:20.406080+0000","last_clean":"2024-04-25T19:15:20.406080+0000","last_became_active":"2024-04-25T19:15:19.415967+0000","last_became_peered":"2024-04-25T19:15:19.415967+0000","last_unstale":"2024-04-25T19:15:20.406080+0000","last_undegraded":"2024-04-25T19:15:20.406080+0000","last_fullsized":"2024-04-25T19:15:20.406080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:30:52.958731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405773+0000","last_change":"2024-04-25T19:15:19.417596+0000","last_active":"2024-04-25T19:15:20.405773+0000","last_peered":"2024-04-25T19:15:20.405773+0000","last_clean":"2024-04-25T19:15:20.405773+0000","last_became_active":"2024-04-25T19:15:19.417230+0000","last_became_peered":"2024-04-25T19:15:19.417230+0000","last_unstale":"2024-04-25T19:15:20.405773+0000","last_undegraded":"2024-04-25T19:15:20.405773+0000","last_fullsized":"2024-04-25T19:15:20.405773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:28:12.387651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822843+0000","last_change":"2024-04-25T19:15:19.420564+0000","last_active":"2024-04-25T19:15:20.822843+0000","last_peered":"2024-04-25T19:15:20.822843+0000","last_clean":"2024-04-25T19:15:20.822843+0000","last_became_active":"2024-04-25T19:15:19.420044+0000","last_became_peered":"2024-04-25T19:15:19.420044+0000","last_unstale":"2024-04-25T19:15:20.822843+0000","last_undegraded":"2024-04-25T19:15:20.822843+0000","last_fullsized":"2024-04-25T19:15:20.822843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:04.066030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405269+0000","last_change":"2024-04-25T19:15:19.415500+0000","last_active":"2024-04-25T19:15:20.405269+0000","last_peered":"2024-04-25T19:15:20.405269+0000","last_clean":"2024-04-25T19:15:20.405269+0000","last_became_active":"2024-04-25T19:15:19.414007+0000","last_became_peered":"2024-04-25T19:15:19.414007+0000","last_unstale":"2024-04-25T19:15:20.405269+0000","last_undegraded":"2024-04-25T19:15:20.405269+0000","last_fullsized":"2024-04-25T19:15:20.405269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:56:12.320555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804235+0000","last_change":"2024-04-25T19:15:19.423884+0000","last_active":"2024-04-25T19:15:20.804235+0000","last_peered":"2024-04-25T19:15:20.804235+0000","last_clean":"2024-04-25T19:15:20.804235+0000","last_became_active":"2024-04-25T19:15:19.422575+0000","last_became_peered":"2024-04-25T19:15:19.422575+0000","last_unstale":"2024-04-25T19:15:20.804235+0000","last_undegraded":"2024-04-25T19:15:20.804235+0000","last_fullsized":"2024-04-25T19:15:20.804235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:30.988950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404396+0000","last_change":"2024-04-25T19:15:19.424267+0000","last_active":"2024-04-25T19:15:20.404396+0000","last_peered":"2024-04-25T19:15:20.404396+0000","last_clean":"2024-04-25T19:15:20.404396+0000","last_became_active":"2024-04-25T19:15:19.422953+0000","last_became_peered":"2024-04-25T19:15:19.422953+0000","last_unstale":"2024-04-25T19:15:20.404396+0000","last_undegraded":"2024-04-25T19:15:20.404396+0000","last_fullsized":"2024-04-25T19:15:20.404396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:42.823233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847983+0000","last_change":"2024-04-25T19:15:19.420374+0000","last_active":"2024-04-25T19:15:20.847983+0000","last_peered":"2024-04-25T19:15:20.847983+0000","last_clean":"2024-04-25T19:15:20.847983+0000","last_became_active":"2024-04-25T19:15:19.420015+0000","last_became_peered":"2024-04-25T19:15:19.420015+0000","last_unstale":"2024-04-25T19:15:20.847983+0000","last_undegraded":"2024-04-25T19:15:20.847983+0000","last_fullsized":"2024-04-25T19:15:20.847983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:04:26.719359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848705+0000","last_change":"2024-04-25T19:15:19.434476+0000","last_active":"2024-04-25T19:15:20.848705+0000","last_peered":"2024-04-25T19:15:20.848705+0000","last_clean":"2024-04-25T19:15:20.848705+0000","last_became_active":"2024-04-25T19:15:19.433836+0000","last_became_peered":"2024-04-25T19:15:19.433836+0000","last_unstale":"2024-04-25T19:15:20.848705+0000","last_undegraded":"2024-04-25T19:15:20.848705+0000","last_fullsized":"2024-04-25T19:15:20.848705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:13.234829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847359+0000","last_change":"2024-04-25T19:15:19.415521+0000","last_active":"2024-04-25T19:15:20.847359+0000","last_peered":"2024-04-25T19:15:20.847359+0000","last_clean":"2024-04-25T19:15:20.847359+0000","last_became_active":"2024-04-25T19:15:19.415177+0000","last_became_peered":"2024-04-25T19:15:19.415177+0000","last_unstale":"2024-04-25T19:15:20.847359+0000","last_undegraded":"2024-04-25T19:15:20.847359+0000","last_fullsized":"2024-04-25T19:15:20.847359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:24.355619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425590+0000","last_change":"2024-04-25T19:15:19.430291+0000","last_active":"2024-04-25T19:15:20.425590+0000","last_peered":"2024-04-25T19:15:20.425590+0000","last_clean":"2024-04-25T19:15:20.425590+0000","last_became_active":"2024-04-25T19:15:19.427226+0000","last_became_peered":"2024-04-25T19:15:19.427226+0000","last_unstale":"2024-04-25T19:15:20.425590+0000","last_undegraded":"2024-04-25T19:15:20.425590+0000","last_fullsized":"2024-04-25T19:15:20.425590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:38.464211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405750+0000","last_change":"2024-04-25T19:15:19.421696+0000","last_active":"2024-04-25T19:15:20.405750+0000","last_peered":"2024-04-25T19:15:20.405750+0000","last_clean":"2024-04-25T19:15:20.405750+0000","last_became_active":"2024-04-25T19:15:19.421300+0000","last_became_peered":"2024-04-25T19:15:19.421300+0000","last_unstale":"2024-04-25T19:15:20.405750+0000","last_undegraded":"2024-04-25T19:15:20.405750+0000","last_fullsized":"2024-04-25T19:15:20.405750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:15.226398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804410+0000","last_change":"2024-04-25T19:15:19.427880+0000","last_active":"2024-04-25T19:15:20.804410+0000","last_peered":"2024-04-25T19:15:20.804410+0000","last_clean":"2024-04-25T19:15:20.804410+0000","last_became_active":"2024-04-25T19:15:19.427579+0000","last_became_peered":"2024-04-25T19:15:19.427579+0000","last_unstale":"2024-04-25T19:15:20.804410+0000","last_undegraded":"2024-04-25T19:15:20.804410+0000","last_fullsized":"2024-04-25T19:15:20.804410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:47:10.657715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848145+0000","last_change":"2024-04-25T19:15:19.430409+0000","last_active":"2024-04-25T19:15:20.848145+0000","last_peered":"2024-04-25T19:15:20.848145+0000","last_clean":"2024-04-25T19:15:20.848145+0000","last_became_active":"2024-04-25T19:15:19.428305+0000","last_became_peered":"2024-04-25T19:15:19.428305+0000","last_unstale":"2024-04-25T19:15:20.848145+0000","last_undegraded":"2024-04-25T19:15:20.848145+0000","last_fullsized":"2024-04-25T19:15:20.848145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:10:01.268963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405396+0000","last_change":"2024-04-25T19:15:19.424435+0000","last_active":"2024-04-25T19:15:20.405396+0000","last_peered":"2024-04-25T19:15:20.405396+0000","last_clean":"2024-04-25T19:15:20.405396+0000","last_became_active":"2024-04-25T19:15:19.422763+0000","last_became_peered":"2024-04-25T19:15:19.422763+0000","last_unstale":"2024-04-25T19:15:20.405396+0000","last_undegraded":"2024-04-25T19:15:20.405396+0000","last_fullsized":"2024-04-25T19:15:20.405396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:58.092412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848072+0000","last_change":"2024-04-25T19:15:19.420275+0000","last_active":"2024-04-25T19:15:20.848072+0000","last_peered":"2024-04-25T19:15:20.848072+0000","last_clean":"2024-04-25T19:15:20.848072+0000","last_became_active":"2024-04-25T19:15:19.419936+0000","last_became_peered":"2024-04-25T19:15:19.419936+0000","last_unstale":"2024-04-25T19:15:20.848072+0000","last_undegraded":"2024-04-25T19:15:20.848072+0000","last_fullsized":"2024-04-25T19:15:20.848072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:55.710620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405087+0000","last_change":"2024-04-25T19:15:19.423408+0000","last_active":"2024-04-25T19:15:20.405087+0000","last_peered":"2024-04-25T19:15:20.405087+0000","last_clean":"2024-04-25T19:15:20.405087+0000","last_became_active":"2024-04-25T19:15:19.423116+0000","last_became_peered":"2024-04-25T19:15:19.423116+0000","last_unstale":"2024-04-25T19:15:20.405087+0000","last_undegraded":"2024-04-25T19:15:20.405087+0000","last_fullsized":"2024-04-25T19:15:20.405087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:18:57.787013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.406015+0000","last_change":"2024-04-25T19:15:19.428826+0000","last_active":"2024-04-25T19:15:20.406015+0000","last_peered":"2024-04-25T19:15:20.406015+0000","last_clean":"2024-04-25T19:15:20.406015+0000","last_became_active":"2024-04-25T19:15:19.428474+0000","last_became_peered":"2024-04-25T19:15:19.428474+0000","last_unstale":"2024-04-25T19:15:20.406015+0000","last_undegraded":"2024-04-25T19:15:20.406015+0000","last_fullsized":"2024-04-25T19:15:20.406015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:19.974851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404739+0000","last_change":"2024-04-25T19:15:19.423353+0000","last_active":"2024-04-25T19:15:20.404739+0000","last_peered":"2024-04-25T19:15:20.404739+0000","last_clean":"2024-04-25T19:15:20.404739+0000","last_became_active":"2024-04-25T19:15:19.422795+0000","last_became_peered":"2024-04-25T19:15:19.422795+0000","last_unstale":"2024-04-25T19:15:20.404739+0000","last_undegraded":"2024-04-25T19:15:20.404739+0000","last_fullsized":"2024-04-25T19:15:20.404739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:34.119976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405076+0000","last_change":"2024-04-25T19:15:19.421809+0000","last_active":"2024-04-25T19:15:20.405076+0000","last_peered":"2024-04-25T19:15:20.405076+0000","last_clean":"2024-04-25T19:15:20.405076+0000","last_became_active":"2024-04-25T19:15:19.421515+0000","last_became_peered":"2024-04-25T19:15:19.421515+0000","last_unstale":"2024-04-25T19:15:20.405076+0000","last_undegraded":"2024-04-25T19:15:20.405076+0000","last_fullsized":"2024-04-25T19:15:20.405076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:22.603291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804477+0000","last_change":"2024-04-25T19:15:19.410183+0000","last_active":"2024-04-25T19:15:20.804477+0000","last_peered":"2024-04-25T19:15:20.804477+0000","last_clean":"2024-04-25T19:15:20.804477+0000","last_became_active":"2024-04-25T19:15:19.409857+0000","last_became_peered":"2024-04-25T19:15:19.409857+0000","last_unstale":"2024-04-25T19:15:20.804477+0000","last_undegraded":"2024-04-25T19:15:20.804477+0000","last_fullsized":"2024-04-25T19:15:20.804477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:30.180786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848239+0000","last_change":"2024-04-25T19:15:19.425524+0000","last_active":"2024-04-25T19:15:20.848239+0000","last_peered":"2024-04-25T19:15:20.848239+0000","last_clean":"2024-04-25T19:15:20.848239+0000","last_became_active":"2024-04-25T19:15:19.424578+0000","last_became_peered":"2024-04-25T19:15:19.424578+0000","last_unstale":"2024-04-25T19:15:20.848239+0000","last_undegraded":"2024-04-25T19:15:20.848239+0000","last_fullsized":"2024-04-25T19:15:20.848239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:02.103879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.847243+0000","last_change":"2024-04-25T19:15:19.426720+0000","last_active":"2024-04-25T19:15:20.847243+0000","last_peered":"2024-04-25T19:15:20.847243+0000","last_clean":"2024-04-25T19:15:20.847243+0000","last_became_active":"2024-04-25T19:15:19.426139+0000","last_became_peered":"2024-04-25T19:15:19.426139+0000","last_unstale":"2024-04-25T19:15:20.847243+0000","last_undegraded":"2024-04-25T19:15:20.847243+0000","last_fullsized":"2024-04-25T19:15:20.847243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:31.608993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405390+0000","last_change":"2024-04-25T19:15:19.413805+0000","last_active":"2024-04-25T19:15:20.405390+0000","last_peered":"2024-04-25T19:15:20.405390+0000","last_clean":"2024-04-25T19:15:20.405390+0000","last_became_active":"2024-04-25T19:15:19.413455+0000","last_became_peered":"2024-04-25T19:15:19.413455+0000","last_unstale":"2024-04-25T19:15:20.405390+0000","last_undegraded":"2024-04-25T19:15:20.405390+0000","last_fullsized":"2024-04-25T19:15:20.405390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:05:45.008076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822709+0000","last_change":"2024-04-25T19:15:19.423770+0000","last_active":"2024-04-25T19:15:20.822709+0000","last_peered":"2024-04-25T19:15:20.822709+0000","last_clean":"2024-04-25T19:15:20.822709+0000","last_became_active":"2024-04-25T19:15:19.423406+0000","last_became_peered":"2024-04-25T19:15:19.423406+0000","last_unstale":"2024-04-25T19:15:20.822709+0000","last_undegraded":"2024-04-25T19:15:20.822709+0000","last_fullsized":"2024-04-25T19:15:20.822709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:41:23.669832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:29.482042+0000","last_change":"2024-04-25T19:15:18.422035+0000","last_active":"2024-04-25T19:15:29.482042+0000","last_peered":"2024-04-25T19:15:29.482042+0000","last_clean":"2024-04-25T19:15:29.482042+0000","last_became_active":"2024-04-25T19:15:18.421658+0000","last_became_peered":"2024-04-25T19:15:18.421658+0000","last_unstale":"2024-04-25T19:15:29.482042+0000","last_undegraded":"2024-04-25T19:15:29.482042+0000","last_fullsized":"2024-04-25T19:15:29.482042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:23.349126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804571+0000","last_change":"2024-04-25T19:15:19.421805+0000","last_active":"2024-04-25T19:15:20.804571+0000","last_peered":"2024-04-25T19:15:20.804571+0000","last_clean":"2024-04-25T19:15:20.804571+0000","last_became_active":"2024-04-25T19:15:19.421497+0000","last_became_peered":"2024-04-25T19:15:19.421497+0000","last_unstale":"2024-04-25T19:15:20.804571+0000","last_undegraded":"2024-04-25T19:15:20.804571+0000","last_fullsized":"2024-04-25T19:15:20.804571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:29:29.949508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'7","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:41.495006+0000","last_change":"2024-04-25T19:15:18.416894+0000","last_active":"2024-04-25T19:15:41.495006+0000","last_peered":"2024-04-25T19:15:41.495006+0000","last_clean":"2024-04-25T19:15:41.495006+0000","last_became_active":"2024-04-25T19:15:18.416539+0000","last_became_peered":"2024-04-25T19:15:18.416539+0000","last_unstale":"2024-04-25T19:15:41.495006+0000","last_undegraded":"2024-04-25T19:15:41.495006+0000","last_fullsized":"2024-04-25T19:15:41.495006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:30.388606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:35.471396+0000","last_change":"2024-04-25T19:15:18.420129+0000","last_active":"2024-04-25T19:15:35.471396+0000","last_peered":"2024-04-25T19:15:35.471396+0000","last_clean":"2024-04-25T19:15:35.471396+0000","last_became_active":"2024-04-25T19:15:18.419732+0000","last_became_peered":"2024-04-25T19:15:18.419732+0000","last_unstale":"2024-04-25T19:15:35.471396+0000","last_undegraded":"2024-04-25T19:15:35.471396+0000","last_fullsized":"2024-04-25T19:15:35.471396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:41:48.673989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405615+0000","last_change":"2024-04-25T19:15:19.415655+0000","last_active":"2024-04-25T19:15:20.405615+0000","last_peered":"2024-04-25T19:15:20.405615+0000","last_clean":"2024-04-25T19:15:20.405615+0000","last_became_active":"2024-04-25T19:15:19.415287+0000","last_became_peered":"2024-04-25T19:15:19.415287+0000","last_unstale":"2024-04-25T19:15:20.405615+0000","last_undegraded":"2024-04-25T19:15:20.405615+0000","last_fullsized":"2024-04-25T19:15:20.405615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:49.155289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:41.492883+0000","last_change":"2024-04-25T19:15:18.420711+0000","last_active":"2024-04-25T19:15:41.492883+0000","last_peered":"2024-04-25T19:15:41.492883+0000","last_clean":"2024-04-25T19:15:41.492883+0000","last_became_active":"2024-04-25T19:15:18.420362+0000","last_became_peered":"2024-04-25T19:15:18.420362+0000","last_unstale":"2024-04-25T19:15:41.492883+0000","last_undegraded":"2024-04-25T19:15:41.492883+0000","last_fullsized":"2024-04-25T19:15:41.492883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:48.245193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404375+0000","last_change":"2024-04-25T19:15:19.424356+0000","last_active":"2024-04-25T19:15:20.404375+0000","last_peered":"2024-04-25T19:15:20.404375+0000","last_clean":"2024-04-25T19:15:20.404375+0000","last_became_active":"2024-04-25T19:15:19.423556+0000","last_became_peered":"2024-04-25T19:15:19.423556+0000","last_unstale":"2024-04-25T19:15:20.404375+0000","last_undegraded":"2024-04-25T19:15:20.404375+0000","last_fullsized":"2024-04-25T19:15:20.404375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:19:35.427261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:29.476002+0000","last_change":"2024-04-25T19:15:18.419250+0000","last_active":"2024-04-25T19:15:29.476002+0000","last_peered":"2024-04-25T19:15:29.476002+0000","last_clean":"2024-04-25T19:15:29.476002+0000","last_became_active":"2024-04-25T19:15:18.418927+0000","last_became_peered":"2024-04-25T19:15:18.418927+0000","last_unstale":"2024-04-25T19:15:29.476002+0000","last_undegraded":"2024-04-25T19:15:29.476002+0000","last_fullsized":"2024-04-25T19:15:29.476002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:26.012965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848606+0000","last_change":"2024-04-25T19:15:19.430117+0000","last_active":"2024-04-25T19:15:20.848606+0000","last_peered":"2024-04-25T19:15:20.848606+0000","last_clean":"2024-04-25T19:15:20.848606+0000","last_became_active":"2024-04-25T19:15:19.428985+0000","last_became_peered":"2024-04-25T19:15:19.428985+0000","last_unstale":"2024-04-25T19:15:20.848606+0000","last_undegraded":"2024-04-25T19:15:20.848606+0000","last_fullsized":"2024-04-25T19:15:20.848606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:36:29.069541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:47.499556+0000","last_change":"2024-04-25T19:15:18.422931+0000","last_active":"2024-04-25T19:15:47.499556+0000","last_peered":"2024-04-25T19:15:47.499556+0000","last_clean":"2024-04-25T19:15:47.499556+0000","last_became_active":"2024-04-25T19:15:18.422610+0000","last_became_peered":"2024-04-25T19:15:18.422610+0000","last_unstale":"2024-04-25T19:15:47.499556+0000","last_undegraded":"2024-04-25T19:15:47.499556+0000","last_fullsized":"2024-04-25T19:15:47.499556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:18.489484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804598+0000","last_change":"2024-04-25T19:15:19.416109+0000","last_active":"2024-04-25T19:15:20.804598+0000","last_peered":"2024-04-25T19:15:20.804598+0000","last_clean":"2024-04-25T19:15:20.804598+0000","last_became_active":"2024-04-25T19:15:19.415734+0000","last_became_peered":"2024-04-25T19:15:19.415734+0000","last_unstale":"2024-04-25T19:15:20.804598+0000","last_undegraded":"2024-04-25T19:15:20.804598+0000","last_fullsized":"2024-04-25T19:15:20.804598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:57:31.406303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:29.488292+0000","last_change":"2024-04-25T19:15:18.428677+0000","last_active":"2024-04-25T19:15:29.488292+0000","last_peered":"2024-04-25T19:15:29.488292+0000","last_clean":"2024-04-25T19:15:29.488292+0000","last_became_active":"2024-04-25T19:15:18.428432+0000","last_became_peered":"2024-04-25T19:15:18.428432+0000","last_unstale":"2024-04-25T19:15:29.488292+0000","last_undegraded":"2024-04-25T19:15:29.488292+0000","last_fullsized":"2024-04-25T19:15:29.488292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:38.478595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822814+0000","last_change":"2024-04-25T19:15:19.428245+0000","last_active":"2024-04-25T19:15:20.822814+0000","last_peered":"2024-04-25T19:15:20.822814+0000","last_clean":"2024-04-25T19:15:20.822814+0000","last_became_active":"2024-04-25T19:15:19.427941+0000","last_became_peered":"2024-04-25T19:15:19.427941+0000","last_unstale":"2024-04-25T19:15:20.822814+0000","last_undegraded":"2024-04-25T19:15:20.822814+0000","last_fullsized":"2024-04-25T19:15:20.822814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:09.947923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'10","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:47.500021+0000","last_change":"2024-04-25T19:15:18.420881+0000","last_active":"2024-04-25T19:15:47.500021+0000","last_peered":"2024-04-25T19:15:47.500021+0000","last_clean":"2024-04-25T19:15:47.500021+0000","last_became_active":"2024-04-25T19:15:18.420303+0000","last_became_peered":"2024-04-25T19:15:18.420303+0000","last_unstale":"2024-04-25T19:15:47.500021+0000","last_undegraded":"2024-04-25T19:15:47.500021+0000","last_fullsized":"2024-04-25T19:15:47.500021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:39:24.356298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425356+0000","last_change":"2024-04-25T19:15:19.432735+0000","last_active":"2024-04-25T19:15:20.425356+0000","last_peered":"2024-04-25T19:15:20.425356+0000","last_clean":"2024-04-25T19:15:20.425356+0000","last_became_active":"2024-04-25T19:15:19.431828+0000","last_became_peered":"2024-04-25T19:15:19.431828+0000","last_unstale":"2024-04-25T19:15:20.425356+0000","last_undegraded":"2024-04-25T19:15:20.425356+0000","last_fullsized":"2024-04-25T19:15:20.425356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:20.189230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'9","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:47.496364+0000","last_change":"2024-04-25T19:15:18.421008+0000","last_active":"2024-04-25T19:15:47.496364+0000","last_peered":"2024-04-25T19:15:47.496364+0000","last_clean":"2024-04-25T19:15:47.496364+0000","last_became_active":"2024-04-25T19:15:18.420402+0000","last_became_peered":"2024-04-25T19:15:18.420402+0000","last_unstale":"2024-04-25T19:15:47.496364+0000","last_undegraded":"2024-04-25T19:15:47.496364+0000","last_fullsized":"2024-04-25T19:15:47.496364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:05:28.723958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804304+0000","last_change":"2024-04-25T19:15:19.424805+0000","last_active":"2024-04-25T19:15:20.804304+0000","last_peered":"2024-04-25T19:15:20.804304+0000","last_clean":"2024-04-25T19:15:20.804304+0000","last_became_active":"2024-04-25T19:15:19.424512+0000","last_became_peered":"2024-04-25T19:15:19.424512+0000","last_unstale":"2024-04-25T19:15:20.804304+0000","last_undegraded":"2024-04-25T19:15:20.804304+0000","last_fullsized":"2024-04-25T19:15:20.804304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:50.423186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:41.492047+0000","last_change":"2024-04-25T19:15:18.426805+0000","last_active":"2024-04-25T19:15:41.492047+0000","last_peered":"2024-04-25T19:15:41.492047+0000","last_clean":"2024-04-25T19:15:41.492047+0000","last_became_active":"2024-04-25T19:15:18.426472+0000","last_became_peered":"2024-04-25T19:15:18.426472+0000","last_unstale":"2024-04-25T19:15:41.492047+0000","last_undegraded":"2024-04-25T19:15:41.492047+0000","last_fullsized":"2024-04-25T19:15:41.492047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:29.373363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425459+0000","last_change":"2024-04-25T19:15:19.428780+0000","last_active":"2024-04-25T19:15:20.425459+0000","last_peered":"2024-04-25T19:15:20.425459+0000","last_clean":"2024-04-25T19:15:20.425459+0000","last_became_active":"2024-04-25T19:15:19.428450+0000","last_became_peered":"2024-04-25T19:15:19.428450+0000","last_unstale":"2024-04-25T19:15:20.425459+0000","last_undegraded":"2024-04-25T19:15:20.425459+0000","last_fullsized":"2024-04-25T19:15:20.425459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:01.203880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:41.492432+0000","last_change":"2024-04-25T19:15:18.417740+0000","last_active":"2024-04-25T19:15:41.492432+0000","last_peered":"2024-04-25T19:15:41.492432+0000","last_clean":"2024-04-25T19:15:41.492432+0000","last_became_active":"2024-04-25T19:15:18.417404+0000","last_became_peered":"2024-04-25T19:15:18.417404+0000","last_unstale":"2024-04-25T19:15:41.492432+0000","last_undegraded":"2024-04-25T19:15:41.492432+0000","last_fullsized":"2024-04-25T19:15:41.492432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:21.737283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405391+0000","last_change":"2024-04-25T19:15:19.411214+0000","last_active":"2024-04-25T19:15:20.405391+0000","last_peered":"2024-04-25T19:15:20.405391+0000","last_clean":"2024-04-25T19:15:20.405391+0000","last_became_active":"2024-04-25T19:15:19.410814+0000","last_became_peered":"2024-04-25T19:15:19.410814+0000","last_unstale":"2024-04-25T19:15:20.405391+0000","last_undegraded":"2024-04-25T19:15:20.405391+0000","last_fullsized":"2024-04-25T19:15:20.405391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:42.430277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'17","reported_seq":31,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:41.489606+0000","last_change":"2024-04-25T19:15:18.422226+0000","last_active":"2024-04-25T19:15:41.489606+0000","last_peered":"2024-04-25T19:15:41.489606+0000","last_clean":"2024-04-25T19:15:41.489606+0000","last_became_active":"2024-04-25T19:15:18.421900+0000","last_became_peered":"2024-04-25T19:15:18.421900+0000","last_unstale":"2024-04-25T19:15:41.489606+0000","last_undegraded":"2024-04-25T19:15:41.489606+0000","last_fullsized":"2024-04-25T19:15:41.489606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:18.040114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425347+0000","last_change":"2024-04-25T19:15:19.428885+0000","last_active":"2024-04-25T19:15:20.425347+0000","last_peered":"2024-04-25T19:15:20.425347+0000","last_clean":"2024-04-25T19:15:20.425347+0000","last_became_active":"2024-04-25T19:15:19.428563+0000","last_became_peered":"2024-04-25T19:15:19.428563+0000","last_unstale":"2024-04-25T19:15:20.425347+0000","last_undegraded":"2024-04-25T19:15:20.425347+0000","last_fullsized":"2024-04-25T19:15:20.425347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:43.469189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:24.414802+0000","last_change":"2024-04-25T19:15:18.422125+0000","last_active":"2024-04-25T19:15:24.414802+0000","last_peered":"2024-04-25T19:15:24.414802+0000","last_clean":"2024-04-25T19:15:24.414802+0000","last_became_active":"2024-04-25T19:15:18.421777+0000","last_became_peered":"2024-04-25T19:15:18.421777+0000","last_unstale":"2024-04-25T19:15:24.414802+0000","last_undegraded":"2024-04-25T19:15:24.414802+0000","last_fullsized":"2024-04-25T19:15:24.414802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:24.289010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425489+0000","last_change":"2024-04-25T19:15:19.429675+0000","last_active":"2024-04-25T19:15:20.425489+0000","last_peered":"2024-04-25T19:15:20.425489+0000","last_clean":"2024-04-25T19:15:20.425489+0000","last_became_active":"2024-04-25T19:15:19.429316+0000","last_became_peered":"2024-04-25T19:15:19.429316+0000","last_unstale":"2024-04-25T19:15:20.425489+0000","last_undegraded":"2024-04-25T19:15:20.425489+0000","last_fullsized":"2024-04-25T19:15:20.425489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:09:55.521944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:47.495217+0000","last_change":"2024-04-25T19:15:18.418661+0000","last_active":"2024-04-25T19:15:47.495217+0000","last_peered":"2024-04-25T19:15:47.495217+0000","last_clean":"2024-04-25T19:15:47.495217+0000","last_became_active":"2024-04-25T19:15:18.418359+0000","last_became_peered":"2024-04-25T19:15:18.418359+0000","last_unstale":"2024-04-25T19:15:47.495217+0000","last_undegraded":"2024-04-25T19:15:47.495217+0000","last_fullsized":"2024-04-25T19:15:47.495217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:49:22.023652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425687+0000","last_change":"2024-04-25T19:15:19.434853+0000","last_active":"2024-04-25T19:15:20.425687+0000","last_peered":"2024-04-25T19:15:20.425687+0000","last_clean":"2024-04-25T19:15:20.425687+0000","last_became_active":"2024-04-25T19:15:19.434508+0000","last_became_peered":"2024-04-25T19:15:19.434508+0000","last_unstale":"2024-04-25T19:15:20.425687+0000","last_undegraded":"2024-04-25T19:15:20.425687+0000","last_fullsized":"2024-04-25T19:15:20.425687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:52.965864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:41.494028+0000","last_change":"2024-04-25T19:15:18.428626+0000","last_active":"2024-04-25T19:15:41.494028+0000","last_peered":"2024-04-25T19:15:41.494028+0000","last_clean":"2024-04-25T19:15:41.494028+0000","last_became_active":"2024-04-25T19:15:18.428332+0000","last_became_peered":"2024-04-25T19:15:18.428332+0000","last_unstale":"2024-04-25T19:15:41.494028+0000","last_undegraded":"2024-04-25T19:15:41.494028+0000","last_fullsized":"2024-04-25T19:15:41.494028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:15.595305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848535+0000","last_change":"2024-04-25T19:15:19.428754+0000","last_active":"2024-04-25T19:15:20.848535+0000","last_peered":"2024-04-25T19:15:20.848535+0000","last_clean":"2024-04-25T19:15:20.848535+0000","last_became_active":"2024-04-25T19:15:19.427992+0000","last_became_peered":"2024-04-25T19:15:19.427992+0000","last_unstale":"2024-04-25T19:15:20.848535+0000","last_undegraded":"2024-04-25T19:15:20.848535+0000","last_fullsized":"2024-04-25T19:15:20.848535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:43:48.443923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:41.491818+0000","last_change":"2024-04-25T19:15:18.419645+0000","last_active":"2024-04-25T19:15:41.491818+0000","last_peered":"2024-04-25T19:15:41.491818+0000","last_clean":"2024-04-25T19:15:41.491818+0000","last_became_active":"2024-04-25T19:15:18.419286+0000","last_became_peered":"2024-04-25T19:15:18.419286+0000","last_unstale":"2024-04-25T19:15:41.491818+0000","last_undegraded":"2024-04-25T19:15:41.491818+0000","last_fullsized":"2024-04-25T19:15:41.491818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:08.295208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425261+0000","last_change":"2024-04-25T19:15:19.432395+0000","last_active":"2024-04-25T19:15:20.425261+0000","last_peered":"2024-04-25T19:15:20.425261+0000","last_clean":"2024-04-25T19:15:20.425261+0000","last_became_active":"2024-04-25T19:15:19.431739+0000","last_became_peered":"2024-04-25T19:15:19.431739+0000","last_unstale":"2024-04-25T19:15:20.425261+0000","last_undegraded":"2024-04-25T19:15:20.425261+0000","last_fullsized":"2024-04-25T19:15:20.425261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:50:03.964030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":110,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.425087+0000","last_change":"2024-04-25T19:15:15.604825+0000","last_active":"2024-04-25T19:15:20.425087+0000","last_peered":"2024-04-25T19:15:20.425087+0000","last_clean":"2024-04-25T19:15:20.425087+0000","last_became_active":"2024-04-25T19:15:15.603730+0000","last_became_peered":"2024-04-25T19:15:15.603730+0000","last_unstale":"2024-04-25T19:15:20.425087+0000","last_undegraded":"2024-04-25T19:15:20.425087+0000","last_fullsized":"2024-04-25T19:15:20.425087+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:14.367890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:14.367890+0000","last_clean_scrub_stamp":"2024-04-25T19:15:14.367890+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:02.411899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:41.491282+0000","last_change":"2024-04-25T19:15:18.426331+0000","last_active":"2024-04-25T19:15:41.491282+0000","last_peered":"2024-04-25T19:15:41.491282+0000","last_clean":"2024-04-25T19:15:41.491282+0000","last_became_active":"2024-04-25T19:15:18.425993+0000","last_became_peered":"2024-04-25T19:15:18.425993+0000","last_unstale":"2024-04-25T19:15:41.491282+0000","last_undegraded":"2024-04-25T19:15:41.491282+0000","last_fullsized":"2024-04-25T19:15:41.491282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:17.440484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405021+0000","last_change":"2024-04-25T19:15:19.417337+0000","last_active":"2024-04-25T19:15:20.405021+0000","last_peered":"2024-04-25T19:15:20.405021+0000","last_clean":"2024-04-25T19:15:20.405021+0000","last_became_active":"2024-04-25T19:15:19.416808+0000","last_became_peered":"2024-04-25T19:15:19.416808+0000","last_unstale":"2024-04-25T19:15:20.405021+0000","last_undegraded":"2024-04-25T19:15:20.405021+0000","last_fullsized":"2024-04-25T19:15:20.405021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:57.797904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:35.470254+0000","last_change":"2024-04-25T19:15:18.426358+0000","last_active":"2024-04-25T19:15:35.470254+0000","last_peered":"2024-04-25T19:15:35.470254+0000","last_clean":"2024-04-25T19:15:35.470254+0000","last_became_active":"2024-04-25T19:15:18.426037+0000","last_became_peered":"2024-04-25T19:15:18.426037+0000","last_unstale":"2024-04-25T19:15:35.470254+0000","last_undegraded":"2024-04-25T19:15:35.470254+0000","last_fullsized":"2024-04-25T19:15:35.470254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:26:28.945667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804206+0000","last_change":"2024-04-25T19:15:19.426219+0000","last_active":"2024-04-25T19:15:20.804206+0000","last_peered":"2024-04-25T19:15:20.804206+0000","last_clean":"2024-04-25T19:15:20.804206+0000","last_became_active":"2024-04-25T19:15:19.425626+0000","last_became_peered":"2024-04-25T19:15:19.425626+0000","last_unstale":"2024-04-25T19:15:20.804206+0000","last_undegraded":"2024-04-25T19:15:20.804206+0000","last_fullsized":"2024-04-25T19:15:20.804206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:58.372454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:35.470403+0000","last_change":"2024-04-25T19:15:18.424560+0000","last_active":"2024-04-25T19:15:35.470403+0000","last_peered":"2024-04-25T19:15:35.470403+0000","last_clean":"2024-04-25T19:15:35.470403+0000","last_became_active":"2024-04-25T19:15:18.424179+0000","last_became_peered":"2024-04-25T19:15:18.424179+0000","last_unstale":"2024-04-25T19:15:35.470403+0000","last_undegraded":"2024-04-25T19:15:35.470403+0000","last_fullsized":"2024-04-25T19:15:35.470403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:06.184399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404401+0000","last_change":"2024-04-25T19:15:19.415219+0000","last_active":"2024-04-25T19:15:20.404401+0000","last_peered":"2024-04-25T19:15:20.404401+0000","last_clean":"2024-04-25T19:15:20.404401+0000","last_became_active":"2024-04-25T19:15:19.414877+0000","last_became_peered":"2024-04-25T19:15:19.414877+0000","last_unstale":"2024-04-25T19:15:20.404401+0000","last_undegraded":"2024-04-25T19:15:20.404401+0000","last_fullsized":"2024-04-25T19:15:20.404401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:15.063419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:47.497653+0000","last_change":"2024-04-25T19:15:18.423402+0000","last_active":"2024-04-25T19:15:47.497653+0000","last_peered":"2024-04-25T19:15:47.497653+0000","last_clean":"2024-04-25T19:15:47.497653+0000","last_became_active":"2024-04-25T19:15:18.422330+0000","last_became_peered":"2024-04-25T19:15:18.422330+0000","last_unstale":"2024-04-25T19:15:47.497653+0000","last_undegraded":"2024-04-25T19:15:47.497653+0000","last_fullsized":"2024-04-25T19:15:47.497653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:39:17.512502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822708+0000","last_change":"2024-04-25T19:15:19.421096+0000","last_active":"2024-04-25T19:15:20.822708+0000","last_peered":"2024-04-25T19:15:20.822708+0000","last_clean":"2024-04-25T19:15:20.822708+0000","last_became_active":"2024-04-25T19:15:19.420598+0000","last_became_peered":"2024-04-25T19:15:19.420598+0000","last_unstale":"2024-04-25T19:15:20.822708+0000","last_undegraded":"2024-04-25T19:15:20.822708+0000","last_fullsized":"2024-04-25T19:15:20.822708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:20.733888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404677+0000","last_change":"2024-04-25T19:15:18.422979+0000","last_active":"2024-04-25T19:15:20.404677+0000","last_peered":"2024-04-25T19:15:20.404677+0000","last_clean":"2024-04-25T19:15:20.404677+0000","last_became_active":"2024-04-25T19:15:18.422621+0000","last_became_peered":"2024-04-25T19:15:18.422621+0000","last_unstale":"2024-04-25T19:15:20.404677+0000","last_undegraded":"2024-04-25T19:15:20.404677+0000","last_fullsized":"2024-04-25T19:15:20.404677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:47:56.160615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848468+0000","last_change":"2024-04-25T19:15:19.426379+0000","last_active":"2024-04-25T19:15:20.848468+0000","last_peered":"2024-04-25T19:15:20.848468+0000","last_clean":"2024-04-25T19:15:20.848468+0000","last_became_active":"2024-04-25T19:15:19.425997+0000","last_became_peered":"2024-04-25T19:15:19.425997+0000","last_unstale":"2024-04-25T19:15:20.848468+0000","last_undegraded":"2024-04-25T19:15:20.848468+0000","last_fullsized":"2024-04-25T19:15:20.848468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:32.972274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:41.491934+0000","last_change":"2024-04-25T19:15:18.427748+0000","last_active":"2024-04-25T19:15:41.491934+0000","last_peered":"2024-04-25T19:15:41.491934+0000","last_clean":"2024-04-25T19:15:41.491934+0000","last_became_active":"2024-04-25T19:15:18.427374+0000","last_became_peered":"2024-04-25T19:15:18.427374+0000","last_unstale":"2024-04-25T19:15:41.491934+0000","last_undegraded":"2024-04-25T19:15:41.491934+0000","last_fullsized":"2024-04-25T19:15:41.491934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:47:41.469558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.404710+0000","last_change":"2024-04-25T19:15:19.422196+0000","last_active":"2024-04-25T19:15:20.404710+0000","last_peered":"2024-04-25T19:15:20.404710+0000","last_clean":"2024-04-25T19:15:20.404710+0000","last_became_active":"2024-04-25T19:15:19.421835+0000","last_became_peered":"2024-04-25T19:15:19.421835+0000","last_unstale":"2024-04-25T19:15:20.404710+0000","last_undegraded":"2024-04-25T19:15:20.404710+0000","last_fullsized":"2024-04-25T19:15:20.404710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:38.500369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'7","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:41.493792+0000","last_change":"2024-04-25T19:15:18.419086+0000","last_active":"2024-04-25T19:15:41.493792+0000","last_peered":"2024-04-25T19:15:41.493792+0000","last_clean":"2024-04-25T19:15:41.493792+0000","last_became_active":"2024-04-25T19:15:18.418775+0000","last_became_peered":"2024-04-25T19:15:18.418775+0000","last_unstale":"2024-04-25T19:15:41.493792+0000","last_undegraded":"2024-04-25T19:15:41.493792+0000","last_fullsized":"2024-04-25T19:15:41.493792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:20.074753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405244+0000","last_change":"2024-04-25T19:15:19.432155+0000","last_active":"2024-04-25T19:15:20.405244+0000","last_peered":"2024-04-25T19:15:20.405244+0000","last_clean":"2024-04-25T19:15:20.405244+0000","last_became_active":"2024-04-25T19:15:19.431905+0000","last_became_peered":"2024-04-25T19:15:19.431905+0000","last_unstale":"2024-04-25T19:15:20.405244+0000","last_undegraded":"2024-04-25T19:15:20.405244+0000","last_fullsized":"2024-04-25T19:15:20.405244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:52.752469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'7","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:47.501382+0000","last_change":"2024-04-25T19:15:18.423144+0000","last_active":"2024-04-25T19:15:47.501382+0000","last_peered":"2024-04-25T19:15:47.501382+0000","last_clean":"2024-04-25T19:15:47.501382+0000","last_became_active":"2024-04-25T19:15:18.422838+0000","last_became_peered":"2024-04-25T19:15:18.422838+0000","last_unstale":"2024-04-25T19:15:47.501382+0000","last_undegraded":"2024-04-25T19:15:47.501382+0000","last_fullsized":"2024-04-25T19:15:47.501382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:28.026912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804520+0000","last_change":"2024-04-25T19:15:19.420582+0000","last_active":"2024-04-25T19:15:20.804520+0000","last_peered":"2024-04-25T19:15:20.804520+0000","last_clean":"2024-04-25T19:15:20.804520+0000","last_became_active":"2024-04-25T19:15:19.420270+0000","last_became_peered":"2024-04-25T19:15:19.420270+0000","last_unstale":"2024-04-25T19:15:20.804520+0000","last_undegraded":"2024-04-25T19:15:20.804520+0000","last_fullsized":"2024-04-25T19:15:20.804520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:06.827819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:29.474676+0000","last_change":"2024-04-25T19:15:18.425469+0000","last_active":"2024-04-25T19:15:29.474676+0000","last_peered":"2024-04-25T19:15:29.474676+0000","last_clean":"2024-04-25T19:15:29.474676+0000","last_became_active":"2024-04-25T19:15:18.425084+0000","last_became_peered":"2024-04-25T19:15:18.425084+0000","last_unstale":"2024-04-25T19:15:29.474676+0000","last_undegraded":"2024-04-25T19:15:29.474676+0000","last_fullsized":"2024-04-25T19:15:29.474676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:14.363158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.822624+0000","last_change":"2024-04-25T19:15:19.425159+0000","last_active":"2024-04-25T19:15:20.822624+0000","last_peered":"2024-04-25T19:15:20.822624+0000","last_clean":"2024-04-25T19:15:20.822624+0000","last_became_active":"2024-04-25T19:15:19.424847+0000","last_became_peered":"2024-04-25T19:15:19.424847+0000","last_unstale":"2024-04-25T19:15:20.822624+0000","last_undegraded":"2024-04-25T19:15:20.822624+0000","last_fullsized":"2024-04-25T19:15:20.822624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:28.715286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"21'7","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:47.500688+0000","last_change":"2024-04-25T19:15:18.424645+0000","last_active":"2024-04-25T19:15:47.500688+0000","last_peered":"2024-04-25T19:15:47.500688+0000","last_clean":"2024-04-25T19:15:47.500688+0000","last_became_active":"2024-04-25T19:15:18.424241+0000","last_became_peered":"2024-04-25T19:15:18.424241+0000","last_unstale":"2024-04-25T19:15:47.500688+0000","last_undegraded":"2024-04-25T19:15:47.500688+0000","last_fullsized":"2024-04-25T19:15:47.500688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:29.373363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405640+0000","last_change":"2024-04-25T19:15:19.421095+0000","last_active":"2024-04-25T19:15:20.405640+0000","last_peered":"2024-04-25T19:15:20.405640+0000","last_clean":"2024-04-25T19:15:20.405640+0000","last_became_active":"2024-04-25T19:15:19.420795+0000","last_became_peered":"2024-04-25T19:15:19.420795+0000","last_unstale":"2024-04-25T19:15:20.405640+0000","last_undegraded":"2024-04-25T19:15:20.405640+0000","last_fullsized":"2024-04-25T19:15:20.405640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:13.061506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:41.490164+0000","last_change":"2024-04-25T19:15:18.423841+0000","last_active":"2024-04-25T19:15:41.490164+0000","last_peered":"2024-04-25T19:15:41.490164+0000","last_clean":"2024-04-25T19:15:41.490164+0000","last_became_active":"2024-04-25T19:15:18.423500+0000","last_became_peered":"2024-04-25T19:15:18.423500+0000","last_unstale":"2024-04-25T19:15:41.490164+0000","last_undegraded":"2024-04-25T19:15:41.490164+0000","last_fullsized":"2024-04-25T19:15:41.490164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:58.933085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405564+0000","last_change":"2024-04-25T19:15:19.416232+0000","last_active":"2024-04-25T19:15:20.405564+0000","last_peered":"2024-04-25T19:15:20.405564+0000","last_clean":"2024-04-25T19:15:20.405564+0000","last_became_active":"2024-04-25T19:15:19.415828+0000","last_became_peered":"2024-04-25T19:15:19.415828+0000","last_unstale":"2024-04-25T19:15:20.405564+0000","last_undegraded":"2024-04-25T19:15:20.405564+0000","last_fullsized":"2024-04-25T19:15:20.405564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:59.751840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:29.474678+0000","last_change":"2024-04-25T19:15:18.423918+0000","last_active":"2024-04-25T19:15:29.474678+0000","last_peered":"2024-04-25T19:15:29.474678+0000","last_clean":"2024-04-25T19:15:29.474678+0000","last_became_active":"2024-04-25T19:15:18.423588+0000","last_became_peered":"2024-04-25T19:15:18.423588+0000","last_unstale":"2024-04-25T19:15:29.474678+0000","last_undegraded":"2024-04-25T19:15:29.474678+0000","last_fullsized":"2024-04-25T19:15:29.474678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:16.378285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405402+0000","last_change":"2024-04-25T19:15:19.422984+0000","last_active":"2024-04-25T19:15:20.405402+0000","last_peered":"2024-04-25T19:15:20.405402+0000","last_clean":"2024-04-25T19:15:20.405402+0000","last_became_active":"2024-04-25T19:15:19.422651+0000","last_became_peered":"2024-04-25T19:15:19.422651+0000","last_unstale":"2024-04-25T19:15:20.405402+0000","last_undegraded":"2024-04-25T19:15:20.405402+0000","last_fullsized":"2024-04-25T19:15:20.405402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:54.984914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848417+0000","last_change":"2024-04-25T19:15:19.423982+0000","last_active":"2024-04-25T19:15:20.848417+0000","last_peered":"2024-04-25T19:15:20.848417+0000","last_clean":"2024-04-25T19:15:20.848417+0000","last_became_active":"2024-04-25T19:15:19.423341+0000","last_became_peered":"2024-04-25T19:15:19.423341+0000","last_unstale":"2024-04-25T19:15:20.848417+0000","last_undegraded":"2024-04-25T19:15:20.848417+0000","last_fullsized":"2024-04-25T19:15:20.848417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:51.808352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:47.500561+0000","last_change":"2024-04-25T19:15:18.424455+0000","last_active":"2024-04-25T19:15:47.500561+0000","last_peered":"2024-04-25T19:15:47.500561+0000","last_clean":"2024-04-25T19:15:47.500561+0000","last_became_active":"2024-04-25T19:15:18.424119+0000","last_became_peered":"2024-04-25T19:15:18.424119+0000","last_unstale":"2024-04-25T19:15:47.500561+0000","last_undegraded":"2024-04-25T19:15:47.500561+0000","last_fullsized":"2024-04-25T19:15:47.500561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:43.135567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.848336+0000","last_change":"2024-04-25T19:15:19.424064+0000","last_active":"2024-04-25T19:15:20.848336+0000","last_peered":"2024-04-25T19:15:20.848336+0000","last_clean":"2024-04-25T19:15:20.848336+0000","last_became_active":"2024-04-25T19:15:19.423187+0000","last_became_peered":"2024-04-25T19:15:19.423187+0000","last_unstale":"2024-04-25T19:15:20.848336+0000","last_undegraded":"2024-04-25T19:15:20.848336+0000","last_fullsized":"2024-04-25T19:15:20.848336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:43:31.590700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'11","reported_seq":25,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:47.501382+0000","last_change":"2024-04-25T19:15:18.423151+0000","last_active":"2024-04-25T19:15:47.501382+0000","last_peered":"2024-04-25T19:15:47.501382+0000","last_clean":"2024-04-25T19:15:47.501382+0000","last_became_active":"2024-04-25T19:15:18.422831+0000","last_became_peered":"2024-04-25T19:15:18.422831+0000","last_unstale":"2024-04-25T19:15:47.501382+0000","last_undegraded":"2024-04-25T19:15:47.501382+0000","last_fullsized":"2024-04-25T19:15:47.501382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:43.244963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.804534+0000","last_change":"2024-04-25T19:15:19.416717+0000","last_active":"2024-04-25T19:15:20.804534+0000","last_peered":"2024-04-25T19:15:20.804534+0000","last_clean":"2024-04-25T19:15:20.804534+0000","last_became_active":"2024-04-25T19:15:19.416163+0000","last_became_peered":"2024-04-25T19:15:19.416163+0000","last_unstale":"2024-04-25T19:15:20.804534+0000","last_undegraded":"2024-04-25T19:15:20.804534+0000","last_fullsized":"2024-04-25T19:15:20.804534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:46.955159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:24.414530+0000","last_change":"2024-04-25T19:15:18.422875+0000","last_active":"2024-04-25T19:15:24.414530+0000","last_peered":"2024-04-25T19:15:24.414530+0000","last_clean":"2024-04-25T19:15:24.414530+0000","last_became_active":"2024-04-25T19:15:18.422489+0000","last_became_peered":"2024-04-25T19:15:18.422489+0000","last_unstale":"2024-04-25T19:15:24.414530+0000","last_undegraded":"2024-04-25T19:15:24.414530+0000","last_fullsized":"2024-04-25T19:15:24.414530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:30.602500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:20.405477+0000","last_change":"2024-04-25T19:15:19.416987+0000","last_active":"2024-04-25T19:15:20.405477+0000","last_peered":"2024-04-25T19:15:20.405477+0000","last_clean":"2024-04-25T19:15:20.405477+0000","last_became_active":"2024-04-25T19:15:19.414173+0000","last_became_peered":"2024-04-25T19:15:19.414173+0000","last_unstale":"2024-04-25T19:15:20.405477+0000","last_undegraded":"2024-04-25T19:15:20.405477+0000","last_fullsized":"2024-04-25T19:15:20.405477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:32:43.188653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'13","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-25T19:15:47.500644+0000","last_change":"2024-04-25T19:15:18.423867+0000","last_active":"2024-04-25T19:15:47.500644+0000","last_peered":"2024-04-25T19:15:47.500644+0000","last_clean":"2024-04-25T19:15:47.500644+0000","last_became_active":"2024-04-25T19:15:18.423532+0000","last_became_peered":"2024-04-25T19:15:18.423532+0000","last_unstale":"2024-04-25T19:15:47.500644+0000","last_undegraded":"2024-04-25T19:15:47.500644+0000","last_fullsized":"2024-04-25T19:15:47.500644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:04.531338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8034,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":204,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":16068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":178,"ondisk_log_size":178,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64838,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":319488,"data_stored":668073,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":78007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":79537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":241664,"data_stored":76935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":249856,"data_stored":78041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":249856,"data_stored":77067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":77137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":666847,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T19:15:58.850 INFO:tasks.mon_thrash.ceph_manager:clean! 2024-04-25T19:15:58.851 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:15:58.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.989+0000 7f8e4084c640 1 Processor -- start 2024-04-25T19:15:58.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.989+0000 7f8e4084c640 1 -- start start 2024-04-25T19:15:58.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.991+0000 7f8e4084c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8e3c06de20 0x7f8e3c06d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.991+0000 7f8e4084c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e3c156f10 0x7f8e3c15d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.991+0000 7f8e4084c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e3c06e310 con 0x7f8e3c0cc9e0 2024-04-25T19:15:58.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.991+0000 7f8e4084c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f8e3c15d2f0 con 0x7f8e3c06de20 2024-04-25T19:15:58.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.991+0000 7f8e4084c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8e3c15dcc0 con 0x7f8e3c156f10 2024-04-25T19:15:58.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.991+0000 7f8e3ad9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e3c156f10 0x7f8e3c15d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.991+0000 7f8e3ad9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e3c156f10 0x7f8e3c15d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:44676/0 (socket says 172.21.15.76:44676) 2024-04-25T19:15:58.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.991+0000 7f8e3ad9d640 1 -- 172.21.15.76:0/407065446 learned_addr learned my addr 172.21.15.76:0/407065446 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:58.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.991+0000 7f8e3a59c640 1 --2- 172.21.15.76:0/407065446 >> v2:172.21.15.105:3300/0 conn(0x7f8e3c06de20 0x7f8e3c06d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.991+0000 7f8e3ad9d640 1 -- 172.21.15.76:0/407065446 >> v2:172.21.15.105:3300/0 conn(0x7f8e3c06de20 msgr2=0x7f8e3c06d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:58.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.991+0000 7f8e3ad9d640 1 --2- 172.21.15.76:0/407065446 >> v2:172.21.15.105:3300/0 conn(0x7f8e3c06de20 0x7f8e3c06d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.991+0000 7f8e3ad9d640 1 -- 172.21.15.76:0/407065446 >> v1:172.21.15.105:6789/0 conn(0x7f8e3c0cc9e0 legacy=0x7f8e3c0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:58.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.991+0000 7f8e3ad9d640 1 -- 172.21.15.76:0/407065446 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3c15ccc0 con 0x7f8e3c156f10 2024-04-25T19:15:58.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.992+0000 7f8e3ad9d640 1 --2- 172.21.15.76:0/407065446 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e3c156f10 0x7f8e3c15d780 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8e34009870 tx=0x7f8e3402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7e66ffa93ec57bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:58.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.992+0000 7f8e3959a640 1 -- 172.21.15.76:0/407065446 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e34037070 con 0x7f8e3c156f10 2024-04-25T19:15:58.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.992+0000 7f8e3959a640 1 -- 172.21.15.76:0/407065446 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e34032440 con 0x7f8e3c156f10 2024-04-25T19:15:58.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.992+0000 7f8e3959a640 1 -- 172.21.15.76:0/407065446 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e34032810 con 0x7f8e3c156f10 2024-04-25T19:15:58.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.993+0000 7f8e4084c640 1 -- 172.21.15.76:0/407065446 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e3c156f10 msgr2=0x7f8e3c15d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:58.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.993+0000 7f8e4084c640 1 --2- 172.21.15.76:0/407065446 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e3c156f10 0x7f8e3c15d780 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8e34009870 tx=0x7f8e3402a1c0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.993+0000 7f8e4084c640 1 -- 172.21.15.76:0/407065446 shutdown_connections 2024-04-25T19:15:58.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.993+0000 7f8e4084c640 1 --2- 172.21.15.76:0/407065446 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e3c156f10 0x7f8e3c15d780 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.993+0000 7f8e4084c640 1 --2- 172.21.15.76:0/407065446 >> v2:172.21.15.105:3300/0 conn(0x7f8e3c06de20 0x7f8e3c06d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:58.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.993+0000 7f8e4084c640 1 -- 172.21.15.76:0/407065446 >> 172.21.15.76:0/407065446 conn(0x7f8e3c0684e0 msgr2=0x7f8e3c0688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:58.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.993+0000 7f8e4084c640 1 -- 172.21.15.76:0/407065446 shutdown_connections 2024-04-25T19:15:58.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.993+0000 7f8e4084c640 1 -- 172.21.15.76:0/407065446 wait complete. 2024-04-25T19:15:58.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.995+0000 7f8e4084c640 1 Processor -- start 2024-04-25T19:15:58.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.995+0000 7f8e4084c640 1 -- start start 2024-04-25T19:15:58.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.996+0000 7f8e4084c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e3c06de20 0x7f8e3c1522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.996+0000 7f8e4084c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e3c0cc9e0 0x7f8e3c150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.996+0000 7f8e4084c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e3c156f10 0x7f8e3c1f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:58.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.996+0000 7f8e4084c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c15cf80 con 0x7f8e3c0cc9e0 2024-04-25T19:15:58.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.996+0000 7f8e4084c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c0cb980 con 0x7f8e3c156f10 2024-04-25T19:15:58.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.996+0000 7f8e4084c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8e3c157320 con 0x7f8e3c06de20 2024-04-25T19:15:58.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.996+0000 7f8e39d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e3c0cc9e0 0x7f8e3c150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:58.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.996+0000 7f8e39d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e3c0cc9e0 0x7f8e3c150ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38170/0 (socket says 172.21.15.76:38170) 2024-04-25T19:15:59.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.996+0000 7f8e39d9b640 1 -- 172.21.15.76:0/1982646412 learned_addr learned my addr 172.21.15.76:0/1982646412 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:59.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.996+0000 7f8e3ad9d640 1 --2- 172.21.15.76:0/1982646412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e3c156f10 0x7f8e3c1f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:59.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.996+0000 7f8e3a59c640 1 --2- 172.21.15.76:0/1982646412 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e3c06de20 0x7f8e3c1522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:59.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.997+0000 7f8e39d9b640 1 -- 172.21.15.76:0/1982646412 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e3c06de20 msgr2=0x7f8e3c1522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:59.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.997+0000 7f8e39d9b640 1 --2- 172.21.15.76:0/1982646412 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e3c06de20 0x7f8e3c1522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.997+0000 7f8e39d9b640 1 -- 172.21.15.76:0/1982646412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e3c156f10 msgr2=0x7f8e3c1f6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:59.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.997+0000 7f8e39d9b640 1 --2- 172.21.15.76:0/1982646412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e3c156f10 0x7f8e3c1f6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.997+0000 7f8e39d9b640 1 -- 172.21.15.76:0/1982646412 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3c157690 con 0x7f8e3c0cc9e0 2024-04-25T19:15:59.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.997+0000 7f8e39d9b640 1 --2- 172.21.15.76:0/1982646412 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e3c0cc9e0 0x7f8e3c150ee0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000c3f0 tx=0x7f8e3000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:59.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.997+0000 7f8e2affd640 1 -- 172.21.15.76:0/1982646412 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e300052d0 con 0x7f8e3c0cc9e0 2024-04-25T19:15:59.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.997+0000 7f8e2affd640 1 -- 172.21.15.76:0/1982646412 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e30005490 con 0x7f8e3c0cc9e0 2024-04-25T19:15:59.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.997+0000 7f8e4084c640 1 -- 172.21.15.76:0/1982646412 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e3c152810 con 0x7f8e3c0cc9e0 2024-04-25T19:15:59.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.997+0000 7f8e2affd640 1 -- 172.21.15.76:0/1982646412 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e30007260 con 0x7f8e3c0cc9e0 2024-04-25T19:15:59.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.997+0000 7f8e4084c640 1 -- 172.21.15.76:0/1982646412 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e3c157900 con 0x7f8e3c0cc9e0 2024-04-25T19:15:59.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.998+0000 7f8e4084c640 1 -- 172.21.15.76:0/1982646412 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e3c0cd3d0 con 0x7f8e3c0cc9e0 2024-04-25T19:15:59.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.999+0000 7f8e2affd640 1 -- 172.21.15.76:0/1982646412 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8e30007400 con 0x7f8e3c0cc9e0 2024-04-25T19:15:59.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.999+0000 7f8e2affd640 1 --2- 172.21.15.76:0/1982646412 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8e1c0744b0 0x7f8e1c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:59.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.999+0000 7f8e2affd640 1 -- 172.21.15.76:0/1982646412 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8e30094610 con 0x7f8e3c0cc9e0 2024-04-25T19:15:59.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:58.999+0000 7f8e3a59c640 1 --2- 172.21.15.76:0/1982646412 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8e1c0744b0 0x7f8e1c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:59.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.000+0000 7f8e3a59c640 1 --2- 172.21.15.76:0/1982646412 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8e1c0744b0 0x7f8e1c0769c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c011340 tx=0x7f8e2c00b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:59.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.004+0000 7f8e2affd640 1 -- 172.21.15.76:0/1982646412 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8e3c0cd3d0 con 0x7f8e3c0cc9e0 2024-04-25T19:15:59.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.180+0000 7f8e4084c640 1 -- 172.21.15.76:0/1982646412 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e3c077e30 con 0x7f8e3c0cc9e0 2024-04-25T19:15:59.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.181+0000 7f8e2affd640 1 -- 172.21.15.76:0/1982646412 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7f8e3c077e30 con 0x7f8e3c0cc9e0 2024-04-25T19:15:59.183 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:15:59.183 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":21,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","created":"2024-04-25T19:15:01.693137+0000","modified":"2024-04-25T19:15:20.401211+0000","last_up_change":"2024-04-25T19:15:11.891681+0000","last_in_change":"2024-04-25T19:15:03.898065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:15:13.846514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:15:16.606533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:15:17.719197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72a16c02-48f9-45aa-9447-fcf755ebf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409574998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409574998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409574998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409574998}]},"public_addr":"172.21.15.76:6817/1409574998","cluster_addr":"172.21.15.76:6819/1409574998","heartbeat_back_addr":"172.21.15.76:6823/1409574998","heartbeat_front_addr":"172.21.15.76:6821/1409574998","state":["exists","up"]},{"osd":1,"uuid":"818d50f0-76c6-45f8-bf79-5e304f14c5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6801","nonce":1302663821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6803","nonce":1302663821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6807","nonce":1302663821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6805","nonce":1302663821}]},"public_addr":"172.21.15.76:6801/1302663821","cluster_addr":"172.21.15.76:6803/1302663821","heartbeat_back_addr":"172.21.15.76:6807/1302663821","heartbeat_front_addr":"172.21.15.76:6805/1302663821","state":["exists","up"]},{"osd":2,"uuid":"5714c8f5-ae23-473c-8cf1-db73811acded","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6809","nonce":1992938181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6811","nonce":1992938181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6815","nonce":1992938181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6813","nonce":1992938181}]},"public_addr":"172.21.15.76:6809/1992938181","cluster_addr":"172.21.15.76:6811/1992938181","heartbeat_back_addr":"172.21.15.76:6815/1992938181","heartbeat_front_addr":"172.21.15.76:6813/1992938181","state":["exists","up"]},{"osd":3,"uuid":"13204cdb-d8e6-4f97-bc6e-9561f442decf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6825","nonce":3418257163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6827","nonce":3418257163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6831","nonce":3418257163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6829","nonce":3418257163}]},"public_addr":"172.21.15.76:6825/3418257163","cluster_addr":"172.21.15.76:6827/3418257163","heartbeat_back_addr":"172.21.15.76:6831/3418257163","heartbeat_front_addr":"172.21.15.76:6829/3418257163","state":["exists","up"]},{"osd":4,"uuid":"5bea5166-1986-4d56-b5f5-ecc8cbe7f3ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6801","nonce":2125002468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6803","nonce":2125002468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6807","nonce":2125002468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6805","nonce":2125002468}]},"public_addr":"172.21.15.105:6801/2125002468","cluster_addr":"172.21.15.105:6803/2125002468","heartbeat_back_addr":"172.21.15.105:6807/2125002468","heartbeat_front_addr":"172.21.15.105:6805/2125002468","state":["exists","up"]},{"osd":5,"uuid":"96f89dcc-270d-4421-b8e8-3b92ba5f9844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6819","nonce":3413103121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6821","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6823","nonce":3413103121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6831","nonce":3413103121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6828","nonce":3413103121}]},"public_addr":"172.21.15.105:6819/3413103121","cluster_addr":"172.21.15.105:6823/3413103121","heartbeat_back_addr":"172.21.15.105:6831/3413103121","heartbeat_front_addr":"172.21.15.105:6828/3413103121","state":["exists","up"]},{"osd":6,"uuid":"93eb087a-d9ba-4096-b85d-b74d96e819a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6809","nonce":2721832954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6811","nonce":2721832954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6815","nonce":2721832954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6813","nonce":2721832954}]},"public_addr":"172.21.15.105:6809/2721832954","cluster_addr":"172.21.15.105:6811/2721832954","heartbeat_back_addr":"172.21.15.105:6815/2721832954","heartbeat_front_addr":"172.21.15.105:6813/2721832954","state":["exists","up"]},{"osd":7,"uuid":"2f80f790-ad2d-460e-a37d-47bf1771ef43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6818","nonce":548350430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6822","nonce":548350430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6829","nonce":548350430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6825","nonce":548350430}]},"public_addr":"172.21.15.105:6818/548350430","cluster_addr":"172.21.15.105:6822/548350430","heartbeat_back_addr":"172.21.15.105:6829/548350430","heartbeat_front_addr":"172.21.15.105:6825/548350430","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.106901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.072819+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.126865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.053464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:08.982614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.104825+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.112714+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.107387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:15:59.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.183+0000 7f8e4084c640 1 -- 172.21.15.76:0/1982646412 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8e1c0744b0 msgr2=0x7f8e1c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:59.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.183+0000 7f8e4084c640 1 --2- 172.21.15.76:0/1982646412 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8e1c0744b0 0x7f8e1c0769c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c011340 tx=0x7f8e2c00b040 comp rx=0 tx=0).stop 2024-04-25T19:15:59.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.183+0000 7f8e4084c640 1 -- 172.21.15.76:0/1982646412 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e3c0cc9e0 msgr2=0x7f8e3c150ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:59.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.183+0000 7f8e4084c640 1 --2- 172.21.15.76:0/1982646412 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e3c0cc9e0 0x7f8e3c150ee0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000c3f0 tx=0x7f8e3000c8b0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.183+0000 7f8e4084c640 1 -- 172.21.15.76:0/1982646412 shutdown_connections 2024-04-25T19:15:59.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.183+0000 7f8e4084c640 1 --2- 172.21.15.76:0/1982646412 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8e1c0744b0 0x7f8e1c0769c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.183+0000 7f8e4084c640 1 --2- 172.21.15.76:0/1982646412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e3c156f10 0x7f8e3c1f6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.183+0000 7f8e4084c640 1 --2- 172.21.15.76:0/1982646412 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e3c0cc9e0 0x7f8e3c150ee0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.183+0000 7f8e4084c640 1 --2- 172.21.15.76:0/1982646412 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8e3c06de20 0x7f8e3c1522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.183+0000 7f8e4084c640 1 -- 172.21.15.76:0/1982646412 >> 172.21.15.76:0/1982646412 conn(0x7f8e3c0684e0 msgr2=0x7f8e3c15a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:59.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.183+0000 7f8e4084c640 1 -- 172.21.15.76:0/1982646412 shutdown_connections 2024-04-25T19:15:59.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.183+0000 7f8e4084c640 1 -- 172.21.15.76:0/1982646412 wait complete. 2024-04-25T19:15:59.196 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T19:15:13.846514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T19:15:16.606533+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T19:15:17.719197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T19:15:59.196 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:15:59.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.305+0000 7fd06e820640 1 Processor -- start 2024-04-25T19:15:59.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.305+0000 7fd06e820640 1 -- start start 2024-04-25T19:15:59.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.307+0000 7fd06e820640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd06806de20 0x7fd06806d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:59.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.307+0000 7fd06e820640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd068156f10 0x7fd06815d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:59.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.307+0000 7fd06e820640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd06806e310 con 0x7fd0680cc9e0 2024-04-25T19:15:59.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.307+0000 7fd06e820640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd06815d2f0 con 0x7fd06806de20 2024-04-25T19:15:59.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.307+0000 7fd06e820640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fd06815dcc0 con 0x7fd068156f10 2024-04-25T19:15:59.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.307+0000 7fd067fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd06806de20 0x7fd06806d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:59.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.307+0000 7fd067fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd06806de20 0x7fd06806d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38180/0 (socket says 172.21.15.76:38180) 2024-04-25T19:15:59.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.307+0000 7fd067fff640 1 -- 172.21.15.76:0/3193238335 learned_addr learned my addr 172.21.15.76:0/3193238335 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:59.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.307+0000 7fd06cdbd640 1 --2- 172.21.15.76:0/3193238335 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd068156f10 0x7fd06815d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:59.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.307+0000 7fd067fff640 1 -- 172.21.15.76:0/3193238335 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd068156f10 msgr2=0x7fd06815d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:59.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.307+0000 7fd067fff640 1 --2- 172.21.15.76:0/3193238335 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd068156f10 0x7fd06815d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.307+0000 7fd067fff640 1 -- 172.21.15.76:0/3193238335 >> v1:172.21.15.105:6789/0 conn(0x7fd0680cc9e0 legacy=0x7fd0680cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:59.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.307+0000 7fd067fff640 1 -- 172.21.15.76:0/3193238335 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06815ccc0 con 0x7fd06806de20 2024-04-25T19:15:59.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.307+0000 7fd06cdbd640 1 --2- 172.21.15.76:0/3193238335 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd068156f10 0x7fd06815d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:59.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.308+0000 7fd067fff640 1 --2- 172.21.15.76:0/3193238335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd06806de20 0x7fd06806d6d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd058009870 tx=0x7fd05802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7507d5a05696a22f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:59.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.308+0000 7fd066ffd640 1 -- 172.21.15.76:0/3193238335 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058037070 con 0x7fd06806de20 2024-04-25T19:15:59.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.308+0000 7fd066ffd640 1 -- 172.21.15.76:0/3193238335 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd058032440 con 0x7fd06806de20 2024-04-25T19:15:59.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.308+0000 7fd066ffd640 1 -- 172.21.15.76:0/3193238335 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058032810 con 0x7fd06806de20 2024-04-25T19:15:59.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.309+0000 7fd06e820640 1 -- 172.21.15.76:0/3193238335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd06806de20 msgr2=0x7fd06806d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:59.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.309+0000 7fd06e820640 1 --2- 172.21.15.76:0/3193238335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd06806de20 0x7fd06806d6d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd058009870 tx=0x7fd05802a1c0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.309+0000 7fd06e820640 1 -- 172.21.15.76:0/3193238335 shutdown_connections 2024-04-25T19:15:59.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.309+0000 7fd06e820640 1 --2- 172.21.15.76:0/3193238335 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd068156f10 0x7fd06815d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.309+0000 7fd06e820640 1 --2- 172.21.15.76:0/3193238335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd06806de20 0x7fd06806d6d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.309+0000 7fd06e820640 1 -- 172.21.15.76:0/3193238335 >> 172.21.15.76:0/3193238335 conn(0x7fd0680684e0 msgr2=0x7fd0680688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:59.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.309+0000 7fd06e820640 1 -- 172.21.15.76:0/3193238335 shutdown_connections 2024-04-25T19:15:59.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.309+0000 7fd06e820640 1 -- 172.21.15.76:0/3193238335 wait complete. 2024-04-25T19:15:59.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.310+0000 7fd06e820640 1 Processor -- start 2024-04-25T19:15:59.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.311+0000 7fd06e820640 1 -- start start 2024-04-25T19:15:59.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.312+0000 7fd06e820640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd06806de20 0x7fd0681f70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:59.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.312+0000 7fd06e820640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0680cc9e0 0x7fd0681f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:59.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.312+0000 7fd06e820640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd068156f10 0x7fd0681f3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:59.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.312+0000 7fd06e820640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd06815cf80 con 0x7fd068156f10 2024-04-25T19:15:59.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.312+0000 7fd06e820640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd0680cb980 con 0x7fd0680cc9e0 2024-04-25T19:15:59.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.312+0000 7fd06e820640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fd068157320 con 0x7fd06806de20 2024-04-25T19:15:59.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.312+0000 7fd06cdbd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd068156f10 0x7fd0681f3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:59.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.312+0000 7fd06cdbd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd068156f10 0x7fd0681f3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38182/0 (socket says 172.21.15.76:38182) 2024-04-25T19:15:59.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.312+0000 7fd06cdbd640 1 -- 172.21.15.76:0/1924022056 learned_addr learned my addr 172.21.15.76:0/1924022056 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:59.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.312+0000 7fd0677fe640 1 --2- 172.21.15.76:0/1924022056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0680cc9e0 0x7fd0681f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:59.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.312+0000 7fd067fff640 1 --2- 172.21.15.76:0/1924022056 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd06806de20 0x7fd0681f70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:59.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.312+0000 7fd06cdbd640 1 -- 172.21.15.76:0/1924022056 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd06806de20 msgr2=0x7fd0681f70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:59.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.312+0000 7fd06cdbd640 1 --2- 172.21.15.76:0/1924022056 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd06806de20 0x7fd0681f70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.312+0000 7fd06cdbd640 1 -- 172.21.15.76:0/1924022056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0680cc9e0 msgr2=0x7fd0681f2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:59.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.312+0000 7fd06cdbd640 1 --2- 172.21.15.76:0/1924022056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0680cc9e0 0x7fd0681f2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.312+0000 7fd06cdbd640 1 -- 172.21.15.76:0/1924022056 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd068152660 con 0x7fd068156f10 2024-04-25T19:15:59.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.312+0000 7fd06cdbd640 1 --2- 172.21.15.76:0/1924022056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd068156f10 0x7fd0681f3970 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd060007dc0 tx=0x7fd06000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:59.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.312+0000 7fd064ff9640 1 -- 172.21.15.76:0/1924022056 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06001a070 con 0x7fd068156f10 2024-04-25T19:15:59.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.313+0000 7fd064ff9640 1 -- 172.21.15.76:0/1924022056 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd060003990 con 0x7fd068156f10 2024-04-25T19:15:59.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.313+0000 7fd06e820640 1 -- 172.21.15.76:0/1924022056 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd06815a0e0 con 0x7fd068156f10 2024-04-25T19:15:59.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.313+0000 7fd064ff9640 1 -- 172.21.15.76:0/1924022056 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd060003d80 con 0x7fd068156f10 2024-04-25T19:15:59.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.313+0000 7fd06e820640 1 -- 172.21.15.76:0/1924022056 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0681f68a0 con 0x7fd068156f10 2024-04-25T19:15:59.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.313+0000 7fd06e820640 1 -- 172.21.15.76:0/1924022056 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd06806a390 con 0x7fd068156f10 2024-04-25T19:15:59.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.314+0000 7fd064ff9640 1 -- 172.21.15.76:0/1924022056 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd06001d070 con 0x7fd068156f10 2024-04-25T19:15:59.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.314+0000 7fd064ff9640 1 --2- 172.21.15.76:0/1924022056 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fd0540744b0 0x7fd0540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:59.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.315+0000 7fd064ff9640 1 -- 172.21.15.76:0/1924022056 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd0600953e0 con 0x7fd068156f10 2024-04-25T19:15:59.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.315+0000 7fd067fff640 1 --2- 172.21.15.76:0/1924022056 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fd0540744b0 0x7fd0540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:59.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.315+0000 7fd067fff640 1 --2- 172.21.15.76:0/1924022056 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fd0540744b0 0x7fd0540769c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd058034000 tx=0x7fd05802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:59.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.319+0000 7fd064ff9640 1 -- 172.21.15.76:0/1924022056 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd060010070 con 0x7fd068156f10 2024-04-25T19:15:59.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.498+0000 7fd06e820640 1 -- 172.21.15.76:0/1924022056 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd0680cf100 con 0x7fd068156f10 2024-04-25T19:15:59.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.499+0000 7fd064ff9640 1 -- 172.21.15.76:0/1924022056 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fd06005f5b0 con 0x7fd068156f10 2024-04-25T19:15:59.501 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-04-25T19:15:59.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.501+0000 7fd06e820640 1 -- 172.21.15.76:0/1924022056 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fd0540744b0 msgr2=0x7fd0540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:59.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.501+0000 7fd06e820640 1 --2- 172.21.15.76:0/1924022056 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fd0540744b0 0x7fd0540769c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd058034000 tx=0x7fd05802e000 comp rx=0 tx=0).stop 2024-04-25T19:15:59.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.501+0000 7fd06e820640 1 -- 172.21.15.76:0/1924022056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd068156f10 msgr2=0x7fd0681f3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:59.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.501+0000 7fd06e820640 1 --2- 172.21.15.76:0/1924022056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd068156f10 0x7fd0681f3970 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd060007dc0 tx=0x7fd06000d4a0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.501+0000 7fd06e820640 1 -- 172.21.15.76:0/1924022056 shutdown_connections 2024-04-25T19:15:59.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.501+0000 7fd06e820640 1 --2- 172.21.15.76:0/1924022056 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fd0540744b0 0x7fd0540769c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.501+0000 7fd06e820640 1 --2- 172.21.15.76:0/1924022056 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd068156f10 0x7fd0681f3970 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.501+0000 7fd06e820640 1 --2- 172.21.15.76:0/1924022056 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0680cc9e0 0x7fd0681f2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.501+0000 7fd06e820640 1 --2- 172.21.15.76:0/1924022056 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd06806de20 0x7fd0681f70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.501+0000 7fd06e820640 1 -- 172.21.15.76:0/1924022056 >> 172.21.15.76:0/1924022056 conn(0x7fd0680684e0 msgr2=0x7fd06815bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:59.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.501+0000 7fd06e820640 1 -- 172.21.15.76:0/1924022056 shutdown_connections 2024-04-25T19:15:59.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.501+0000 7fd06e820640 1 -- 172.21.15.76:0/1924022056 wait complete. 2024-04-25T19:15:59.513 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T19:15:59.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.623+0000 7fe7e4946640 1 Processor -- start 2024-04-25T19:15:59.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.623+0000 7fe7e4946640 1 -- start start 2024-04-25T19:15:59.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.624+0000 7fe7e4946640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7e006e0c0 0x7fe7e006e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:59.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.624+0000 7fe7e4946640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7e015d250 con 0x7fe7e01609c0 2024-04-25T19:15:59.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.624+0000 7fe7e4946640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7e015d510 con 0x7fe7e015cd10 2024-04-25T19:15:59.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.624+0000 7fe7e4946640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fe7e015d7f0 con 0x7fe7e006e0c0 2024-04-25T19:15:59.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.624+0000 7fe7ded9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe7e01609c0 0x7fe7e0072ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:32994/0 (socket says 172.21.15.76:32994) 2024-04-25T19:15:59.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.625+0000 7fe7ded9d640 1 -- 172.21.15.76:0/335777142 learned_addr learned my addr 172.21.15.76:0/335777142 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:59.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.625+0000 7fe7d7fff640 1 --2- 172.21.15.76:0/335777142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7e006e0c0 0x7fe7e006e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:59.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.625+0000 7fe7ddd9b640 1 -- 172.21.15.76:0/335777142 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 154895194 0 0) 0x7fe7e015d250 con 0x7fe7e01609c0 2024-04-25T19:15:59.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.625+0000 7fe7ddd9b640 1 -- 172.21.15.76:0/335777142 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7cc003520 con 0x7fe7e01609c0 2024-04-25T19:15:59.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.625+0000 7fe7ddd9b640 1 -- 172.21.15.76:0/335777142 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 82474398 0 0) 0x7fe7e015d510 con 0x7fe7e015cd10 2024-04-25T19:15:59.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.625+0000 7fe7ddd9b640 1 -- 172.21.15.76:0/335777142 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7e015d250 con 0x7fe7e015cd10 2024-04-25T19:15:59.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.625+0000 7fe7ddd9b640 1 -- 172.21.15.76:0/335777142 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2923554648 0 0) 0x7fe7cc003520 con 0x7fe7e01609c0 2024-04-25T19:15:59.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.625+0000 7fe7d7fff640 1 -- 172.21.15.76:0/335777142 >> v1:172.21.15.105:6789/0 conn(0x7fe7e015cd10 legacy=0x7fe7e0156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:59.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.625+0000 7fe7d7fff640 1 -- 172.21.15.76:0/335777142 >> v1:172.21.15.76:6789/0 conn(0x7fe7e01609c0 legacy=0x7fe7e0072ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:59.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.625+0000 7fe7d7fff640 1 -- 172.21.15.76:0/335777142 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e015dad0 con 0x7fe7e006e0c0 2024-04-25T19:15:59.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.626+0000 7fe7d7fff640 1 --2- 172.21.15.76:0/335777142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7e006e0c0 0x7fe7e006e4d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe7c80097d0 tx=0x7fe7c802a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd02d8a4b7b22fd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:59.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.626+0000 7fe7ddd9b640 1 -- 172.21.15.76:0/335777142 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8037070 con 0x7fe7e006e0c0 2024-04-25T19:15:59.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.626+0000 7fe7ddd9b640 1 -- 172.21.15.76:0/335777142 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7c802ac30 con 0x7fe7e006e0c0 2024-04-25T19:15:59.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.626+0000 7fe7ddd9b640 1 -- 172.21.15.76:0/335777142 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d80019f0 con 0x7fe7e006e0c0 2024-04-25T19:15:59.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.627+0000 7fe7e4946640 1 -- 172.21.15.76:0/335777142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7e006e0c0 msgr2=0x7fe7e006e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:59.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.627+0000 7fe7e4946640 1 --2- 172.21.15.76:0/335777142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7e006e0c0 0x7fe7e006e4d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe7c80097d0 tx=0x7fe7c802a1a0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.627+0000 7fe7e4946640 1 -- 172.21.15.76:0/335777142 shutdown_connections 2024-04-25T19:15:59.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.627+0000 7fe7e4946640 1 --2- 172.21.15.76:0/335777142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7e006e0c0 0x7fe7e006e4d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.627+0000 7fe7e4946640 1 -- 172.21.15.76:0/335777142 >> 172.21.15.76:0/335777142 conn(0x7fe7e0070b00 msgr2=0x7fe7e0070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:59.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.627+0000 7fe7e4946640 1 -- 172.21.15.76:0/335777142 shutdown_connections 2024-04-25T19:15:59.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.627+0000 7fe7e4946640 1 -- 172.21.15.76:0/335777142 wait complete. 2024-04-25T19:15:59.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.629+0000 7fe7e4946640 1 Processor -- start 2024-04-25T19:15:59.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.629+0000 7fe7e4946640 1 -- start start 2024-04-25T19:15:59.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.630+0000 7fe7e4946640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7e006e0c0 0x7fe7e00b4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:59.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.630+0000 7fe7e4946640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7e015cd10 0x7fe7e01503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:59.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.630+0000 7fe7e4946640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7e01609c0 0x7fe7e0150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:59.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.630+0000 7fe7e4946640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e00c5390 con 0x7fe7e006e0c0 2024-04-25T19:15:59.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.630+0000 7fe7e4946640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e00c5210 con 0x7fe7e01609c0 2024-04-25T19:15:59.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.630+0000 7fe7e4946640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fe7e00731e0 con 0x7fe7e015cd10 2024-04-25T19:15:59.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.630+0000 7fe7de59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7e006e0c0 0x7fe7e00b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:59.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.630+0000 7fe7de59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7e006e0c0 0x7fe7e00b4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43434/0 (socket says 172.21.15.76:43434) 2024-04-25T19:15:59.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.630+0000 7fe7de59c640 1 -- 172.21.15.76:0/2151394828 learned_addr learned my addr 172.21.15.76:0/2151394828 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:59.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.630+0000 7fe7ded9d640 1 --2- 172.21.15.76:0/2151394828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7e01609c0 0x7fe7e0150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:59.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.630+0000 7fe7d7fff640 1 --2- 172.21.15.76:0/2151394828 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7e015cd10 0x7fe7e01503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:59.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.630+0000 7fe7de59c640 1 -- 172.21.15.76:0/2151394828 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7e015cd10 msgr2=0x7fe7e01503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:59.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.630+0000 7fe7de59c640 1 --2- 172.21.15.76:0/2151394828 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7e015cd10 0x7fe7e01503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.630+0000 7fe7de59c640 1 -- 172.21.15.76:0/2151394828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7e01609c0 msgr2=0x7fe7e0150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:59.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.630+0000 7fe7de59c640 1 --2- 172.21.15.76:0/2151394828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7e01609c0 0x7fe7e0150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.630+0000 7fe7de59c640 1 -- 172.21.15.76:0/2151394828 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e00b4760 con 0x7fe7e006e0c0 2024-04-25T19:15:59.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.630+0000 7fe7de59c640 1 --2- 172.21.15.76:0/2151394828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7e006e0c0 0x7fe7e00b4220 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000c460 tx=0x7fe7d000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:59.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.631+0000 7fe7d6ffd640 1 -- 172.21.15.76:0/2151394828 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d00052d0 con 0x7fe7e006e0c0 2024-04-25T19:15:59.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.631+0000 7fe7d6ffd640 1 -- 172.21.15.76:0/2151394828 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7d0005490 con 0x7fe7e006e0c0 2024-04-25T19:15:59.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.631+0000 7fe7e4946640 1 -- 172.21.15.76:0/2151394828 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e00b4f30 con 0x7fe7e006e0c0 2024-04-25T19:15:59.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.631+0000 7fe7d6ffd640 1 -- 172.21.15.76:0/2151394828 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0012870 con 0x7fe7e006e0c0 2024-04-25T19:15:59.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.631+0000 7fe7e4946640 1 -- 172.21.15.76:0/2151394828 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e00b3a20 con 0x7fe7e006e0c0 2024-04-25T19:15:59.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.632+0000 7fe7e4946640 1 -- 172.21.15.76:0/2151394828 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7e006ef80 con 0x7fe7e006e0c0 2024-04-25T19:15:59.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.632+0000 7fe7d6ffd640 1 -- 172.21.15.76:0/2151394828 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe7d0012a10 con 0x7fe7e006e0c0 2024-04-25T19:15:59.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.633+0000 7fe7d6ffd640 1 --2- 172.21.15.76:0/2151394828 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe7c00744e0 0x7fe7c00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:59.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.633+0000 7fe7d6ffd640 1 -- 172.21.15.76:0/2151394828 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe7d0094d00 con 0x7fe7e006e0c0 2024-04-25T19:15:59.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.633+0000 7fe7d7fff640 1 --2- 172.21.15.76:0/2151394828 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe7c00744e0 0x7fe7c00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:59.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.633+0000 7fe7d7fff640 1 --2- 172.21.15.76:0/2151394828 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe7c00744e0 0x7fe7c00769f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8034000 tx=0x7fe7c802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:59.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.637+0000 7fe7d6ffd640 1 -- 172.21.15.76:0/2151394828 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe7e006ef80 con 0x7fe7e006e0c0 2024-04-25T19:15:59.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.814+0000 7fe7e4946640 1 -- 172.21.15.76:0/2151394828 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe7e006a390 con 0x7fe7e006e0c0 2024-04-25T19:15:59.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.814+0000 7fe7d6ffd640 1 -- 172.21.15.76:0/2151394828 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fe7d005ee50 con 0x7fe7e006e0c0 2024-04-25T19:15:59.816 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-04-25T19:15:59.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.816+0000 7fe7e4946640 1 -- 172.21.15.76:0/2151394828 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe7c00744e0 msgr2=0x7fe7c00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:59.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.816+0000 7fe7e4946640 1 --2- 172.21.15.76:0/2151394828 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe7c00744e0 0x7fe7c00769f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8034000 tx=0x7fe7c802e000 comp rx=0 tx=0).stop 2024-04-25T19:15:59.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.816+0000 7fe7e4946640 1 -- 172.21.15.76:0/2151394828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7e006e0c0 msgr2=0x7fe7e00b4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:59.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.816+0000 7fe7e4946640 1 --2- 172.21.15.76:0/2151394828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7e006e0c0 0x7fe7e00b4220 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000c460 tx=0x7fe7d000c920 comp rx=0 tx=0).stop 2024-04-25T19:15:59.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.816+0000 7fe7e4946640 1 -- 172.21.15.76:0/2151394828 shutdown_connections 2024-04-25T19:15:59.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.816+0000 7fe7e4946640 1 --2- 172.21.15.76:0/2151394828 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe7c00744e0 0x7fe7c00769f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.816+0000 7fe7e4946640 1 --2- 172.21.15.76:0/2151394828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7e01609c0 0x7fe7e0150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.816+0000 7fe7e4946640 1 --2- 172.21.15.76:0/2151394828 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe7e015cd10 0x7fe7e01503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.816+0000 7fe7e4946640 1 --2- 172.21.15.76:0/2151394828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7e006e0c0 0x7fe7e00b4220 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.816+0000 7fe7e4946640 1 -- 172.21.15.76:0/2151394828 >> 172.21.15.76:0/2151394828 conn(0x7fe7e0070b00 msgr2=0x7fe7e015b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:59.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.816+0000 7fe7e4946640 1 -- 172.21.15.76:0/2151394828 shutdown_connections 2024-04-25T19:15:59.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.816+0000 7fe7e4946640 1 -- 172.21.15.76:0/2151394828 wait complete. 2024-04-25T19:15:59.829 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T19:15:59.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.937+0000 7fb82b5d5640 1 Processor -- start 2024-04-25T19:15:59.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.937+0000 7fb82b5d5640 1 -- start start 2024-04-25T19:15:59.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.938+0000 7fb82b5d5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb8240cab60 0x7fb82415af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:59.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.938+0000 7fb82b5d5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8240c56a0 con 0x7fb824158060 2024-04-25T19:15:59.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.938+0000 7fb82b5d5640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8240c5960 con 0x7fb8240ca750 2024-04-25T19:15:59.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.938+0000 7fb82b5d5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fb8240c5c40 con 0x7fb8240cab60 2024-04-25T19:15:59.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.938+0000 7fb829371640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb824158060 0x7fb8240c4de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:32998/0 (socket says 172.21.15.76:32998) 2024-04-25T19:15:59.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.938+0000 7fb829371640 1 -- 172.21.15.76:0/3825524244 learned_addr learned my addr 172.21.15.76:0/3825524244 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:15:59.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.939+0000 7fb81bfff640 1 -- 172.21.15.76:0/3825524244 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2084411933 0 0) 0x7fb8240c56a0 con 0x7fb824158060 2024-04-25T19:15:59.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.939+0000 7fb81bfff640 1 -- 172.21.15.76:0/3825524244 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb810003520 con 0x7fb824158060 2024-04-25T19:15:59.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.939+0000 7fb829b72640 1 --2- 172.21.15.76:0/3825524244 >> v2:172.21.15.105:3300/0 conn(0x7fb8240cab60 0x7fb82415af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:59.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.939+0000 7fb81bfff640 1 -- 172.21.15.76:0/3825524244 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2005806991 0 0) 0x7fb810003520 con 0x7fb824158060 2024-04-25T19:15:59.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.939+0000 7fb81bfff640 1 -- 172.21.15.76:0/3825524244 >> v2:172.21.15.105:3300/0 conn(0x7fb8240cab60 msgr2=0x7fb82415af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:59.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.939+0000 7fb81bfff640 1 --2- 172.21.15.76:0/3825524244 >> v2:172.21.15.105:3300/0 conn(0x7fb8240cab60 0x7fb82415af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.939+0000 7fb81bfff640 1 -- 172.21.15.76:0/3825524244 >> v1:172.21.15.105:6789/0 conn(0x7fb8240ca750 legacy=0x7fb8240cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:59.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.939+0000 7fb81bfff640 1 -- 172.21.15.76:0/3825524244 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8240c5f20 con 0x7fb824158060 2024-04-25T19:15:59.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.939+0000 7fb81bfff640 1 -- 172.21.15.76:0/3825524244 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7fb820001690 con 0x7fb824158060 2024-04-25T19:15:59.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.939+0000 7fb81bfff640 1 -- 172.21.15.76:0/3825524244 >> v1:172.21.15.76:6789/0 conn(0x7fb824158060 legacy=0x7fb8240c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:59.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.939+0000 7fb81bfff640 1 --2- 172.21.15.76:0/3825524244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8100039b0 0x7fb810003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:59.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.939+0000 7fb81bfff640 1 --2- 172.21.15.76:0/3825524244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb810004800 0x7fb810004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:59.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.939+0000 7fb81bfff640 1 --2- 172.21.15.76:0/3825524244 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8100051c0 0x7fb81000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:59.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.939+0000 7fb81bfff640 1 -- 172.21.15.76:0/3825524244 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb8240c5c40 con 0x7fb8100039b0 2024-04-25T19:15:59.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.939+0000 7fb81bfff640 1 -- 172.21.15.76:0/3825524244 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb81000c150 con 0x7fb810004800 2024-04-25T19:15:59.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.939+0000 7fb81bfff640 1 -- 172.21.15.76:0/3825524244 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb81000c350 con 0x7fb8100051c0 2024-04-25T19:15:59.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.939+0000 7fb829371640 1 --2- 172.21.15.76:0/3825524244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8100039b0 0x7fb810003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:59.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.939+0000 7fb828b70640 1 --2- 172.21.15.76:0/3825524244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb810004800 0x7fb810004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:59.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.939+0000 7fb829b72640 1 --2- 172.21.15.76:0/3825524244 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8100051c0 0x7fb81000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:59.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.939+0000 7fb829371640 1 -- 172.21.15.76:0/3825524244 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8100051c0 msgr2=0x7fb81000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:59.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.939+0000 7fb829371640 1 --2- 172.21.15.76:0/3825524244 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8100051c0 0x7fb81000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.939+0000 7fb829371640 1 -- 172.21.15.76:0/3825524244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb810004800 msgr2=0x7fb810004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:15:59.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.939+0000 7fb829371640 1 --2- 172.21.15.76:0/3825524244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb810004800 0x7fb810004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.939+0000 7fb829371640 1 -- 172.21.15.76:0/3825524244 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb810003520 con 0x7fb8100039b0 2024-04-25T19:15:59.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.939+0000 7fb829b72640 1 -- 172.21.15.76:0/3825524244 reap_dead start 2024-04-25T19:15:59.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.940+0000 7fb829371640 1 --2- 172.21.15.76:0/3825524244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8100039b0 0x7fb810003e20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb82000c210 tx=0x7fb810005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d03a133de8ae681 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:59.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.940+0000 7fb81bfff640 1 -- 172.21.15.76:0/3825524244 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb820007d30 con 0x7fb8100039b0 2024-04-25T19:15:59.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.940+0000 7fb81bfff640 1 -- 172.21.15.76:0/3825524244 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb820005af0 con 0x7fb8100039b0 2024-04-25T19:15:59.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.940+0000 7fb81bfff640 1 -- 172.21.15.76:0/3825524244 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb820007070 con 0x7fb8100039b0 2024-04-25T19:15:59.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.941+0000 7fb82b5d5640 1 -- 172.21.15.76:0/3825524244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8100039b0 msgr2=0x7fb810003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:59.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.941+0000 7fb82b5d5640 1 --2- 172.21.15.76:0/3825524244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8100039b0 0x7fb810003e20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb82000c210 tx=0x7fb810005830 comp rx=0 tx=0).stop 2024-04-25T19:15:59.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.941+0000 7fb82b5d5640 1 -- 172.21.15.76:0/3825524244 shutdown_connections 2024-04-25T19:15:59.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.941+0000 7fb82b5d5640 1 --2- 172.21.15.76:0/3825524244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8100039b0 0x7fb810003e20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.941+0000 7fb82b5d5640 1 -- 172.21.15.76:0/3825524244 >> 172.21.15.76:0/3825524244 conn(0x7fb824068b10 msgr2=0x7fb824072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:15:59.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.941+0000 7fb82b5d5640 1 -- 172.21.15.76:0/3825524244 shutdown_connections 2024-04-25T19:15:59.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.941+0000 7fb82b5d5640 1 -- 172.21.15.76:0/3825524244 wait complete. 2024-04-25T19:15:59.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.943+0000 7fb82b5d5640 1 Processor -- start 2024-04-25T19:15:59.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.943+0000 7fb82b5d5640 1 -- start start 2024-04-25T19:15:59.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.944+0000 7fb82b5d5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8100039b0 0x7fb8240ce890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:59.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.944+0000 7fb82b5d5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8241f0890 0x7fb824151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:59.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.944+0000 7fb82b5d5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb82414e7a0 0x7fb82414ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:59.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.944+0000 7fb82b5d5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb8240c4de0 con 0x7fb8100039b0 2024-04-25T19:15:59.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.944+0000 7fb82b5d5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb8240c6200 con 0x7fb8241f0890 2024-04-25T19:15:59.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.944+0000 7fb82b5d5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb8240c4f60 con 0x7fb82414e7a0 2024-04-25T19:15:59.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.944+0000 7fb829371640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8100039b0 0x7fb8240ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:59.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.944+0000 7fb829371640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8100039b0 0x7fb8240ce890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43450/0 (socket says 172.21.15.76:43450) 2024-04-25T19:15:59.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.944+0000 7fb829371640 1 -- 172.21.15.76:0/4096729502 learned_addr learned my addr 172.21.15.76:0/4096729502 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:15:59.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.944+0000 7fb829b72640 1 --2- 172.21.15.76:0/4096729502 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb82414e7a0 0x7fb82414ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:59.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.944+0000 7fb828b70640 1 --2- 172.21.15.76:0/4096729502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8241f0890 0x7fb824151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:59.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.944+0000 7fb829371640 1 -- 172.21.15.76:0/4096729502 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb82414e7a0 msgr2=0x7fb82414ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:59.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.944+0000 7fb829371640 1 --2- 172.21.15.76:0/4096729502 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb82414e7a0 0x7fb82414ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.944+0000 7fb829371640 1 -- 172.21.15.76:0/4096729502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8241f0890 msgr2=0x7fb824151220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:15:59.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.944+0000 7fb829371640 1 --2- 172.21.15.76:0/4096729502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8241f0890 0x7fb824151220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:15:59.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.944+0000 7fb829371640 1 -- 172.21.15.76:0/4096729502 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8241527a0 con 0x7fb8100039b0 2024-04-25T19:15:59.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.944+0000 7fb828b70640 1 --2- 172.21.15.76:0/4096729502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8241f0890 0x7fb824151220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:15:59.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.945+0000 7fb829371640 1 --2- 172.21.15.76:0/4096729502 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8100039b0 0x7fb8240ce890 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb820001990 tx=0x7fb8240c7810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:59.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.945+0000 7fb819ffb640 1 -- 172.21.15.76:0/4096729502 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb820040030 con 0x7fb8100039b0 2024-04-25T19:15:59.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.945+0000 7fb819ffb640 1 -- 172.21.15.76:0/4096729502 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb820001490 con 0x7fb8100039b0 2024-04-25T19:15:59.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.945+0000 7fb82b5d5640 1 -- 172.21.15.76:0/4096729502 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8241f3e30 con 0x7fb8100039b0 2024-04-25T19:15:59.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.945+0000 7fb819ffb640 1 -- 172.21.15.76:0/4096729502 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb820037040 con 0x7fb8100039b0 2024-04-25T19:15:59.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.945+0000 7fb82b5d5640 1 -- 172.21.15.76:0/4096729502 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8241f43f0 con 0x7fb8100039b0 2024-04-25T19:15:59.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.946+0000 7fb82b5d5640 1 -- 172.21.15.76:0/4096729502 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8240c6440 con 0x7fb8100039b0 2024-04-25T19:15:59.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.946+0000 7fb819ffb640 1 -- 172.21.15.76:0/4096729502 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb820043070 con 0x7fb8100039b0 2024-04-25T19:15:59.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.947+0000 7fb819ffb640 1 --2- 172.21.15.76:0/4096729502 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb8080744b0 0x7fb8080769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:15:59.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.947+0000 7fb819ffb640 1 -- 172.21.15.76:0/4096729502 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb8200b4be0 con 0x7fb8100039b0 2024-04-25T19:15:59.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.947+0000 7fb828b70640 1 --2- 172.21.15.76:0/4096729502 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb8080744b0 0x7fb8080769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:15:59.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.947+0000 7fb828b70640 1 --2- 172.21.15.76:0/4096729502 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb8080744b0 0x7fb8080769c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb8241f1870 tx=0x7fb814009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:15:59.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:15:59.951+0000 7fb819ffb640 1 -- 172.21.15.76:0/4096729502 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb8240c6440 con 0x7fb8100039b0 2024-04-25T19:16:00.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.127+0000 7fb82b5d5640 1 -- 172.21.15.76:0/4096729502 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb82406a390 con 0x7fb8100039b0 2024-04-25T19:16:00.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.127+0000 7fb819ffb640 1 -- 172.21.15.76:0/4096729502 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fb82004a070 con 0x7fb8100039b0 2024-04-25T19:16:00.129 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-04-25T19:16:00.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.129+0000 7fb82b5d5640 1 -- 172.21.15.76:0/4096729502 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb8080744b0 msgr2=0x7fb8080769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:00.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.129+0000 7fb82b5d5640 1 --2- 172.21.15.76:0/4096729502 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb8080744b0 0x7fb8080769c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb8241f1870 tx=0x7fb814009040 comp rx=0 tx=0).stop 2024-04-25T19:16:00.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.129+0000 7fb82b5d5640 1 -- 172.21.15.76:0/4096729502 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8100039b0 msgr2=0x7fb8240ce890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:00.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.129+0000 7fb82b5d5640 1 --2- 172.21.15.76:0/4096729502 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8100039b0 0x7fb8240ce890 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb820001990 tx=0x7fb8240c7810 comp rx=0 tx=0).stop 2024-04-25T19:16:00.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.129+0000 7fb82b5d5640 1 -- 172.21.15.76:0/4096729502 shutdown_connections 2024-04-25T19:16:00.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.129+0000 7fb82b5d5640 1 --2- 172.21.15.76:0/4096729502 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb8080744b0 0x7fb8080769c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:00.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.129+0000 7fb82b5d5640 1 --2- 172.21.15.76:0/4096729502 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb82414e7a0 0x7fb82414ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:00.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.129+0000 7fb82b5d5640 1 --2- 172.21.15.76:0/4096729502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8241f0890 0x7fb824151220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:00.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.129+0000 7fb82b5d5640 1 --2- 172.21.15.76:0/4096729502 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8100039b0 0x7fb8240ce890 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:00.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.129+0000 7fb82b5d5640 1 -- 172.21.15.76:0/4096729502 >> 172.21.15.76:0/4096729502 conn(0x7fb824068b10 msgr2=0x7fb8241539b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:00.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.129+0000 7fb82b5d5640 1 -- 172.21.15.76:0/4096729502 shutdown_connections 2024-04-25T19:16:00.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.129+0000 7fb82b5d5640 1 -- 172.21.15.76:0/4096729502 wait complete. 2024-04-25T19:16:00.142 DEBUG:tasks.mon_thrash:Yielding 2024-04-25T19:16:00.142 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T19:16:00.160 INFO:tasks.workunit:Pulling workunits from ref 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T19:16:00.160 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T19:16:00.160 INFO:tasks.workunit:timeout=3h 2024-04-25T19:16:00.161 INFO:tasks.workunit:cleanup=True 2024-04-25T19:16:00.161 DEBUG:teuthology.orchestra.run.smithi076:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T19:16:00.161 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:16:00.177 INFO:teuthology.orchestra.run.smithi076.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T19:16:00.177 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-25T19:16:00.177 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 2bh/43d Inode: 1 Links: 2 2024-04-25T19:16:00.177 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T19:16:00.177 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-25T19:16:00.177 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-25T19:16:00.177 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-04-25 19:15:24.402852900 +0000 2024-04-25T19:16:00.177 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-04-25 19:15:56.865039639 +0000 2024-04-25T19:16:00.177 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: - 2024-04-25T19:16:00.177 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T19:16:00.178 DEBUG:teuthology.orchestra.run.smithi076:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T19:16:00.249 DEBUG:teuthology.orchestra.run.smithi076:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T19:16:00.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.271+0000 7f9c6eb14640 1 Processor -- start 2024-04-25T19:16:00.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.271+0000 7f9c6eb14640 1 -- start start 2024-04-25T19:16:00.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.272+0000 7f9c6eb14640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9c680c95c0 0x7f9c68156c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:00.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.272+0000 7f9c6eb14640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c6815cae0 0x7f9c680cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:00.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.272+0000 7f9c6eb14640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c680c7670 con 0x7f9c68074650 2024-04-25T19:16:00.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.272+0000 7f9c6eb14640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f9c6815d040 con 0x7f9c680c95c0 2024-04-25T19:16:00.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.272+0000 7f9c6eb14640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f9c6815d320 con 0x7f9c6815cae0 2024-04-25T19:16:00.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.273+0000 7f9c6c8b0640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9c680c95c0 0x7f9c68156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:00.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.273+0000 7f9c67fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c6815cae0 0x7f9c680cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:00.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.273+0000 7f9c6c8b0640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9c680c95c0 0x7f9c68156c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:52716/0 (socket says 172.21.15.76:52716) 2024-04-25T19:16:00.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.273+0000 7f9c6d0b1640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9c68074650 0x7f9c680c7e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.76:36194/0 (socket says 172.21.15.76:36194) 2024-04-25T19:16:00.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.273+0000 7f9c6c8b0640 1 -- 172.21.15.76:0/2901494493 learned_addr learned my addr 172.21.15.76:0/2901494493 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:00.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.273+0000 7f9c6c8b0640 1 -- 172.21.15.76:0/2901494493 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c6815cae0 msgr2=0x7f9c680cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:00.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.273+0000 7f9c6c8b0640 1 --2- 172.21.15.76:0/2901494493 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c6815cae0 0x7f9c680cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:00.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.273+0000 7f9c6c8b0640 1 -- 172.21.15.76:0/2901494493 >> v1:172.21.15.105:6789/0 conn(0x7f9c68074650 legacy=0x7f9c680c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:00.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.273+0000 7f9c6c8b0640 1 -- 172.21.15.76:0/2901494493 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6815d600 con 0x7f9c680c95c0 2024-04-25T19:16:00.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.274+0000 7f9c6c8b0640 1 --2- 172.21.15.76:0/2901494493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c680c95c0 0x7f9c68156c10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009870 tx=0x7f9c5c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ee7a334b77a1025 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:00.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.274+0000 7f9c677fe640 1 -- 172.21.15.76:0/2901494493 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c037070 con 0x7f9c680c95c0 2024-04-25T19:16:00.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.274+0000 7f9c677fe640 1 -- 172.21.15.76:0/2901494493 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c5c032440 con 0x7f9c680c95c0 2024-04-25T19:16:00.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.274+0000 7f9c677fe640 1 -- 172.21.15.76:0/2901494493 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c032810 con 0x7f9c680c95c0 2024-04-25T19:16:00.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.274+0000 7f9c6eb14640 1 -- 172.21.15.76:0/2901494493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c680c95c0 msgr2=0x7f9c68156c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:00.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.274+0000 7f9c6eb14640 1 --2- 172.21.15.76:0/2901494493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c680c95c0 0x7f9c68156c10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009870 tx=0x7f9c5c02a270 comp rx=0 tx=0).stop 2024-04-25T19:16:00.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.275+0000 7f9c6eb14640 1 -- 172.21.15.76:0/2901494493 shutdown_connections 2024-04-25T19:16:00.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.275+0000 7f9c6eb14640 1 --2- 172.21.15.76:0/2901494493 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c6815cae0 0x7f9c680cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:00.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.275+0000 7f9c6eb14640 1 --2- 172.21.15.76:0/2901494493 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c680c95c0 0x7f9c68156c10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:00.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.275+0000 7f9c6eb14640 1 -- 172.21.15.76:0/2901494493 >> 172.21.15.76:0/2901494493 conn(0x7f9c68071a30 msgr2=0x7f9c68071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:00.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.275+0000 7f9c6eb14640 1 -- 172.21.15.76:0/2901494493 shutdown_connections 2024-04-25T19:16:00.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.275+0000 7f9c6eb14640 1 -- 172.21.15.76:0/2901494493 wait complete. 2024-04-25T19:16:00.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.276+0000 7f9c6eb14640 1 Processor -- start 2024-04-25T19:16:00.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.276+0000 7f9c6eb14640 1 -- start start 2024-04-25T19:16:00.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.278+0000 7f9c6eb14640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c68074650 0x7f9c681f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:00.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.278+0000 7f9c6eb14640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c680c95c0 0x7f9c681544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:00.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.278+0000 7f9c6eb14640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c6815cae0 0x7f9c6814f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:00.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.278+0000 7f9c6eb14640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9c680c70a0 con 0x7f9c6815cae0 2024-04-25T19:16:00.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.278+0000 7f9c6eb14640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9c680c78e0 con 0x7f9c680c95c0 2024-04-25T19:16:00.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.278+0000 7f9c6eb14640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f9c680c7220 con 0x7f9c68074650 2024-04-25T19:16:00.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.278+0000 7f9c6d0b1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c6815cae0 0x7f9c6814f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:00.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.278+0000 7f9c6d0b1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c6815cae0 0x7f9c6814f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43464/0 (socket says 172.21.15.76:43464) 2024-04-25T19:16:00.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.278+0000 7f9c6d0b1640 1 -- 172.21.15.76:0/2069237114 learned_addr learned my addr 172.21.15.76:0/2069237114 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:00.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.278+0000 7f9c6c8b0640 1 --2- 172.21.15.76:0/2069237114 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c68074650 0x7f9c681f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:00.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.278+0000 7f9c67fff640 1 --2- 172.21.15.76:0/2069237114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c680c95c0 0x7f9c681544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:00.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.278+0000 7f9c6d0b1640 1 -- 172.21.15.76:0/2069237114 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c68074650 msgr2=0x7f9c681f6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:00.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.278+0000 7f9c6d0b1640 1 --2- 172.21.15.76:0/2069237114 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c68074650 0x7f9c681f6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:00.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.278+0000 7f9c6d0b1640 1 -- 172.21.15.76:0/2069237114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c680c95c0 msgr2=0x7f9c681544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:00.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.278+0000 7f9c6d0b1640 1 --2- 172.21.15.76:0/2069237114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c680c95c0 0x7f9c681544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:00.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.278+0000 7f9c6d0b1640 1 -- 172.21.15.76:0/2069237114 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6814fb00 con 0x7f9c6815cae0 2024-04-25T19:16:00.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.278+0000 7f9c6d0b1640 1 --2- 172.21.15.76:0/2069237114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c6815cae0 0x7f9c6814f5c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000c730 tx=0x7f9c6000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:00.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.279+0000 7f9c657fa640 1 -- 172.21.15.76:0/2069237114 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60018070 con 0x7f9c6815cae0 2024-04-25T19:16:00.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.279+0000 7f9c657fa640 1 -- 172.21.15.76:0/2069237114 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c60004d60 con 0x7f9c6815cae0 2024-04-25T19:16:00.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.279+0000 7f9c6eb14640 1 -- 172.21.15.76:0/2069237114 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c681f2f70 con 0x7f9c6815cae0 2024-04-25T19:16:00.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.279+0000 7f9c657fa640 1 -- 172.21.15.76:0/2069237114 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6000a800 con 0x7f9c6815cae0 2024-04-25T19:16:00.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.279+0000 7f9c6eb14640 1 -- 172.21.15.76:0/2069237114 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c681f3530 con 0x7f9c6815cae0 2024-04-25T19:16:00.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.280+0000 7f9c6eb14640 1 -- 172.21.15.76:0/2069237114 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c6806a390 con 0x7f9c6815cae0 2024-04-25T19:16:00.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.281+0000 7f9c657fa640 1 -- 172.21.15.76:0/2069237114 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9c6001b070 con 0x7f9c6815cae0 2024-04-25T19:16:00.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.281+0000 7f9c657fa640 1 --2- 172.21.15.76:0/2069237114 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9c4c0744b0 0x7f9c4c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:00.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.281+0000 7f9c657fa640 1 -- 172.21.15.76:0/2069237114 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9c60094aa0 con 0x7f9c6815cae0 2024-04-25T19:16:00.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.281+0000 7f9c6c8b0640 1 --2- 172.21.15.76:0/2069237114 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9c4c0744b0 0x7f9c4c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:00.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.282+0000 7f9c6c8b0640 1 --2- 172.21.15.76:0/2069237114 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9c4c0744b0 0x7f9c4c0769c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c034000 tx=0x7f9c5c02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:00.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.286+0000 7f9c657fa640 1 -- 172.21.15.76:0/2069237114 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c6005ecc0 con 0x7f9c6815cae0 2024-04-25T19:16:00.288 INFO:tasks.workunit.client.0.smithi076.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T19:16:00.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.509+0000 7f9c6eb14640 1 -- 172.21.15.76:0/2069237114 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c68074430 con 0x7f9c6815cae0 2024-04-25T19:16:00.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.509+0000 7f9c657fa640 1 -- 172.21.15.76:0/2069237114 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6915 (secure 0 0 0) 0x7f9c60062fc0 con 0x7f9c6815cae0 2024-04-25T19:16:00.511 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:16:00.511 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/3003966182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3003966182},{"type":"v1","addr":"172.21.15.76:6836","nonce":3003966182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T19:15:18.769449+0000","modified":"2024-04-25T19:15:48.500543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4284,"mds_1":4278,"mds_2":4241,"mds_3":4238,"mds_4":4235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4235":{"gid":4235,"name":"b","rank":4,"incarnation":15,"state":"up:active","state_seq":10,"addr":"172.21.15.105:6836/1809877606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1809877606},{"type":"v1","addr":"172.21.15.105:6836","nonce":1809877606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4238":{"gid":4238,"name":"f","rank":3,"incarnation":13,"state":"up:active","state_seq":9,"addr":"172.21.15.105:6838/2724142282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2724142282},{"type":"v1","addr":"172.21.15.105:6838","nonce":2724142282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4241":{"gid":4241,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.105:6839/1798232582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6837","nonce":1798232582},{"type":"v1","addr":"172.21.15.105:6839","nonce":1798232582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.76:6833/2101494685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2101494685},{"type":"v1","addr":"172.21.15.76:6833","nonce":2101494685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4284":{"gid":4284,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6837/4226277412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":4226277412},{"type":"v1","addr":"172.21.15.76:6837","nonce":4226277412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4235,4238,4241,4278,4284]},"id":1}]} 2024-04-25T19:16:00.511 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 16 2024-04-25T19:16:00.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.511+0000 7f9c6eb14640 1 -- 172.21.15.76:0/2069237114 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9c4c0744b0 msgr2=0x7f9c4c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:00.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.511+0000 7f9c6eb14640 1 --2- 172.21.15.76:0/2069237114 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9c4c0744b0 0x7f9c4c0769c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c034000 tx=0x7f9c5c02e000 comp rx=0 tx=0).stop 2024-04-25T19:16:00.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.511+0000 7f9c6eb14640 1 -- 172.21.15.76:0/2069237114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c6815cae0 msgr2=0x7f9c6814f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:00.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.511+0000 7f9c6eb14640 1 --2- 172.21.15.76:0/2069237114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c6815cae0 0x7f9c6814f5c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000c730 tx=0x7f9c6000cbf0 comp rx=0 tx=0).stop 2024-04-25T19:16:00.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.511+0000 7f9c6eb14640 1 -- 172.21.15.76:0/2069237114 shutdown_connections 2024-04-25T19:16:00.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.511+0000 7f9c6eb14640 1 --2- 172.21.15.76:0/2069237114 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9c4c0744b0 0x7f9c4c0769c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:00.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.511+0000 7f9c6eb14640 1 --2- 172.21.15.76:0/2069237114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c6815cae0 0x7f9c6814f5c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:00.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.511+0000 7f9c6eb14640 1 --2- 172.21.15.76:0/2069237114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c680c95c0 0x7f9c681544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:00.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.511+0000 7f9c6eb14640 1 --2- 172.21.15.76:0/2069237114 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c68074650 0x7f9c681f6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:00.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.511+0000 7f9c6eb14640 1 -- 172.21.15.76:0/2069237114 >> 172.21.15.76:0/2069237114 conn(0x7f9c68071a30 msgr2=0x7f9c6815a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:00.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.511+0000 7f9c6eb14640 1 -- 172.21.15.76:0/2069237114 shutdown_connections 2024-04-25T19:16:00.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.511+0000 7f9c6eb14640 1 -- 172.21.15.76:0/2069237114 wait complete. 2024-04-25T19:16:00.524 INFO:tasks.mon_thrash.mon_thrasher:start thrashing 2024-04-25T19:16:00.525 INFO:tasks.mon_thrash.mon_thrasher:seed: 1714072558, revive delay: 20.0, thrash delay: 10.0 thrash many: False, maintain quorum: True store thrash: False, probability: 50 freeze mon: prob 10.0 duration 15.0 2024-04-25T19:16:00.525 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T19:16:00.525 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:16:00.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.673+0000 7f3e467f3640 1 Processor -- start 2024-04-25T19:16:00.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.674+0000 7f3e467f3640 1 -- start start 2024-04-25T19:16:00.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.675+0000 7f3e467f3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3e400cea70 0x7f3e400cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:00.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.675+0000 7f3e467f3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3e4006e160 0x7f3e4006e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:00.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.675+0000 7f3e467f3640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e4015ce80 con 0x7f3e400d0810 2024-04-25T19:16:00.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.675+0000 7f3e467f3640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f3e4015d160 con 0x7f3e400cea70 2024-04-25T19:16:00.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.675+0000 7f3e467f3640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f3e4015d440 con 0x7f3e4006e160 2024-04-25T19:16:00.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.675+0000 7f3e3f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3e400cea70 0x7f3e400cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:00.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.675+0000 7f3e3f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3e400cea70 0x7f3e400cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43474/0 (socket says 172.21.15.76:43474) 2024-04-25T19:16:00.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.675+0000 7f3e3f7fe640 1 -- 172.21.15.76:0/2458489623 learned_addr learned my addr 172.21.15.76:0/2458489623 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:00.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.675+0000 7f3e44d90640 1 --2- 172.21.15.76:0/2458489623 >> v2:172.21.15.105:3300/0 conn(0x7f3e4006e160 0x7f3e4006e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:00.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.675+0000 7f3e3f7fe640 1 -- 172.21.15.76:0/2458489623 >> v2:172.21.15.105:3300/0 conn(0x7f3e4006e160 msgr2=0x7f3e4006e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:00.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.675+0000 7f3e3f7fe640 1 --2- 172.21.15.76:0/2458489623 >> v2:172.21.15.105:3300/0 conn(0x7f3e4006e160 0x7f3e4006e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:00.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.675+0000 7f3e3f7fe640 1 -- 172.21.15.76:0/2458489623 >> v1:172.21.15.105:6789/0 conn(0x7f3e400d0810 legacy=0x7f3e4006f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:00.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.675+0000 7f3e3f7fe640 1 -- 172.21.15.76:0/2458489623 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e4015d720 con 0x7f3e400cea70 2024-04-25T19:16:00.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.676+0000 7f3e3f7fe640 1 --2- 172.21.15.76:0/2458489623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e400cea70 0x7f3e400cee80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3e40090090 tx=0x7f3e3402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9133e0c50506286e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:00.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.676+0000 7f3e3effd640 1 -- 172.21.15.76:0/2458489623 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34037070 con 0x7f3e400cea70 2024-04-25T19:16:00.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.676+0000 7f3e3effd640 1 -- 172.21.15.76:0/2458489623 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e34032440 con 0x7f3e400cea70 2024-04-25T19:16:00.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.676+0000 7f3e3effd640 1 -- 172.21.15.76:0/2458489623 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34032810 con 0x7f3e400cea70 2024-04-25T19:16:00.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.677+0000 7f3e467f3640 1 -- 172.21.15.76:0/2458489623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e400cea70 msgr2=0x7f3e400cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:00.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.677+0000 7f3e467f3640 1 --2- 172.21.15.76:0/2458489623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e400cea70 0x7f3e400cee80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3e40090090 tx=0x7f3e3402a270 comp rx=0 tx=0).stop 2024-04-25T19:16:00.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.677+0000 7f3e467f3640 1 -- 172.21.15.76:0/2458489623 shutdown_connections 2024-04-25T19:16:00.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.677+0000 7f3e467f3640 1 --2- 172.21.15.76:0/2458489623 >> v2:172.21.15.105:3300/0 conn(0x7f3e4006e160 0x7f3e4006e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:00.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.677+0000 7f3e467f3640 1 --2- 172.21.15.76:0/2458489623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e400cea70 0x7f3e400cee80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:00.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.677+0000 7f3e467f3640 1 -- 172.21.15.76:0/2458489623 >> 172.21.15.76:0/2458489623 conn(0x7f3e400b3c00 msgr2=0x7f3e400b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:00.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.677+0000 7f3e467f3640 1 -- 172.21.15.76:0/2458489623 shutdown_connections 2024-04-25T19:16:00.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.677+0000 7f3e467f3640 1 -- 172.21.15.76:0/2458489623 wait complete. 2024-04-25T19:16:00.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.679+0000 7f3e467f3640 1 Processor -- start 2024-04-25T19:16:00.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.679+0000 7f3e467f3640 1 -- start start 2024-04-25T19:16:00.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.680+0000 7f3e467f3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e4006e160 0x7f3e401533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:00.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.680+0000 7f3e467f3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e400cea70 0x7f3e40150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:00.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.680+0000 7f3e467f3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e400d0810 0x7f3e40150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:00.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.680+0000 7f3e467f3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4006eab0 con 0x7f3e400d0810 2024-04-25T19:16:00.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.680+0000 7f3e467f3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4006dc40 con 0x7f3e400cea70 2024-04-25T19:16:00.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.680+0000 7f3e467f3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f3e4015cb70 con 0x7f3e4006e160 2024-04-25T19:16:00.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.680+0000 7f3e44d90640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e400d0810 0x7f3e40150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:00.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.680+0000 7f3e44d90640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e400d0810 0x7f3e40150ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43488/0 (socket says 172.21.15.76:43488) 2024-04-25T19:16:00.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.680+0000 7f3e44d90640 1 -- 172.21.15.76:0/2933421737 learned_addr learned my addr 172.21.15.76:0/2933421737 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:00.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.680+0000 7f3e3f7fe640 1 --2- 172.21.15.76:0/2933421737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e400cea70 0x7f3e40150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:00.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.680+0000 7f3e3ffff640 1 --2- 172.21.15.76:0/2933421737 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e4006e160 0x7f3e401533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:00.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.681+0000 7f3e44d90640 1 -- 172.21.15.76:0/2933421737 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e4006e160 msgr2=0x7f3e401533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:00.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.681+0000 7f3e44d90640 1 --2- 172.21.15.76:0/2933421737 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e4006e160 0x7f3e401533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:00.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.681+0000 7f3e44d90640 1 -- 172.21.15.76:0/2933421737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e400cea70 msgr2=0x7f3e40150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:00.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.681+0000 7f3e44d90640 1 --2- 172.21.15.76:0/2933421737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e400cea70 0x7f3e40150580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:00.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.681+0000 7f3e44d90640 1 -- 172.21.15.76:0/2933421737 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e401567b0 con 0x7f3e400d0810 2024-04-25T19:16:00.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.681+0000 7f3e44d90640 1 --2- 172.21.15.76:0/2933421737 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e400d0810 0x7f3e40150ac0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800efc0 tx=0x7f3e3800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:00.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.681+0000 7f3e3cff9640 1 -- 172.21.15.76:0/2933421737 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38018020 con 0x7f3e400d0810 2024-04-25T19:16:00.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.681+0000 7f3e3cff9640 1 -- 172.21.15.76:0/2933421737 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e38014480 con 0x7f3e400d0810 2024-04-25T19:16:00.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.681+0000 7f3e467f3640 1 -- 172.21.15.76:0/2933421737 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e401554f0 con 0x7f3e400d0810 2024-04-25T19:16:00.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.681+0000 7f3e3cff9640 1 -- 172.21.15.76:0/2933421737 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38014870 con 0x7f3e400d0810 2024-04-25T19:16:00.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.681+0000 7f3e467f3640 1 -- 172.21.15.76:0/2933421737 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e40155790 con 0x7f3e400d0810 2024-04-25T19:16:00.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.682+0000 7f3e467f3640 1 -- 172.21.15.76:0/2933421737 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e40151780 con 0x7f3e400d0810 2024-04-25T19:16:00.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.683+0000 7f3e3cff9640 1 -- 172.21.15.76:0/2933421737 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3e38005d10 con 0x7f3e400d0810 2024-04-25T19:16:00.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.683+0000 7f3e3cff9640 1 --2- 172.21.15.76:0/2933421737 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f3e2c0744e0 0x7f3e2c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:00.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.683+0000 7f3e3cff9640 1 -- 172.21.15.76:0/2933421737 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3e38095aa0 con 0x7f3e400d0810 2024-04-25T19:16:00.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.684+0000 7f3e3ffff640 1 --2- 172.21.15.76:0/2933421737 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f3e2c0744e0 0x7f3e2c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:00.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.684+0000 7f3e3ffff640 1 --2- 172.21.15.76:0/2933421737 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f3e2c0744e0 0x7f3e2c0769f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3e30007fd0 tx=0x7f3e30009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:00.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.688+0000 7f3e3cff9640 1 -- 172.21.15.76:0/2933421737 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3e38022070 con 0x7f3e400d0810 2024-04-25T19:16:00.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.934+0000 7f3e467f3640 1 -- 172.21.15.76:0/2933421737 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3e4006a390 con 0x7f3e400d0810 2024-04-25T19:16:00.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.935+0000 7f3e3cff9640 1 -- 172.21.15.76:0/2933421737 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7f3e38010040 con 0x7f3e400d0810 2024-04-25T19:16:00.937 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":59,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:16:00.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.937+0000 7f3e467f3640 1 -- 172.21.15.76:0/2933421737 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f3e2c0744e0 msgr2=0x7f3e2c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:00.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.937+0000 7f3e467f3640 1 --2- 172.21.15.76:0/2933421737 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f3e2c0744e0 0x7f3e2c0769f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3e30007fd0 tx=0x7f3e30009040 comp rx=0 tx=0).stop 2024-04-25T19:16:00.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.937+0000 7f3e467f3640 1 -- 172.21.15.76:0/2933421737 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e400d0810 msgr2=0x7f3e40150ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:00.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.937+0000 7f3e467f3640 1 --2- 172.21.15.76:0/2933421737 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e400d0810 0x7f3e40150ac0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800efc0 tx=0x7f3e3800c490 comp rx=0 tx=0).stop 2024-04-25T19:16:00.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.937+0000 7f3e467f3640 1 -- 172.21.15.76:0/2933421737 shutdown_connections 2024-04-25T19:16:00.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.937+0000 7f3e467f3640 1 --2- 172.21.15.76:0/2933421737 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f3e2c0744e0 0x7f3e2c0769f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:00.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.937+0000 7f3e467f3640 1 --2- 172.21.15.76:0/2933421737 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e400d0810 0x7f3e40150ac0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:00.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.937+0000 7f3e467f3640 1 --2- 172.21.15.76:0/2933421737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3e400cea70 0x7f3e40150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:00.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.937+0000 7f3e467f3640 1 --2- 172.21.15.76:0/2933421737 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3e4006e160 0x7f3e401533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:00.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.937+0000 7f3e467f3640 1 -- 172.21.15.76:0/2933421737 >> 172.21.15.76:0/2933421737 conn(0x7f3e400b3c00 msgr2=0x7f3e400d1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:00.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.937+0000 7f3e467f3640 1 -- 172.21.15.76:0/2933421737 shutdown_connections 2024-04-25T19:16:00.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:00.937+0000 7f3e467f3640 1 -- 172.21.15.76:0/2933421737 wait complete. 2024-04-25T19:16:00.950 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T19:16:00.950 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-25T19:16:00.950 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T19:16:00.950 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T19:16:01.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.056+0000 7ff27f630640 1 Processor -- start 2024-04-25T19:16:01.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.057+0000 7ff27f630640 1 -- start start 2024-04-25T19:16:01.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.058+0000 7ff27f630640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff27806e0c0 0x7ff27806e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.058+0000 7ff27f630640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff2781609c0 0x7ff278160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.058+0000 7ff27f630640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff27815d250 con 0x7ff27815cd10 2024-04-25T19:16:01.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.058+0000 7ff27f630640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff27815d510 con 0x7ff27806e0c0 2024-04-25T19:16:01.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.058+0000 7ff27f630640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7ff27815d7f0 con 0x7ff2781609c0 2024-04-25T19:16:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.058+0000 7ff27d3cc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff27815cd10 0x7ff278156c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33006/0 (socket says 172.21.15.76:33006) 2024-04-25T19:16:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.058+0000 7ff27d3cc640 1 -- 172.21.15.76:0/1784768328 learned_addr learned my addr 172.21.15.76:0/1784768328 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:16:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.058+0000 7ff27cbcb640 1 --2- 172.21.15.76:0/1784768328 >> v2:172.21.15.76:3300/0 conn(0x7ff27806e0c0 0x7ff27806e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.058+0000 7ff27dbcd640 1 --2- 172.21.15.76:0/1784768328 >> v2:172.21.15.105:3300/0 conn(0x7ff2781609c0 0x7ff278160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.058+0000 7ff26ffff640 1 -- 172.21.15.76:0/1784768328 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2831745200 0 0) 0x7ff27815d250 con 0x7ff27815cd10 2024-04-25T19:16:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.058+0000 7ff26ffff640 1 -- 172.21.15.76:0/1784768328 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff264003520 con 0x7ff27815cd10 2024-04-25T19:16:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.058+0000 7ff27cbcb640 1 -- 172.21.15.76:0/1784768328 >> v2:172.21.15.105:3300/0 conn(0x7ff2781609c0 msgr2=0x7ff278160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.058+0000 7ff27cbcb640 1 --2- 172.21.15.76:0/1784768328 >> v2:172.21.15.105:3300/0 conn(0x7ff2781609c0 0x7ff278160dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.058+0000 7ff27cbcb640 1 -- 172.21.15.76:0/1784768328 >> v1:172.21.15.76:6789/0 conn(0x7ff27815cd10 legacy=0x7ff278156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.058+0000 7ff27cbcb640 1 -- 172.21.15.76:0/1784768328 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27815dad0 con 0x7ff27806e0c0 2024-04-25T19:16:01.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.059+0000 7ff27cbcb640 1 --2- 172.21.15.76:0/1784768328 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff27806e0c0 0x7ff27806e4d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff2780ccb90 tx=0x7ff26802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d352ee06861ce3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:01.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.059+0000 7ff26ffff640 1 -- 172.21.15.76:0/1784768328 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268037070 con 0x7ff27806e0c0 2024-04-25T19:16:01.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.059+0000 7ff26ffff640 1 -- 172.21.15.76:0/1784768328 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff268032440 con 0x7ff27806e0c0 2024-04-25T19:16:01.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.059+0000 7ff26ffff640 1 -- 172.21.15.76:0/1784768328 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268032810 con 0x7ff27806e0c0 2024-04-25T19:16:01.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.060+0000 7ff27f630640 1 -- 172.21.15.76:0/1784768328 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff27806e0c0 msgr2=0x7ff27806e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.060+0000 7ff27f630640 1 --2- 172.21.15.76:0/1784768328 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff27806e0c0 0x7ff27806e4d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff2780ccb90 tx=0x7ff26802a1a0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.060+0000 7ff27f630640 1 -- 172.21.15.76:0/1784768328 shutdown_connections 2024-04-25T19:16:01.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.060+0000 7ff27f630640 1 --2- 172.21.15.76:0/1784768328 >> v2:172.21.15.105:3300/0 conn(0x7ff2781609c0 0x7ff278160dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.060+0000 7ff27f630640 1 --2- 172.21.15.76:0/1784768328 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff27806e0c0 0x7ff27806e4d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.060+0000 7ff27f630640 1 -- 172.21.15.76:0/1784768328 >> 172.21.15.76:0/1784768328 conn(0x7ff278070b00 msgr2=0x7ff278070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:01.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.060+0000 7ff27f630640 1 -- 172.21.15.76:0/1784768328 shutdown_connections 2024-04-25T19:16:01.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.060+0000 7ff27f630640 1 -- 172.21.15.76:0/1784768328 wait complete. 2024-04-25T19:16:01.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.062+0000 7ff27f630640 1 Processor -- start 2024-04-25T19:16:01.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.062+0000 7ff27f630640 1 -- start start 2024-04-25T19:16:01.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.063+0000 7ff27f630640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff27806e0c0 0x7ff27815e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.063+0000 7ff27f630640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff27815cd10 0x7ff2780d0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.063+0000 7ff27f630640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2781609c0 0x7ff27814c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.063+0000 7ff27f630640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff2780c5390 con 0x7ff27806e0c0 2024-04-25T19:16:01.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.063+0000 7ff27f630640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff2780c5210 con 0x7ff27815cd10 2024-04-25T19:16:01.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.063+0000 7ff27f630640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff278072ad0 con 0x7ff2781609c0 2024-04-25T19:16:01.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.063+0000 7ff27d3cc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff27806e0c0 0x7ff27815e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.063+0000 7ff27d3cc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff27806e0c0 0x7ff27815e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43500/0 (socket says 172.21.15.76:43500) 2024-04-25T19:16:01.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.063+0000 7ff27d3cc640 1 -- 172.21.15.76:0/2550890407 learned_addr learned my addr 172.21.15.76:0/2550890407 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:01.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.063+0000 7ff27cbcb640 1 --2- 172.21.15.76:0/2550890407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff27815cd10 0x7ff2780d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.063+0000 7ff27dbcd640 1 --2- 172.21.15.76:0/2550890407 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2781609c0 0x7ff27814c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.063+0000 7ff27d3cc640 1 -- 172.21.15.76:0/2550890407 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2781609c0 msgr2=0x7ff27814c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.063+0000 7ff27d3cc640 1 --2- 172.21.15.76:0/2550890407 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff2781609c0 0x7ff27814c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.063+0000 7ff27d3cc640 1 -- 172.21.15.76:0/2550890407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff27815cd10 msgr2=0x7ff2780d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.063+0000 7ff27d3cc640 1 --2- 172.21.15.76:0/2550890407 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff27815cd10 0x7ff2780d0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.063+0000 7ff27d3cc640 1 -- 172.21.15.76:0/2550890407 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27815ecf0 con 0x7ff27806e0c0 2024-04-25T19:16:01.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.064+0000 7ff27d3cc640 1 --2- 172.21.15.76:0/2550890407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff27806e0c0 0x7ff27815e7b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff27400cb00 tx=0x7ff27400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:01.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.064+0000 7ff26dffb640 1 -- 172.21.15.76:0/2550890407 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff274017070 con 0x7ff27806e0c0 2024-04-25T19:16:01.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.064+0000 7ff26dffb640 1 -- 172.21.15.76:0/2550890407 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff27400aca0 con 0x7ff27806e0c0 2024-04-25T19:16:01.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.064+0000 7ff27f630640 1 -- 172.21.15.76:0/2550890407 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2781f6bf0 con 0x7ff27806e0c0 2024-04-25T19:16:01.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.064+0000 7ff26dffb640 1 -- 172.21.15.76:0/2550890407 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff274012620 con 0x7ff27806e0c0 2024-04-25T19:16:01.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.064+0000 7ff27f630640 1 -- 172.21.15.76:0/2550890407 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2781f71b0 con 0x7ff27806e0c0 2024-04-25T19:16:01.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.065+0000 7ff27f630640 1 --2- 172.21.15.76:0/2550890407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff278158670 0x7ff27814cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.065+0000 7ff27f630640 1 -- 172.21.15.76:0/2550890407 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff27814d1e0 con 0x7ff278158670 2024-04-25T19:16:01.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.065+0000 7ff27cbcb640 1 --2- 172.21.15.76:0/2550890407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff278158670 0x7ff27814cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.065+0000 7ff27cbcb640 1 -- 172.21.15.76:0/2550890407 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff27806a390 con 0x7ff278158670 2024-04-25T19:16:01.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.065+0000 7ff27cbcb640 1 --2- 172.21.15.76:0/2550890407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff278158670 0x7ff27814cca0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff278074130 tx=0x7ff2680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:01.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.065+0000 7ff26dffb640 1 -- 172.21.15.76:0/2550890407 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff27401a070 con 0x7ff27806e0c0 2024-04-25T19:16:01.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.066+0000 7ff26dffb640 1 --2- 172.21.15.76:0/2550890407 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff25c0744e0 0x7ff25c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.066+0000 7ff26dffb640 1 -- 172.21.15.76:0/2550890407 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff274094e30 con 0x7ff27806e0c0 2024-04-25T19:16:01.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.066+0000 7ff27dbcd640 1 --2- 172.21.15.76:0/2550890407 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff25c0744e0 0x7ff25c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.066+0000 7ff26dffb640 1 -- 172.21.15.76:0/2550890407 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268042070 con 0x7ff278158670 2024-04-25T19:16:01.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.066+0000 7ff26dffb640 1 -- 172.21.15.76:0/2550890407 <== mon.0 v2:172.21.15.76:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff268032e20 con 0x7ff278158670 2024-04-25T19:16:01.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.066+0000 7ff26dffb640 1 -- 172.21.15.76:0/2550890407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff278158670 msgr2=0x7ff27814cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.066+0000 7ff26dffb640 1 --2- 172.21.15.76:0/2550890407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff278158670 0x7ff27814cca0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff278074130 tx=0x7ff2680046c0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.066+0000 7ff26dffb640 1 -- 172.21.15.76:0/2550890407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff278158670 msgr2=0x7ff27814cca0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:16:01.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.066+0000 7ff26dffb640 1 --2- 172.21.15.76:0/2550890407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff278158670 0x7ff27814cca0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.067+0000 7ff27dbcd640 1 --2- 172.21.15.76:0/2550890407 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff25c0744e0 0x7ff25c0769f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff270007930 tx=0x7ff270008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:01.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.073+0000 7ff27f630640 1 --2- 172.21.15.76:0/2550890407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff27815eec0 0x7ff278153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.073+0000 7ff27f630640 1 -- 172.21.15.76:0/2550890407 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff2780780c0 con 0x7ff27815eec0 2024-04-25T19:16:01.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.073+0000 7ff27cbcb640 1 --2- 172.21.15.76:0/2550890407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff27815eec0 0x7ff278153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.073+0000 7ff27cbcb640 1 -- 172.21.15.76:0/2550890407 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff27815a960 con 0x7ff27815eec0 2024-04-25T19:16:01.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.073+0000 7ff27cbcb640 1 --2- 172.21.15.76:0/2550890407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff27815eec0 0x7ff278153d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff26802a1e0 tx=0x7ff268008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:01.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.073+0000 7ff26dffb640 1 -- 172.21.15.76:0/2550890407 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268037040 con 0x7ff27815eec0 2024-04-25T19:16:01.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.073+0000 7ff26dffb640 1 -- 172.21.15.76:0/2550890407 <== mon.0 v2:172.21.15.76:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5066 (secure 0 0 0) 0x7ff268048ad0 con 0x7ff27815eec0 2024-04-25T19:16:01.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.073+0000 7ff26dffb640 1 -- 172.21.15.76:0/2550890407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff27815eec0 msgr2=0x7ff278153d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.073+0000 7ff26dffb640 1 --2- 172.21.15.76:0/2550890407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff27815eec0 0x7ff278153d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff26802a1e0 tx=0x7ff268008030 comp rx=0 tx=0).stop 2024-04-25T19:16:01.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.073+0000 7ff26dffb640 1 -- 172.21.15.76:0/2550890407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff27815eec0 msgr2=0x7ff278153d40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:16:01.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.073+0000 7ff26dffb640 1 --2- 172.21.15.76:0/2550890407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff27815eec0 0x7ff278153d40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.076 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:16:01.076 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:16:01.076 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:16:01.076 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "leader", 2024-04-25T19:16:01.076 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 4, 2024-04-25T19:16:01.077 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 59599, 2024-04-25T19:16:01.077 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:16:01.077 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:16:01.077 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-04-25T19:16:01.077 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:16:01.077 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:01.077 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 59, 2024-04-25T19:16:01.077 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:16:01.077 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:16:01.078 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:16:01.078 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:01.078 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:01.078 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:01.078 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:01.078 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:01.078 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:01.078 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:01.078 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:01.078 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:01.078 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:01.078 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:01.079 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:01.079 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:16:01.079 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:16:01.079 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:01.079 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:01.079 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:01.079 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:01.079 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:01.079 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:01.079 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:01.079 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:01.079 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:01.080 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:01.080 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:01.080 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:01.080 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.080 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:16:01.080 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:16:01.080 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:16:01.080 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:16:01.080 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:16:01.080 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:16:01.080 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:16:01.080 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:16:01.080 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:16:01.081 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:16:01.081 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:16:01.081 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:16:01.081 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:16:01.081 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:16:01.081 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:16:01.081 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:16:01.081 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:16:01.081 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:01.081 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:01.081 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:01.081 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:01.081 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:01.082 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:01.082 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:01.082 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:01.082 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:01.082 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:01.082 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:01.082 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:01.082 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:16:01.082 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.082 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:16:01.082 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.083 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:16:01.083 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:16:01.083 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:01.083 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:01.083 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.083 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:01.083 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:16:01.083 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:01.083 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.084 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.084 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:01.084 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:16:01.084 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:01.084 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:01.084 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:01.084 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.084 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:16:01.084 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:16:01.084 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:01.084 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:01.084 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:01.084 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.085 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.085 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:16:01.085 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:16:01.085 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:01.085 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:01.085 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.085 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:01.085 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:16:01.085 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:01.085 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.085 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.085 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:01.086 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:16:01.086 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:01.086 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:01.086 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:01.086 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.086 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:16:01.086 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:16:01.086 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:01.086 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:01.086 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:01.086 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.086 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.087 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:16:01.087 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:16:01.087 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:01.087 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:01.087 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.087 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:01.087 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:16:01.087 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:01.087 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.087 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.087 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:01.087 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:16:01.087 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:01.087 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:01.088 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:01.088 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.088 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:16:01.088 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:16:01.088 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:01.088 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:01.088 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:01.088 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:01.088 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:01.088 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.088 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:16:01.089 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:16:01.089 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.089 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:01.089 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:01.089 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:16:01.089 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:01.089 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:01.089 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": [ 2024-04-25T19:16:01.089 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.089 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:01.089 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:01.089 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 2 2024-04-25T19:16:01.089 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:01.090 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:01.090 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": [ 2024-04-25T19:16:01.090 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.090 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:01.090 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:01.090 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 3 2024-04-25T19:16:01.090 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:01.090 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:01.090 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:16:01.090 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.090 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:01.090 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:01.091 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 3 2024-04-25T19:16:01.091 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:01.091 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:01.091 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": [ 2024-04-25T19:16:01.091 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.091 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:01.091 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:01.091 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:16:01.091 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:01.091 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:01.091 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.091 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:16:01.092 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:16:01.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.074+0000 7ff27f630640 1 -- 172.21.15.76:0/2550890407 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff25c0744e0 msgr2=0x7ff25c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.074+0000 7ff27f630640 1 --2- 172.21.15.76:0/2550890407 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff25c0744e0 0x7ff25c0769f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff270007930 tx=0x7ff270008040 comp rx=0 tx=0).stop 2024-04-25T19:16:01.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.074+0000 7ff27f630640 1 -- 172.21.15.76:0/2550890407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff27806e0c0 msgr2=0x7ff27815e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.074+0000 7ff27f630640 1 --2- 172.21.15.76:0/2550890407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff27806e0c0 0x7ff27815e7b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff27400cb00 tx=0x7ff27400a060 comp rx=0 tx=0).stop 2024-04-25T19:16:01.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.074+0000 7ff27dbcd640 1 -- 172.21.15.76:0/2550890407 reap_dead start 2024-04-25T19:16:01.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.074+0000 7ff27f630640 1 -- 172.21.15.76:0/2550890407 shutdown_connections 2024-04-25T19:16:01.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.074+0000 7ff27f630640 1 --2- 172.21.15.76:0/2550890407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff27806e0c0 0x7ff27815e7b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.074+0000 7ff27f630640 1 -- 172.21.15.76:0/2550890407 >> 172.21.15.76:0/2550890407 conn(0x7ff278070b00 msgr2=0x7ff278154e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:01.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.074+0000 7ff27f630640 1 -- 172.21.15.76:0/2550890407 shutdown_connections 2024-04-25T19:16:01.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.074+0000 7ff27f630640 1 -- 172.21.15.76:0/2550890407 wait complete. 2024-04-25T19:16:01.094 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T19:16:01.094 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T19:16:01.094 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T19:16:01.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.201+0000 7fda232af640 1 Processor -- start 2024-04-25T19:16:01.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.202+0000 7fda232af640 1 -- start start 2024-04-25T19:16:01.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.203+0000 7fda232af640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fda1c1570d0 0x7fda1c160260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.203+0000 7fda232af640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fda1c06e450 0x7fda1c0cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.203+0000 7fda232af640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda1c1609a0 con 0x7fda1c06e040 2024-04-25T19:16:01.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.203+0000 7fda232af640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fda1c1574e0 con 0x7fda1c06e450 2024-04-25T19:16:01.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.203+0000 7fda232af640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fda1c160c80 con 0x7fda1c1570d0 2024-04-25T19:16:01.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.203+0000 7fda2184c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fda1c06e450 0x7fda1c0cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.203+0000 7fda2184c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fda1c06e450 0x7fda1c0cdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43544/0 (socket says 172.21.15.76:43544) 2024-04-25T19:16:01.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.203+0000 7fda2184c640 1 -- 172.21.15.76:0/1959893170 learned_addr learned my addr 172.21.15.76:0/1959893170 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:01.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.203+0000 7fda2104b640 1 --2- 172.21.15.76:0/1959893170 >> v2:172.21.15.105:3300/0 conn(0x7fda1c1570d0 0x7fda1c160260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.203+0000 7fda2184c640 1 -- 172.21.15.76:0/1959893170 >> v2:172.21.15.105:3300/0 conn(0x7fda1c1570d0 msgr2=0x7fda1c160260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:01.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.203+0000 7fda2184c640 1 --2- 172.21.15.76:0/1959893170 >> v2:172.21.15.105:3300/0 conn(0x7fda1c1570d0 0x7fda1c160260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.203+0000 7fda2184c640 1 -- 172.21.15.76:0/1959893170 >> v1:172.21.15.105:6789/0 conn(0x7fda1c06e040 legacy=0x7fda1c06cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:01.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.204+0000 7fda2184c640 1 -- 172.21.15.76:0/1959893170 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1c160f60 con 0x7fda1c06e450 2024-04-25T19:16:01.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.204+0000 7fda2184c640 1 --2- 172.21.15.76:0/1959893170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1c06e450 0x7fda1c0cdc70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fda1c08f0f0 tx=0x7fda1402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abcc9fe95f0a043 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:01.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.204+0000 7fda13fff640 1 -- 172.21.15.76:0/1959893170 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda14037070 con 0x7fda1c06e450 2024-04-25T19:16:01.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.204+0000 7fda13fff640 1 -- 172.21.15.76:0/1959893170 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda1402ae50 con 0x7fda1c06e450 2024-04-25T19:16:01.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.204+0000 7fda13fff640 1 -- 172.21.15.76:0/1959893170 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda14032860 con 0x7fda1c06e450 2024-04-25T19:16:01.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.205+0000 7fda232af640 1 -- 172.21.15.76:0/1959893170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1c06e450 msgr2=0x7fda1c0cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.205+0000 7fda232af640 1 --2- 172.21.15.76:0/1959893170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1c06e450 0x7fda1c0cdc70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fda1c08f0f0 tx=0x7fda1402a300 comp rx=0 tx=0).stop 2024-04-25T19:16:01.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.205+0000 7fda232af640 1 -- 172.21.15.76:0/1959893170 shutdown_connections 2024-04-25T19:16:01.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.205+0000 7fda232af640 1 --2- 172.21.15.76:0/1959893170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1c06e450 0x7fda1c0cdc70 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.205+0000 7fda232af640 1 --2- 172.21.15.76:0/1959893170 >> v2:172.21.15.105:3300/0 conn(0x7fda1c1570d0 0x7fda1c160260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.205+0000 7fda232af640 1 -- 172.21.15.76:0/1959893170 >> 172.21.15.76:0/1959893170 conn(0x7fda1c0d22d0 msgr2=0x7fda1c0d26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:01.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.205+0000 7fda232af640 1 -- 172.21.15.76:0/1959893170 shutdown_connections 2024-04-25T19:16:01.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.205+0000 7fda232af640 1 -- 172.21.15.76:0/1959893170 wait complete. 2024-04-25T19:16:01.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.207+0000 7fda232af640 1 Processor -- start 2024-04-25T19:16:01.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.207+0000 7fda232af640 1 -- start start 2024-04-25T19:16:01.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.208+0000 7fda232af640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1c06e040 0x7fda1c0cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.208+0000 7fda232af640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda1c06e450 0x7fda1c1525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.208+0000 7fda232af640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda1c1570d0 0x7fda1c1f3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.208+0000 7fda232af640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c06d480 con 0x7fda1c06e040 2024-04-25T19:16:01.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.208+0000 7fda232af640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c06db20 con 0x7fda1c1570d0 2024-04-25T19:16:01.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.208+0000 7fda232af640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fda1c06d600 con 0x7fda1c06e450 2024-04-25T19:16:01.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.208+0000 7fda2104b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1c06e040 0x7fda1c0cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.208+0000 7fda2104b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1c06e040 0x7fda1c0cac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43548/0 (socket says 172.21.15.76:43548) 2024-04-25T19:16:01.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.208+0000 7fda2104b640 1 -- 172.21.15.76:0/4024462718 learned_addr learned my addr 172.21.15.76:0/4024462718 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:01.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.208+0000 7fda2084a640 1 --2- 172.21.15.76:0/4024462718 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda1c06e450 0x7fda1c1525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.208+0000 7fda2184c640 1 --2- 172.21.15.76:0/4024462718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda1c1570d0 0x7fda1c1f3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.209+0000 7fda2104b640 1 -- 172.21.15.76:0/4024462718 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda1c06e450 msgr2=0x7fda1c1525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.209+0000 7fda2104b640 1 --2- 172.21.15.76:0/4024462718 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fda1c06e450 0x7fda1c1525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.209+0000 7fda2104b640 1 -- 172.21.15.76:0/4024462718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda1c1570d0 msgr2=0x7fda1c1f3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.209+0000 7fda2104b640 1 --2- 172.21.15.76:0/4024462718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda1c1570d0 0x7fda1c1f3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.209+0000 7fda2104b640 1 -- 172.21.15.76:0/4024462718 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1c0cbef0 con 0x7fda1c06e040 2024-04-25T19:16:01.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.209+0000 7fda2104b640 1 --2- 172.21.15.76:0/4024462718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1c06e040 0x7fda1c0cac50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fda18007e40 tx=0x7fda1800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:01.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.209+0000 7fda11ffb640 1 -- 172.21.15.76:0/4024462718 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda18014620 con 0x7fda1c06e040 2024-04-25T19:16:01.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.209+0000 7fda11ffb640 1 -- 172.21.15.76:0/4024462718 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda18019070 con 0x7fda1c06e040 2024-04-25T19:16:01.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.209+0000 7fda232af640 1 -- 172.21.15.76:0/4024462718 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda1c152b10 con 0x7fda1c06e040 2024-04-25T19:16:01.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.209+0000 7fda11ffb640 1 -- 172.21.15.76:0/4024462718 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda18014a00 con 0x7fda1c06e040 2024-04-25T19:16:01.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.209+0000 7fda232af640 1 -- 172.21.15.76:0/4024462718 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda1c0cb650 con 0x7fda1c06e040 2024-04-25T19:16:01.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.210+0000 7fda232af640 1 --2- 172.21.15.76:0/4024462718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda1c1f2ba0 0x7fda1c0cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.210+0000 7fda232af640 1 -- 172.21.15.76:0/4024462718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c06a390 con 0x7fda1c1f2ba0 2024-04-25T19:16:01.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.210+0000 7fda2084a640 1 --2- 172.21.15.76:0/4024462718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda1c1f2ba0 0x7fda1c0cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.210+0000 7fda2084a640 1 -- 172.21.15.76:0/4024462718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fda1c0cd790 con 0x7fda1c1f2ba0 2024-04-25T19:16:01.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.211+0000 7fda11ffb640 1 -- 172.21.15.76:0/4024462718 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fda18014ba0 con 0x7fda1c06e040 2024-04-25T19:16:01.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.211+0000 7fda2084a640 1 --2- 172.21.15.76:0/4024462718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda1c1f2ba0 0x7fda1c0cb930 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0d0fe0 tx=0x7fda0c00f900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:01.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.211+0000 7fda11ffb640 1 --2- 172.21.15.76:0/4024462718 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fda000744b0 0x7fda000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.211+0000 7fda11ffb640 1 -- 172.21.15.76:0/4024462718 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fda18095860 con 0x7fda1c06e040 2024-04-25T19:16:01.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.211+0000 7fda2184c640 1 --2- 172.21.15.76:0/4024462718 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fda000744b0 0x7fda000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.211+0000 7fda11ffb640 1 -- 172.21.15.76:0/4024462718 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c019070 con 0x7fda1c1f2ba0 2024-04-25T19:16:01.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.211+0000 7fda11ffb640 1 -- 172.21.15.76:0/4024462718 <== mon.1 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fda0c005320 con 0x7fda1c1f2ba0 2024-04-25T19:16:01.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.211+0000 7fda11ffb640 1 -- 172.21.15.76:0/4024462718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda1c1f2ba0 msgr2=0x7fda1c0cb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.211+0000 7fda11ffb640 1 --2- 172.21.15.76:0/4024462718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda1c1f2ba0 0x7fda1c0cb930 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0d0fe0 tx=0x7fda0c00f900 comp rx=0 tx=0).stop 2024-04-25T19:16:01.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.211+0000 7fda11ffb640 1 -- 172.21.15.76:0/4024462718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda1c1f2ba0 msgr2=0x7fda1c0cb930 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:16:01.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.211+0000 7fda11ffb640 1 --2- 172.21.15.76:0/4024462718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda1c1f2ba0 0x7fda1c0cb930 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.212+0000 7fda2184c640 1 --2- 172.21.15.76:0/4024462718 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fda000744b0 0x7fda000769c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fda14034000 tx=0x7fda1402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:01.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.218+0000 7fda232af640 1 --2- 172.21.15.76:0/4024462718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda1c1f6050 0x7fda1c1f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.218+0000 7fda232af640 1 -- 172.21.15.76:0/4024462718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c0d0d30 con 0x7fda1c1f6050 2024-04-25T19:16:01.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.218+0000 7fda2084a640 1 --2- 172.21.15.76:0/4024462718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda1c1f6050 0x7fda1c1f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.218+0000 7fda2084a640 1 -- 172.21.15.76:0/4024462718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fda1c0d2ad0 con 0x7fda1c1f6050 2024-04-25T19:16:01.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.219+0000 7fda2084a640 1 --2- 172.21.15.76:0/4024462718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda1c1f6050 0x7fda1c1f6460 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0d0fe0 tx=0x7fda0c000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:01.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.219+0000 7fda11ffb640 1 -- 172.21.15.76:0/4024462718 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c019030 con 0x7fda1c1f6050 2024-04-25T19:16:01.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.219+0000 7fda11ffb640 1 -- 172.21.15.76:0/4024462718 <== mon.1 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4724 (secure 0 0 0) 0x7fda0c006c60 con 0x7fda1c1f6050 2024-04-25T19:16:01.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.219+0000 7fda11ffb640 1 -- 172.21.15.76:0/4024462718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda1c1f6050 msgr2=0x7fda1c1f6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.219+0000 7fda11ffb640 1 --2- 172.21.15.76:0/4024462718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda1c1f6050 0x7fda1c1f6460 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fda1c0d0fe0 tx=0x7fda0c000ed0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.219+0000 7fda11ffb640 1 -- 172.21.15.76:0/4024462718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda1c1f6050 msgr2=0x7fda1c1f6460 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:16:01.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.219+0000 7fda11ffb640 1 --2- 172.21.15.76:0/4024462718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda1c1f6050 0x7fda1c1f6460 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.221 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:16:01.221 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:16:01.221 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:16:01.222 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "peon", 2024-04-25T19:16:01.222 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 4, 2024-04-25T19:16:01.222 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 59700, 2024-04-25T19:16:01.222 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:16:01.222 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:16:01.222 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-04-25T19:16:01.222 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:16:01.222 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:01.222 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 59, 2024-04-25T19:16:01.222 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:16:01.222 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:16:01.222 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:16:01.222 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:01.222 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:01.223 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:01.223 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:01.223 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:01.223 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:01.223 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:01.223 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:01.223 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:01.223 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:01.223 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:01.223 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:01.223 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:16:01.223 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:16:01.223 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:01.223 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:01.223 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:01.223 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:01.224 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:01.224 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:01.224 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:01.224 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:01.224 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:01.224 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:01.224 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:01.224 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:01.224 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.224 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:16:01.224 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:16:01.224 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:16:01.224 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:16:01.224 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:16:01.224 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:16:01.225 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:16:01.225 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:16:01.225 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:16:01.225 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:16:01.225 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:16:01.225 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:16:01.225 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:16:01.225 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:16:01.225 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:16:01.225 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:16:01.225 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:16:01.225 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:01.225 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:01.225 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:01.225 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:01.226 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:01.226 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:01.226 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:01.226 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:01.226 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:01.226 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:01.226 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:01.226 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:01.226 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:16:01.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.226 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:16:01.226 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.226 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:16:01.226 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:16:01.227 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:01.227 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:01.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.227 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:01.227 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:16:01.227 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:01.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.227 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:01.227 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:16:01.227 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:01.227 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:01.227 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:01.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.227 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:16:01.228 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:16:01.228 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:01.228 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:01.228 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:01.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.228 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.228 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:16:01.228 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:16:01.228 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:01.228 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:01.228 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.228 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:01.228 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:16:01.228 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:01.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.229 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:01.229 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:16:01.229 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:01.229 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:01.229 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:01.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.229 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:16:01.229 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:16:01.229 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:01.229 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:01.229 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:01.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.230 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:16:01.230 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:16:01.230 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:01.230 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:01.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.230 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:01.230 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:16:01.230 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:01.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.230 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:01.230 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:16:01.230 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:01.230 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:01.230 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:01.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.231 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:16:01.231 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:16:01.231 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:01.231 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:01.231 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:01.231 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:01.231 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:01.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.231 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:16:01.231 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:16:01.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.232 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:01.232 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:01.232 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:16:01.232 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:01.232 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:01.232 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": [ 2024-04-25T19:16:01.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.232 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:01.232 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:01.232 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 3 2024-04-25T19:16:01.233 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:01.233 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:01.233 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:16:01.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.233 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:01.233 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:01.233 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 2 2024-04-25T19:16:01.233 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:01.233 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:01.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.233 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:16:01.233 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:16:01.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.220+0000 7fda232af640 1 -- 172.21.15.76:0/4024462718 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fda000744b0 msgr2=0x7fda000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.220+0000 7fda232af640 1 --2- 172.21.15.76:0/4024462718 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fda000744b0 0x7fda000769c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fda14034000 tx=0x7fda1402e000 comp rx=0 tx=0).stop 2024-04-25T19:16:01.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.220+0000 7fda232af640 1 -- 172.21.15.76:0/4024462718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1c06e040 msgr2=0x7fda1c0cac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.220+0000 7fda232af640 1 --2- 172.21.15.76:0/4024462718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1c06e040 0x7fda1c0cac50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fda18007e40 tx=0x7fda1800e4b0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.220+0000 7fda2184c640 1 -- 172.21.15.76:0/4024462718 reap_dead start 2024-04-25T19:16:01.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.220+0000 7fda232af640 1 -- 172.21.15.76:0/4024462718 shutdown_connections 2024-04-25T19:16:01.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.220+0000 7fda232af640 1 --2- 172.21.15.76:0/4024462718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1c06e040 0x7fda1c0cac50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.220+0000 7fda232af640 1 -- 172.21.15.76:0/4024462718 >> 172.21.15.76:0/4024462718 conn(0x7fda1c0d22d0 msgr2=0x7fda1c0d1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:01.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.220+0000 7fda232af640 1 -- 172.21.15.76:0/4024462718 shutdown_connections 2024-04-25T19:16:01.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.220+0000 7fda232af640 1 -- 172.21.15.76:0/4024462718 wait complete. 2024-04-25T19:16:01.236 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T19:16:01.236 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T19:16:01.236 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T19:16:01.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.342+0000 7f0785d84640 1 Processor -- start 2024-04-25T19:16:01.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.342+0000 7f0785d84640 1 -- start start 2024-04-25T19:16:01.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.344+0000 7f0785d84640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f07800c8ed0 0x7f07800c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.344+0000 7f0785d84640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07800c9c40 0x7f078015af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.344+0000 7f0785d84640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0780073450 con 0x7f0780154d10 2024-04-25T19:16:01.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.344+0000 7f0785d84640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f0780073ea0 con 0x7f07800c8ed0 2024-04-25T19:16:01.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.344+0000 7f0785d84640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f07800ca050 con 0x7f07800c9c40 2024-04-25T19:16:01.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.344+0000 7f077effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f07800c8ed0 0x7f07800c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.344+0000 7f077f7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0780154d10 0x7f0780068520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33014/0 (socket says 172.21.15.76:33014) 2024-04-25T19:16:01.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.344+0000 7f077f7fe640 1 -- 172.21.15.76:0/2804394134 learned_addr learned my addr 172.21.15.76:0/2804394134 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:16:01.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.344+0000 7f077ffff640 1 --2- 172.21.15.76:0/2804394134 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07800c9c40 0x7f078015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.344+0000 7f077effd640 1 -- 172.21.15.76:0/2804394134 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07800c9c40 msgr2=0x7f078015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:01.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.344+0000 7f077effd640 1 --2- 172.21.15.76:0/2804394134 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07800c9c40 0x7f078015af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.344+0000 7f077effd640 1 -- 172.21.15.76:0/2804394134 >> v1:172.21.15.76:6789/0 conn(0x7f0780154d10 legacy=0x7f0780068520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.344+0000 7f077effd640 1 -- 172.21.15.76:0/2804394134 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f078015aa60 con 0x7f07800c8ed0 2024-04-25T19:16:01.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.344+0000 7f077e7fc640 1 -- 172.21.15.76:0/2804394134 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4066712627 0 0) 0x7f0770001110 con 0x7f0780154d10 2024-04-25T19:16:01.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.345+0000 7f077effd640 1 --2- 172.21.15.76:0/2804394134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07800c8ed0 0x7f07800c9700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0780074020 tx=0x7f077402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793882d8f54ca250 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:01.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.345+0000 7f077e7fc640 1 -- 172.21.15.76:0/2804394134 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0774037070 con 0x7f07800c8ed0 2024-04-25T19:16:01.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.345+0000 7f077e7fc640 1 -- 172.21.15.76:0/2804394134 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0774032440 con 0x7f07800c8ed0 2024-04-25T19:16:01.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.345+0000 7f077e7fc640 1 -- 172.21.15.76:0/2804394134 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0774032810 con 0x7f07800c8ed0 2024-04-25T19:16:01.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.346+0000 7f0785d84640 1 -- 172.21.15.76:0/2804394134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07800c8ed0 msgr2=0x7f07800c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.346+0000 7f0785d84640 1 --2- 172.21.15.76:0/2804394134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07800c8ed0 0x7f07800c9700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0780074020 tx=0x7f077402a180 comp rx=0 tx=0).stop 2024-04-25T19:16:01.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.346+0000 7f0785d84640 1 -- 172.21.15.76:0/2804394134 shutdown_connections 2024-04-25T19:16:01.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.346+0000 7f0785d84640 1 --2- 172.21.15.76:0/2804394134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07800c8ed0 0x7f07800c9700 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.346+0000 7f0785d84640 1 --2- 172.21.15.76:0/2804394134 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07800c9c40 0x7f078015af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.346+0000 7f0785d84640 1 -- 172.21.15.76:0/2804394134 >> 172.21.15.76:0/2804394134 conn(0x7f078006f720 msgr2=0x7f0780070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:01.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.346+0000 7f0785d84640 1 -- 172.21.15.76:0/2804394134 shutdown_connections 2024-04-25T19:16:01.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.346+0000 7f0785d84640 1 -- 172.21.15.76:0/2804394134 wait complete. 2024-04-25T19:16:01.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.347+0000 7f0785d84640 1 Processor -- start 2024-04-25T19:16:01.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.348+0000 7f0785d84640 1 -- start start 2024-04-25T19:16:01.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.349+0000 7f0785d84640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07800c8ed0 0x7f0780149d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.349+0000 7f0785d84640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07800c9c40 0x7f078014a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.349+0000 7f0785d84640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0780154d10 0x7f07800c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.349+0000 7f0785d84640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f07800746b0 con 0x7f07800c9c40 2024-04-25T19:16:01.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.349+0000 7f0785d84640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f07800743d0 con 0x7f0780154d10 2024-04-25T19:16:01.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.349+0000 7f0785d84640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f0780074990 con 0x7f07800c8ed0 2024-04-25T19:16:01.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.349+0000 7f077effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07800c9c40 0x7f078014a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.349+0000 7f077f7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07800c8ed0 0x7f0780149d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.349+0000 7f077effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07800c9c40 0x7f078014a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43572/0 (socket says 172.21.15.76:43572) 2024-04-25T19:16:01.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.349+0000 7f077effd640 1 -- 172.21.15.76:0/1000552319 learned_addr learned my addr 172.21.15.76:0/1000552319 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:01.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.349+0000 7f077ffff640 1 --2- 172.21.15.76:0/1000552319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0780154d10 0x7f07800c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.349+0000 7f077effd640 1 -- 172.21.15.76:0/1000552319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07800c8ed0 msgr2=0x7f0780149d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.349+0000 7f077effd640 1 --2- 172.21.15.76:0/1000552319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07800c8ed0 0x7f0780149d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.349+0000 7f077effd640 1 -- 172.21.15.76:0/1000552319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0780154d10 msgr2=0x7f07800c7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.349+0000 7f077effd640 1 --2- 172.21.15.76:0/1000552319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0780154d10 0x7f07800c7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.349+0000 7f077effd640 1 -- 172.21.15.76:0/1000552319 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07800c4570 con 0x7f07800c9c40 2024-04-25T19:16:01.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.349+0000 7f077effd640 1 --2- 172.21.15.76:0/1000552319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07800c9c40 0x7f078014a270 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0774035040 tx=0x7f07740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:01.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.350+0000 7f0767fff640 1 -- 172.21.15.76:0/1000552319 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0774037050 con 0x7f07800c9c40 2024-04-25T19:16:01.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.350+0000 7f0767fff640 1 -- 172.21.15.76:0/1000552319 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0774038040 con 0x7f07800c9c40 2024-04-25T19:16:01.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.350+0000 7f0767fff640 1 -- 172.21.15.76:0/1000552319 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0774004470 con 0x7f07800c9c40 2024-04-25T19:16:01.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.350+0000 7f0785d84640 1 -- 172.21.15.76:0/1000552319 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07800c4810 con 0x7f07800c9c40 2024-04-25T19:16:01.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.350+0000 7f0785d84640 1 -- 172.21.15.76:0/1000552319 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f078014a7b0 con 0x7f07800c9c40 2024-04-25T19:16:01.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.351+0000 7f0785d84640 1 --2- 172.21.15.76:0/1000552319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07800c8580 0x7f07800c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.351+0000 7f0785d84640 1 -- 172.21.15.76:0/1000552319 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f07801e5550 con 0x7f07800c8580 2024-04-25T19:16:01.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.351+0000 7f077f7fe640 1 --2- 172.21.15.76:0/1000552319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07800c8580 0x7f07800c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.351+0000 7f077f7fe640 1 -- 172.21.15.76:0/1000552319 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f078006a390 con 0x7f07800c8580 2024-04-25T19:16:01.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.351+0000 7f077f7fe640 1 --2- 172.21.15.76:0/1000552319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07800c8580 0x7f07800c7c40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0770000f60 tx=0x7f07700018d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:01.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.351+0000 7f0767fff640 1 -- 172.21.15.76:0/1000552319 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0774003610 con 0x7f07800c9c40 2024-04-25T19:16:01.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.352+0000 7f0767fff640 1 --2- 172.21.15.76:0/1000552319 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f076c0744b0 0x7f076c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.352+0000 7f0767fff640 1 -- 172.21.15.76:0/1000552319 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f07740b59b0 con 0x7f07800c9c40 2024-04-25T19:16:01.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.352+0000 7f077ffff640 1 --2- 172.21.15.76:0/1000552319 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f076c0744b0 0x7f076c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.352+0000 7f0767fff640 1 -- 172.21.15.76:0/1000552319 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770011580 con 0x7f07800c8580 2024-04-25T19:16:01.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.352+0000 7f0767fff640 1 -- 172.21.15.76:0/1000552319 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0770018070 con 0x7f07800c8580 2024-04-25T19:16:01.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.352+0000 7f0767fff640 1 -- 172.21.15.76:0/1000552319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07800c8580 msgr2=0x7f07800c7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.352+0000 7f0767fff640 1 --2- 172.21.15.76:0/1000552319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07800c8580 0x7f07800c7c40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0770000f60 tx=0x7f07700018d0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.352+0000 7f0767fff640 1 -- 172.21.15.76:0/1000552319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07800c8580 msgr2=0x7f07800c7c40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:16:01.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.352+0000 7f0767fff640 1 --2- 172.21.15.76:0/1000552319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07800c8580 0x7f07800c7c40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.353+0000 7f077ffff640 1 --2- 172.21.15.76:0/1000552319 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f076c0744b0 0x7f076c0769c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f07800ce480 tx=0x7f077800b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:01.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.358+0000 7f0785d84640 1 --2- 172.21.15.76:0/1000552319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07801f5d30 0x7f07801f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.359+0000 7f0785d84640 1 -- 172.21.15.76:0/1000552319 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f07800c92e0 con 0x7f07801f5d30 2024-04-25T19:16:01.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.359+0000 7f077f7fe640 1 --2- 172.21.15.76:0/1000552319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07801f5d30 0x7f07801f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.359+0000 7f077f7fe640 1 -- 172.21.15.76:0/1000552319 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f07801f3410 con 0x7f07801f5d30 2024-04-25T19:16:01.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.359+0000 7f077f7fe640 1 --2- 172.21.15.76:0/1000552319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07801f5d30 0x7f07801f6140 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0770000f60 tx=0x7f0770001080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:01.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.360+0000 7f0767fff640 1 -- 172.21.15.76:0/1000552319 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770011dd0 con 0x7f07801f5d30 2024-04-25T19:16:01.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.360+0000 7f0767fff640 1 -- 172.21.15.76:0/1000552319 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5064 (secure 0 0 0) 0x7f0770011dd0 con 0x7f07801f5d30 2024-04-25T19:16:01.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.360+0000 7f0767fff640 1 -- 172.21.15.76:0/1000552319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07801f5d30 msgr2=0x7f07801f6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.360+0000 7f0767fff640 1 --2- 172.21.15.76:0/1000552319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07801f5d30 0x7f07801f6140 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0770000f60 tx=0x7f0770001080 comp rx=0 tx=0).stop 2024-04-25T19:16:01.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.360+0000 7f0767fff640 1 -- 172.21.15.76:0/1000552319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07801f5d30 msgr2=0x7f07801f6140 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:16:01.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.360+0000 7f0767fff640 1 --2- 172.21.15.76:0/1000552319 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f07801f5d30 0x7f07801f6140 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.362 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:16:01.362 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:16:01.362 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:16:01.363 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "peon", 2024-04-25T19:16:01.363 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 4, 2024-04-25T19:16:01.363 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 59841, 2024-04-25T19:16:01.363 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:16:01.363 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:16:01.363 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-04-25T19:16:01.363 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:16:01.363 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:01.363 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 59, 2024-04-25T19:16:01.363 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:16:01.363 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:16:01.363 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:16:01.363 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:01.364 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:01.364 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:01.364 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:01.364 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:01.364 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:01.364 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:01.364 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:01.364 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:01.364 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:01.364 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:01.364 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:01.364 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:16:01.364 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:16:01.364 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:01.364 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:01.365 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:01.365 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:01.365 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:01.365 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:01.365 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:01.365 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:01.365 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:01.365 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:01.365 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:01.365 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:01.365 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.365 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:16:01.365 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:16:01.365 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:16:01.365 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:16:01.366 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:16:01.366 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:16:01.366 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:16:01.366 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:16:01.366 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:16:01.366 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:16:01.366 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:16:01.366 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:16:01.366 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:16:01.366 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:16:01.366 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:16:01.366 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:16:01.366 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:16:01.366 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:01.367 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:01.367 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:01.367 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:01.367 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:01.367 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:01.367 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:01.367 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:01.367 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:01.367 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:01.367 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:01.368 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:01.368 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:16:01.368 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.368 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:16:01.368 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.368 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:16:01.368 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:16:01.368 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:01.368 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:01.368 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.368 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:01.368 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:16:01.368 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:01.368 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.368 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.369 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:01.369 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:16:01.369 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:01.369 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:01.369 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:01.369 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.369 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:16:01.369 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:16:01.369 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:01.369 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:01.369 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:01.369 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.369 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.369 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:16:01.369 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:16:01.370 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:01.370 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:01.370 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.370 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:01.370 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:16:01.370 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:01.370 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.370 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.370 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:01.370 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:16:01.370 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:01.370 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:01.370 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:01.370 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.371 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:16:01.371 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:16:01.371 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:01.371 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:01.371 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:01.371 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.371 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.371 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:16:01.371 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:16:01.371 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:01.371 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:01.371 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.372 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:01.372 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:16:01.372 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:01.372 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.372 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.372 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:01.372 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:16:01.372 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:01.372 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:01.372 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:01.372 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.372 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:16:01.372 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:16:01.372 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:01.373 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:01.373 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:01.373 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:01.373 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:01.373 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.373 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:16:01.373 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:16:01.373 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.373 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:01.373 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:01.373 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:16:01.373 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:01.374 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:01.374 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": [ 2024-04-25T19:16:01.374 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.374 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:01.374 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:01.374 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 4 2024-04-25T19:16:01.374 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:01.374 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:01.374 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": [ 2024-04-25T19:16:01.374 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.375 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:01.375 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:01.375 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 2 2024-04-25T19:16:01.375 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:01.375 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:01.375 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:16:01.375 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.375 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:01.375 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:01.375 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 4 2024-04-25T19:16:01.375 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:01.375 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:01.375 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": [ 2024-04-25T19:16:01.375 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:01.376 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:01.376 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:01.376 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:16:01.376 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:01.376 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:01.376 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:01.376 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:16:01.376 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:16:01.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.360+0000 7f0785d84640 1 -- 172.21.15.76:0/1000552319 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f076c0744b0 msgr2=0x7f076c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.360+0000 7f0785d84640 1 --2- 172.21.15.76:0/1000552319 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f076c0744b0 0x7f076c0769c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f07800ce480 tx=0x7f077800b040 comp rx=0 tx=0).stop 2024-04-25T19:16:01.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.360+0000 7f0785d84640 1 -- 172.21.15.76:0/1000552319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07800c9c40 msgr2=0x7f078014a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.360+0000 7f0785d84640 1 --2- 172.21.15.76:0/1000552319 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07800c9c40 0x7f078014a270 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0774035040 tx=0x7f07740046c0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.360+0000 7f077ffff640 1 -- 172.21.15.76:0/1000552319 reap_dead start 2024-04-25T19:16:01.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.360+0000 7f077ffff640 1 -- 172.21.15.76:0/1000552319 reap_dead start 2024-04-25T19:16:01.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.360+0000 7f0785d84640 1 -- 172.21.15.76:0/1000552319 shutdown_connections 2024-04-25T19:16:01.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.360+0000 7f0785d84640 1 -- 172.21.15.76:0/1000552319 >> 172.21.15.76:0/1000552319 conn(0x7f078006f720 msgr2=0x7f0780159a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:01.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.360+0000 7f0785d84640 1 -- 172.21.15.76:0/1000552319 shutdown_connections 2024-04-25T19:16:01.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.360+0000 7f0785d84640 1 -- 172.21.15.76:0/1000552319 wait complete. 2024-04-25T19:16:01.378 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T19:16:01.379 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-04-25T19:16:01.379 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-25T19:16:01.379 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-04-25T19:16:01.379 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-25T19:16:01.379 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T19:16:01.379 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:16:01.397 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T19:16:01.397 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-04-25T19:16:01.397 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:16:01.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.504+0000 7f4cfb5f3640 1 Processor -- start 2024-04-25T19:16:01.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.504+0000 7f4cfb5f3640 1 -- start start 2024-04-25T19:16:01.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.505+0000 7f4cfb5f3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4cf4073880 0x7f4cf40c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.505+0000 7f4cfb5f3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4cf40c8910 0x7f4cf415ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.505+0000 7f4cfb5f3640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cf4071ad0 con 0x7f4cf4071520 2024-04-25T19:16:01.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.505+0000 7f4cfb5f3640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f4cf4071d70 con 0x7f4cf4073880 2024-04-25T19:16:01.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.505+0000 7f4cfb5f3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f4cf4074150 con 0x7f4cf40c8910 2024-04-25T19:16:01.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.505+0000 7f4cf938f640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f4cf4071520 0x7f4cf4151350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33028/0 (socket says 172.21.15.76:33028) 2024-04-25T19:16:01.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.505+0000 7f4cf938f640 1 -- 172.21.15.76:0/619114131 learned_addr learned my addr 172.21.15.76:0/619114131 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:16:01.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.505+0000 7f4cf9b90640 1 -- 172.21.15.76:0/619114131 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4cf40c8910 msgr2=0x7f4cf415ac70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3301/0 2024-04-25T19:16:01.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.505+0000 7f4cf9b90640 1 --2- 172.21.15.76:0/619114131 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4cf40c8910 0x7f4cf415ac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:01.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.505+0000 7f4cf8b8e640 1 --2- 172.21.15.76:0/619114131 >> v2:172.21.15.105:3300/0 conn(0x7f4cf4073880 0x7f4cf40c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.506+0000 7f4cebfff640 1 -- 172.21.15.76:0/619114131 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4148724409 0 0) 0x7f4cf4071ad0 con 0x7f4cf4071520 2024-04-25T19:16:01.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.506+0000 7f4cebfff640 1 -- 172.21.15.76:0/619114131 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ce0003520 con 0x7f4cf4071520 2024-04-25T19:16:01.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.506+0000 7f4cebfff640 1 -- 172.21.15.76:0/619114131 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1787423571 0 0) 0x7f4ce0003520 con 0x7f4cf4071520 2024-04-25T19:16:01.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.506+0000 7f4cebfff640 1 -- 172.21.15.76:0/619114131 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4cf40c8910 msgr2=0x7f4cf415ac70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:16:01.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.506+0000 7f4cebfff640 1 --2- 172.21.15.76:0/619114131 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4cf40c8910 0x7f4cf415ac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.506+0000 7f4cebfff640 1 -- 172.21.15.76:0/619114131 >> v2:172.21.15.105:3300/0 conn(0x7f4cf4073880 msgr2=0x7f4cf40c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:01.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.506+0000 7f4cebfff640 1 --2- 172.21.15.76:0/619114131 >> v2:172.21.15.105:3300/0 conn(0x7f4cf4073880 0x7f4cf40c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.506+0000 7f4cebfff640 1 -- 172.21.15.76:0/619114131 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf4074430 con 0x7f4cf4071520 2024-04-25T19:16:01.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.506+0000 7f4cebfff640 1 -- 172.21.15.76:0/619114131 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f4cf0001670 con 0x7f4cf4071520 2024-04-25T19:16:01.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.506+0000 7f4cebfff640 1 -- 172.21.15.76:0/619114131 >> v1:172.21.15.76:6789/0 conn(0x7f4cf4071520 legacy=0x7f4cf4151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.506+0000 7f4cebfff640 1 --2- 172.21.15.76:0/619114131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ce00039b0 0x7f4ce0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.506+0000 7f4cebfff640 1 --2- 172.21.15.76:0/619114131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ce0004800 0x7f4ce0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.506+0000 7f4cebfff640 1 --2- 172.21.15.76:0/619114131 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4cf40c8910 0x7f4ce0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.506+0000 7f4cebfff640 1 -- 172.21.15.76:0/619114131 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf4071d70 con 0x7f4ce0004800 2024-04-25T19:16:01.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.506+0000 7f4cebfff640 1 -- 172.21.15.76:0/619114131 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf4074150 con 0x7f4ce00039b0 2024-04-25T19:16:01.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.506+0000 7f4cebfff640 1 -- 172.21.15.76:0/619114131 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f4ce000bbb0 con 0x7f4cf40c8910 2024-04-25T19:16:01.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.506+0000 7f4cf9b90640 1 -- 172.21.15.76:0/619114131 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4cf40c8910 msgr2=0x7f4ce0005190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3301/0 2024-04-25T19:16:01.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.506+0000 7f4cf8b8e640 1 --2- 172.21.15.76:0/619114131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ce0004800 0x7f4ce0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.506+0000 7f4cf9b90640 1 --2- 172.21.15.76:0/619114131 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4cf40c8910 0x7f4ce0005190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:01.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.506+0000 7f4cf938f640 1 --2- 172.21.15.76:0/619114131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ce00039b0 0x7f4ce0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.506+0000 7f4cf8b8e640 1 -- 172.21.15.76:0/619114131 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4cf40c8910 msgr2=0x7f4ce0005190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:16:01.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.506+0000 7f4cf8b8e640 1 --2- 172.21.15.76:0/619114131 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4cf40c8910 0x7f4ce0005190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.506+0000 7f4cf8b8e640 1 -- 172.21.15.76:0/619114131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ce00039b0 msgr2=0x7f4ce0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:01.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.506+0000 7f4cf8b8e640 1 --2- 172.21.15.76:0/619114131 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ce00039b0 0x7f4ce0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.506+0000 7f4cf8b8e640 1 -- 172.21.15.76:0/619114131 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce0003520 con 0x7f4ce0004800 2024-04-25T19:16:01.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.507+0000 7f4cf8b8e640 1 --2- 172.21.15.76:0/619114131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ce0004800 0x7f4ce0004c10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009a90 tx=0x7f4ce0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a77e80f22697032 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:01.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.507+0000 7f4cebfff640 1 -- 172.21.15.76:0/619114131 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4036070 con 0x7f4ce0004800 2024-04-25T19:16:01.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.507+0000 7f4cebfff640 1 -- 172.21.15.76:0/619114131 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce4031400 con 0x7f4ce0004800 2024-04-25T19:16:01.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.507+0000 7f4cebfff640 1 -- 172.21.15.76:0/619114131 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4009690 con 0x7f4ce0004800 2024-04-25T19:16:01.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.508+0000 7f4cfb5f3640 1 -- 172.21.15.76:0/619114131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ce0004800 msgr2=0x7f4ce0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.508+0000 7f4cfb5f3640 1 --2- 172.21.15.76:0/619114131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ce0004800 0x7f4ce0004c10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009a90 tx=0x7f4ce0005830 comp rx=0 tx=0).stop 2024-04-25T19:16:01.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.508+0000 7f4cf9b90640 1 -- 172.21.15.76:0/619114131 reap_dead start 2024-04-25T19:16:01.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.508+0000 7f4cfb5f3640 1 -- 172.21.15.76:0/619114131 shutdown_connections 2024-04-25T19:16:01.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.508+0000 7f4cfb5f3640 1 -- 172.21.15.76:0/619114131 >> 172.21.15.76:0/619114131 conn(0x7f4cf40cecc0 msgr2=0x7f4cf40cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:01.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.508+0000 7f4cfb5f3640 1 -- 172.21.15.76:0/619114131 shutdown_connections 2024-04-25T19:16:01.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.508+0000 7f4cfb5f3640 1 -- 172.21.15.76:0/619114131 wait complete. 2024-04-25T19:16:01.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.510+0000 7f4cfb5f3640 1 Processor -- start 2024-04-25T19:16:01.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.510+0000 7f4cfb5f3640 1 -- start start 2024-04-25T19:16:01.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.511+0000 7f4cfb5f3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4ce0004800 0x7f4cf415ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.511+0000 7f4cfb5f3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cf415d710 0x7f4cf415de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.511+0000 7f4cfb5f3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cf4157b30 0x7f4cf406c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.511+0000 7f4cfb5f3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf4154fc0 con 0x7f4cf415d710 2024-04-25T19:16:01.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.511+0000 7f4cfb5f3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf40749f0 con 0x7f4cf4157b30 2024-04-25T19:16:01.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.511+0000 7f4cfb5f3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f4cf41f41b0 con 0x7f4ce0004800 2024-04-25T19:16:01.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.511+0000 7f4cf938f640 1 -- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4ce0004800 msgr2=0x7f4cf415ca00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3301/0 2024-04-25T19:16:01.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.511+0000 7f4cf938f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4ce0004800 0x7f4cf415ca00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:01.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.511+0000 7f4cf8b8e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cf415d710 0x7f4cf415de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.511+0000 7f4cf8b8e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cf415d710 0x7f4cf415de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43594/0 (socket says 172.21.15.76:43594) 2024-04-25T19:16:01.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.511+0000 7f4cf8b8e640 1 -- 172.21.15.76:0/1303273090 learned_addr learned my addr 172.21.15.76:0/1303273090 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:01.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.511+0000 7f4cf9b90640 1 --2- 172.21.15.76:0/1303273090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cf4157b30 0x7f4cf406c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.511+0000 7f4cf8b8e640 1 -- 172.21.15.76:0/1303273090 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4ce0004800 msgr2=0x7f4cf415ca00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:16:01.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.511+0000 7f4cf8b8e640 1 --2- 172.21.15.76:0/1303273090 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4ce0004800 0x7f4cf415ca00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.511+0000 7f4cf8b8e640 1 -- 172.21.15.76:0/1303273090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cf4157b30 msgr2=0x7f4cf406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.511+0000 7f4cf8b8e640 1 --2- 172.21.15.76:0/1303273090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cf4157b30 0x7f4cf406c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.511+0000 7f4cf8b8e640 1 -- 172.21.15.76:0/1303273090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf415bc40 con 0x7f4cf415d710 2024-04-25T19:16:01.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.512+0000 7f4cf8b8e640 1 --2- 172.21.15.76:0/1303273090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cf415d710 0x7f4cf415de40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009bb0 tx=0x7f4ce4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:01.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.512+0000 7f4ce9ffb640 1 -- 172.21.15.76:0/1303273090 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4041020 con 0x7f4cf415d710 2024-04-25T19:16:01.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.512+0000 7f4ce9ffb640 1 -- 172.21.15.76:0/1303273090 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce403c470 con 0x7f4cf415d710 2024-04-25T19:16:01.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.512+0000 7f4cfb5f3640 1 -- 172.21.15.76:0/1303273090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf415bf20 con 0x7f4cf415d710 2024-04-25T19:16:01.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.512+0000 7f4ce9ffb640 1 -- 172.21.15.76:0/1303273090 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4036040 con 0x7f4cf415d710 2024-04-25T19:16:01.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.512+0000 7f4cfb5f3640 1 -- 172.21.15.76:0/1303273090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf415c4e0 con 0x7f4cf415d710 2024-04-25T19:16:01.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.513+0000 7f4cfb5f3640 1 -- 172.21.15.76:0/1303273090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cf4155140 con 0x7f4cf415d710 2024-04-25T19:16:01.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.514+0000 7f4ce9ffb640 1 -- 172.21.15.76:0/1303273090 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4ce4044020 con 0x7f4cf415d710 2024-04-25T19:16:01.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.514+0000 7f4ce9ffb640 1 --2- 172.21.15.76:0/1303273090 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4cd8074490 0x7f4cd80769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:01.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.514+0000 7f4ce9ffb640 1 -- 172.21.15.76:0/1303273090 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4ce40b5070 con 0x7f4cf415d710 2024-04-25T19:16:01.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.514+0000 7f4cf938f640 1 --2- 172.21.15.76:0/1303273090 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4cd8074490 0x7f4cd80769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:01.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.515+0000 7f4cf938f640 1 --2- 172.21.15.76:0/1303273090 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4cd8074490 0x7f4cd80769a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0004810 tx=0x7f4cf00078f0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:01.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.519+0000 7f4ce9ffb640 1 -- 172.21.15.76:0/1303273090 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4ce407f2d0 con 0x7f4cf415d710 2024-04-25T19:16:01.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.769+0000 7f4cfb5f3640 1 -- 172.21.15.76:0/1303273090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f4cf406a390 con 0x7f4cf415d710 2024-04-25T19:16:01.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.770+0000 7f4ce9ffb640 1 -- 172.21.15.76:0/1303273090 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7f4ce40835d0 con 0x7f4cf415d710 2024-04-25T19:16:01.771 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":60,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:16:01.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.772+0000 7f4cfb5f3640 1 -- 172.21.15.76:0/1303273090 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4cd8074490 msgr2=0x7f4cd80769a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.772+0000 7f4cfb5f3640 1 --2- 172.21.15.76:0/1303273090 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4cd8074490 0x7f4cd80769a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0004810 tx=0x7f4cf00078f0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.772+0000 7f4cfb5f3640 1 -- 172.21.15.76:0/1303273090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cf415d710 msgr2=0x7f4cf415de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:01.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.772+0000 7f4cfb5f3640 1 --2- 172.21.15.76:0/1303273090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cf415d710 0x7f4cf415de40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009bb0 tx=0x7f4ce4004730 comp rx=0 tx=0).stop 2024-04-25T19:16:01.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.772+0000 7f4cfb5f3640 1 -- 172.21.15.76:0/1303273090 shutdown_connections 2024-04-25T19:16:01.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.772+0000 7f4cfb5f3640 1 --2- 172.21.15.76:0/1303273090 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4cd8074490 0x7f4cd80769a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.772+0000 7f4cfb5f3640 1 --2- 172.21.15.76:0/1303273090 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4cf4157b30 0x7f4cf406c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.772+0000 7f4cfb5f3640 1 --2- 172.21.15.76:0/1303273090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cf415d710 0x7f4cf415de40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.772+0000 7f4cfb5f3640 1 --2- 172.21.15.76:0/1303273090 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4ce0004800 0x7f4cf415ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:01.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.772+0000 7f4cfb5f3640 1 -- 172.21.15.76:0/1303273090 >> 172.21.15.76:0/1303273090 conn(0x7f4cf40cecc0 msgr2=0x7f4cf40c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:01.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.772+0000 7f4cfb5f3640 1 -- 172.21.15.76:0/1303273090 shutdown_connections 2024-04-25T19:16:01.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:01.772+0000 7f4cfb5f3640 1 -- 172.21.15.76:0/1303273090 wait complete. 2024-04-25T19:16:02.203 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-04-25T19:16:04.785 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:16:04.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.902+0000 7f71e7851640 1 Processor -- start 2024-04-25T19:16:04.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.902+0000 7f71e7851640 1 -- start start 2024-04-25T19:16:04.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.903+0000 7f71e7851640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f71e0158060 0x7f71e00c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:04.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.903+0000 7f71e7851640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71e00cab60 0x7f71e015af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:04.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.903+0000 7f71e7851640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71e00c56a0 con 0x7f71e00ca750 2024-04-25T19:16:04.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.903+0000 7f71e7851640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f71e00c5960 con 0x7f71e0158060 2024-04-25T19:16:04.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.903+0000 7f71e7851640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f71e00c5c40 con 0x7f71e00cab60 2024-04-25T19:16:04.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.903+0000 7f71e5dee640 1 -- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71e00cab60 msgr2=0x7f71e015af80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3301/0 2024-04-25T19:16:04.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.903+0000 7f71e5dee640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71e00cab60 0x7f71e015af80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:04.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.904+0000 7f71e55ed640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f71e0158060 0x7f71e00c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:04.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.904+0000 7f71e55ed640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f71e0158060 0x7f71e00c4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:52844/0 (socket says 172.21.15.76:52844) 2024-04-25T19:16:04.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.904+0000 7f71e55ed640 1 -- 172.21.15.76:0/3503481642 learned_addr learned my addr 172.21.15.76:0/3503481642 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:04.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.904+0000 7f71e55ed640 1 -- 172.21.15.76:0/3503481642 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71e00cab60 msgr2=0x7f71e015af80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:16:04.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.904+0000 7f71e55ed640 1 --2- 172.21.15.76:0/3503481642 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71e00cab60 0x7f71e015af80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:04.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.904+0000 7f71e55ed640 1 -- 172.21.15.76:0/3503481642 >> v1:172.21.15.105:6789/0 conn(0x7f71e00ca750 legacy=0x7f71e00cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:04.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.904+0000 7f71e55ed640 1 -- 172.21.15.76:0/3503481642 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e00c5f20 con 0x7f71e0158060 2024-04-25T19:16:04.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.905+0000 7f71e55ed640 1 --2- 172.21.15.76:0/3503481642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71e0158060 0x7f71e00c4de0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009870 tx=0x7f71dc02a320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce4e64edc0521942 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:04.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.905+0000 7f71d7fff640 1 -- 172.21.15.76:0/3503481642 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc037070 con 0x7f71e0158060 2024-04-25T19:16:04.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.905+0000 7f71d7fff640 1 -- 172.21.15.76:0/3503481642 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71dc032640 con 0x7f71e0158060 2024-04-25T19:16:04.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.905+0000 7f71d7fff640 1 -- 172.21.15.76:0/3503481642 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc032a10 con 0x7f71e0158060 2024-04-25T19:16:04.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.906+0000 7f71e7851640 1 -- 172.21.15.76:0/3503481642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71e0158060 msgr2=0x7f71e00c4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:04.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.906+0000 7f71e7851640 1 --2- 172.21.15.76:0/3503481642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71e0158060 0x7f71e00c4de0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009870 tx=0x7f71dc02a320 comp rx=0 tx=0).stop 2024-04-25T19:16:04.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.906+0000 7f71e7851640 1 -- 172.21.15.76:0/3503481642 shutdown_connections 2024-04-25T19:16:04.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.906+0000 7f71e7851640 1 --2- 172.21.15.76:0/3503481642 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71e00cab60 0x7f71e015af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:04.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.906+0000 7f71e7851640 1 --2- 172.21.15.76:0/3503481642 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71e0158060 0x7f71e00c4de0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:04.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.906+0000 7f71e7851640 1 -- 172.21.15.76:0/3503481642 >> 172.21.15.76:0/3503481642 conn(0x7f71e0068b10 msgr2=0x7f71e0072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:04.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.906+0000 7f71e7851640 1 -- 172.21.15.76:0/3503481642 shutdown_connections 2024-04-25T19:16:04.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.906+0000 7f71e7851640 1 -- 172.21.15.76:0/3503481642 wait complete. 2024-04-25T19:16:04.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.908+0000 7f71e7851640 1 Processor -- start 2024-04-25T19:16:04.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.908+0000 7f71e7851640 1 -- start start 2024-04-25T19:16:04.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.909+0000 7f71e7851640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71e00ca750 0x7f71e00abc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:04.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.909+0000 7f71e7851640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71e00cab60 0x7f71e01f3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:04.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.909+0000 7f71e7851640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71e0158060 0x7f71e00c9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:04.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.909+0000 7f71e7851640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f71e00c5320 con 0x7f71e00cab60 2024-04-25T19:16:04.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.909+0000 7f71e7851640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f71e00c6200 con 0x7f71e0158060 2024-04-25T19:16:04.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.909+0000 7f71e7851640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f71e00ccef0 con 0x7f71e00ca750 2024-04-25T19:16:04.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.909+0000 7f71e55ed640 1 -- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71e00ca750 msgr2=0x7f71e00abc50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3301/0 2024-04-25T19:16:04.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.909+0000 7f71e55ed640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71e00ca750 0x7f71e00abc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:04.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.909+0000 7f71e4dec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71e00cab60 0x7f71e01f3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:04.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.909+0000 7f71e4dec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71e00cab60 0x7f71e01f3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43600/0 (socket says 172.21.15.76:43600) 2024-04-25T19:16:04.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.909+0000 7f71e4dec640 1 -- 172.21.15.76:0/1232041924 learned_addr learned my addr 172.21.15.76:0/1232041924 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:04.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.909+0000 7f71e5dee640 1 --2- 172.21.15.76:0/1232041924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71e0158060 0x7f71e00c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:04.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.909+0000 7f71e4dec640 1 -- 172.21.15.76:0/1232041924 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71e00ca750 msgr2=0x7f71e00abc50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:16:04.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.909+0000 7f71e4dec640 1 --2- 172.21.15.76:0/1232041924 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71e00ca750 0x7f71e00abc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:04.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.909+0000 7f71e4dec640 1 -- 172.21.15.76:0/1232041924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71e0158060 msgr2=0x7f71e00c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:04.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.909+0000 7f71e4dec640 1 --2- 172.21.15.76:0/1232041924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71e0158060 0x7f71e00c9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:04.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.909+0000 7f71e4dec640 1 -- 172.21.15.76:0/1232041924 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e01f4430 con 0x7f71e00cab60 2024-04-25T19:16:04.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.910+0000 7f71e4dec640 1 --2- 172.21.15.76:0/1232041924 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71e00cab60 0x7f71e01f3ef0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f71d000c370 tx=0x7f71d000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:04.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.910+0000 7f71d5ffb640 1 -- 172.21.15.76:0/1232041924 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0017070 con 0x7f71e00cab60 2024-04-25T19:16:04.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.910+0000 7f71d5ffb640 1 -- 172.21.15.76:0/1232041924 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71d00052d0 con 0x7f71e00cab60 2024-04-25T19:16:04.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.910+0000 7f71e7851640 1 -- 172.21.15.76:0/1232041924 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e01f0960 con 0x7f71e00cab60 2024-04-25T19:16:04.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.910+0000 7f71d5ffb640 1 -- 172.21.15.76:0/1232041924 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0003850 con 0x7f71e00cab60 2024-04-25T19:16:04.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.910+0000 7f71e7851640 1 -- 172.21.15.76:0/1232041924 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e01f0ee0 con 0x7f71e00cab60 2024-04-25T19:16:04.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.911+0000 7f71e7851640 1 -- 172.21.15.76:0/1232041924 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71e00ccb10 con 0x7f71e00cab60 2024-04-25T19:16:04.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.912+0000 7f71d5ffb640 1 -- 172.21.15.76:0/1232041924 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f71d001a070 con 0x7f71e00cab60 2024-04-25T19:16:04.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.912+0000 7f71d5ffb640 1 --2- 172.21.15.76:0/1232041924 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f71c4074490 0x7f71c40769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:04.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.912+0000 7f71d5ffb640 1 -- 172.21.15.76:0/1232041924 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f71d0094440 con 0x7f71e00cab60 2024-04-25T19:16:04.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.912+0000 7f71e55ed640 1 --2- 172.21.15.76:0/1232041924 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f71c4074490 0x7f71c40769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:04.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.913+0000 7f71e55ed640 1 --2- 172.21.15.76:0/1232041924 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f71c4074490 0x7f71c40769a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f71dc034000 tx=0x7f71dc02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:04.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:04.917+0000 7f71d5ffb640 1 -- 172.21.15.76:0/1232041924 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f71e00ccb10 con 0x7f71e00cab60 2024-04-25T19:16:05.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:05.160+0000 7f71e7851640 1 -- 172.21.15.76:0/1232041924 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f71e006a390 con 0x7f71e00cab60 2024-04-25T19:16:05.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:05.160+0000 7f71d5ffb640 1 -- 172.21.15.76:0/1232041924 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7f71d005e610 con 0x7f71e00cab60 2024-04-25T19:16:05.162 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":63,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:16:05.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:05.162+0000 7f71e7851640 1 -- 172.21.15.76:0/1232041924 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f71c4074490 msgr2=0x7f71c40769a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:05.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:05.162+0000 7f71e7851640 1 --2- 172.21.15.76:0/1232041924 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f71c4074490 0x7f71c40769a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f71dc034000 tx=0x7f71dc02e000 comp rx=0 tx=0).stop 2024-04-25T19:16:05.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:05.162+0000 7f71e7851640 1 -- 172.21.15.76:0/1232041924 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71e00cab60 msgr2=0x7f71e01f3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:05.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:05.162+0000 7f71e7851640 1 --2- 172.21.15.76:0/1232041924 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71e00cab60 0x7f71e01f3ef0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f71d000c370 tx=0x7f71d000c830 comp rx=0 tx=0).stop 2024-04-25T19:16:05.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:05.162+0000 7f71e7851640 1 -- 172.21.15.76:0/1232041924 shutdown_connections 2024-04-25T19:16:05.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:05.162+0000 7f71e7851640 1 --2- 172.21.15.76:0/1232041924 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f71c4074490 0x7f71c40769a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:05.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:05.162+0000 7f71e7851640 1 --2- 172.21.15.76:0/1232041924 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71e0158060 0x7f71e00c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:05.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:05.162+0000 7f71e7851640 1 --2- 172.21.15.76:0/1232041924 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71e00cab60 0x7f71e01f3ef0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:05.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:05.162+0000 7f71e7851640 1 --2- 172.21.15.76:0/1232041924 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f71e00ca750 0x7f71e00abc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:05.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:05.162+0000 7f71e7851640 1 -- 172.21.15.76:0/1232041924 >> 172.21.15.76:0/1232041924 conn(0x7f71e0068b10 msgr2=0x7f71e0153210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:05.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:05.162+0000 7f71e7851640 1 -- 172.21.15.76:0/1232041924 shutdown_connections 2024-04-25T19:16:05.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:05.162+0000 7f71e7851640 1 -- 172.21.15.76:0/1232041924 wait complete. 2024-04-25T19:16:08.175 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:16:08.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.294+0000 7f5457d96640 1 Processor -- start 2024-04-25T19:16:08.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.294+0000 7f5457d96640 1 -- start start 2024-04-25T19:16:08.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.296+0000 7f5457d96640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f54500c7b60 0x7f54500cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:08.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.296+0000 7f5457d96640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f54500ce2a0 0x7f545015cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:08.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.296+0000 7f5457d96640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54500c8020 con 0x7f54500c7750 2024-04-25T19:16:08.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.296+0000 7f5457d96640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f54501598a0 con 0x7f54500ce2a0 2024-04-25T19:16:08.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.296+0000 7f5457d96640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f5450159b40 con 0x7f54500c7b60 2024-04-25T19:16:08.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.296+0000 7f5456333640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f54500ce2a0 0x7f545015cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:08.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.296+0000 7f5456333640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f54500ce2a0 0x7f545015cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43602/0 (socket says 172.21.15.76:43602) 2024-04-25T19:16:08.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.296+0000 7f5456333640 1 -- 172.21.15.76:0/3943860020 learned_addr learned my addr 172.21.15.76:0/3943860020 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:08.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.296+0000 7f5455331640 1 --2- 172.21.15.76:0/3943860020 >> v2:172.21.15.105:3300/0 conn(0x7f54500c7b60 0x7f54500cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:08.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.296+0000 7f5456333640 1 -- 172.21.15.76:0/3943860020 >> v2:172.21.15.105:3300/0 conn(0x7f54500c7b60 msgr2=0x7f54500cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:08.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.296+0000 7f5456333640 1 --2- 172.21.15.76:0/3943860020 >> v2:172.21.15.105:3300/0 conn(0x7f54500c7b60 0x7f54500cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:08.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.296+0000 7f5456333640 1 -- 172.21.15.76:0/3943860020 >> v1:172.21.15.105:6789/0 conn(0x7f54500c7750 legacy=0x7f5450153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:08.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.296+0000 7f5456333640 1 -- 172.21.15.76:0/3943860020 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5450159e20 con 0x7f54500ce2a0 2024-04-25T19:16:08.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.297+0000 7f5456333640 1 --2- 172.21.15.76:0/3943860020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54500ce2a0 0x7f545015cd80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5450073d10 tx=0x7f544802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8512e3bba2011abe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:08.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.297+0000 7f5454b30640 1 -- 172.21.15.76:0/3943860020 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448037070 con 0x7f54500ce2a0 2024-04-25T19:16:08.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.297+0000 7f5454b30640 1 -- 172.21.15.76:0/3943860020 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5448032440 con 0x7f54500ce2a0 2024-04-25T19:16:08.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.297+0000 7f5454b30640 1 -- 172.21.15.76:0/3943860020 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448032810 con 0x7f54500ce2a0 2024-04-25T19:16:08.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.298+0000 7f5457d96640 1 -- 172.21.15.76:0/3943860020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54500ce2a0 msgr2=0x7f545015cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:08.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.298+0000 7f5457d96640 1 --2- 172.21.15.76:0/3943860020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54500ce2a0 0x7f545015cd80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5450073d10 tx=0x7f544802a270 comp rx=0 tx=0).stop 2024-04-25T19:16:08.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.298+0000 7f5457d96640 1 -- 172.21.15.76:0/3943860020 shutdown_connections 2024-04-25T19:16:08.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.298+0000 7f5457d96640 1 --2- 172.21.15.76:0/3943860020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54500ce2a0 0x7f545015cd80 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:08.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.298+0000 7f5457d96640 1 --2- 172.21.15.76:0/3943860020 >> v2:172.21.15.105:3300/0 conn(0x7f54500c7b60 0x7f54500cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:08.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.298+0000 7f5457d96640 1 -- 172.21.15.76:0/3943860020 >> 172.21.15.76:0/3943860020 conn(0x7f5450070800 msgr2=0x7f5450070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:08.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.298+0000 7f5457d96640 1 -- 172.21.15.76:0/3943860020 shutdown_connections 2024-04-25T19:16:08.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.298+0000 7f5457d96640 1 -- 172.21.15.76:0/3943860020 wait complete. 2024-04-25T19:16:08.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.299+0000 7f5457d96640 1 Processor -- start 2024-04-25T19:16:08.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.300+0000 7f5457d96640 1 -- start start 2024-04-25T19:16:08.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.301+0000 7f5457d96640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54500c7750 0x7f545014ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:08.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.301+0000 7f5457d96640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54500ce2a0 0x7f54501f0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:08.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.301+0000 7f5457d96640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5450073440 0x7f545014f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:08.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.301+0000 7f5457d96640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f54500c8520 con 0x7f5450073440 2024-04-25T19:16:08.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.301+0000 7f5457d96640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f545015a3e0 con 0x7f54500ce2a0 2024-04-25T19:16:08.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.301+0000 7f5457d96640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f54500c86a0 con 0x7f54500c7750 2024-04-25T19:16:08.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.301+0000 7f5456333640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5450073440 0x7f545014f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:08.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.301+0000 7f5456333640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5450073440 0x7f545014f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43606/0 (socket says 172.21.15.76:43606) 2024-04-25T19:16:08.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.301+0000 7f5456333640 1 -- 172.21.15.76:0/2178675919 learned_addr learned my addr 172.21.15.76:0/2178675919 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:08.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.301+0000 7f5455b32640 1 -- 172.21.15.76:0/2178675919 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54500c7750 msgr2=0x7f545014ff40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3301/0 2024-04-25T19:16:08.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.301+0000 7f5455b32640 1 --2- 172.21.15.76:0/2178675919 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54500c7750 0x7f545014ff40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:08.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.301+0000 7f5455331640 1 --2- 172.21.15.76:0/2178675919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54500ce2a0 0x7f54501f0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:08.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.301+0000 7f5456333640 1 -- 172.21.15.76:0/2178675919 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54500c7750 msgr2=0x7f545014ff40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:16:08.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.301+0000 7f5456333640 1 --2- 172.21.15.76:0/2178675919 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54500c7750 0x7f545014ff40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:08.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.301+0000 7f5456333640 1 -- 172.21.15.76:0/2178675919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54500ce2a0 msgr2=0x7f54501f0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:08.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.301+0000 7f5456333640 1 --2- 172.21.15.76:0/2178675919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54500ce2a0 0x7f54501f0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:08.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.301+0000 7f5456333640 1 -- 172.21.15.76:0/2178675919 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54501f1330 con 0x7f5450073440 2024-04-25T19:16:08.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.301+0000 7f5456333640 1 --2- 172.21.15.76:0/2178675919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5450073440 0x7f545014f1a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5448009960 tx=0x7f54480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:08.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.301+0000 7f54467fc640 1 -- 172.21.15.76:0/2178675919 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448041020 con 0x7f5450073440 2024-04-25T19:16:08.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.302+0000 7f54467fc640 1 -- 172.21.15.76:0/2178675919 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f544803d640 con 0x7f5450073440 2024-04-25T19:16:08.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.302+0000 7f5457d96640 1 -- 172.21.15.76:0/2178675919 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f545014c490 con 0x7f5450073440 2024-04-25T19:16:08.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.302+0000 7f54467fc640 1 -- 172.21.15.76:0/2178675919 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448037040 con 0x7f5450073440 2024-04-25T19:16:08.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.302+0000 7f5457d96640 1 -- 172.21.15.76:0/2178675919 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54501f46e0 con 0x7f5450073440 2024-04-25T19:16:08.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.303+0000 7f5457d96640 1 -- 172.21.15.76:0/2178675919 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f545014c970 con 0x7f5450073440 2024-04-25T19:16:08.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.303+0000 7f54467fc640 1 -- 172.21.15.76:0/2178675919 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5448044020 con 0x7f5450073440 2024-04-25T19:16:08.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.304+0000 7f54467fc640 1 --2- 172.21.15.76:0/2178675919 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5434074490 0x7f54340769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:08.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.304+0000 7f54467fc640 1 -- 172.21.15.76:0/2178675919 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f54480b5510 con 0x7f5450073440 2024-04-25T19:16:08.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.304+0000 7f5455b32640 1 --2- 172.21.15.76:0/2178675919 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5434074490 0x7f54340769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:08.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.305+0000 7f5455b32640 1 --2- 172.21.15.76:0/2178675919 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5434074490 0x7f54340769a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f54500cebe0 tx=0x7f544c007d70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:08.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.309+0000 7f54467fc640 1 -- 172.21.15.76:0/2178675919 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f544804b070 con 0x7f5450073440 2024-04-25T19:16:08.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.556+0000 7f5457d96640 1 -- 172.21.15.76:0/2178675919 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f54500c5980 con 0x7f5450073440 2024-04-25T19:16:08.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.557+0000 7f54467fc640 1 -- 172.21.15.76:0/2178675919 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7f544807fd40 con 0x7f5450073440 2024-04-25T19:16:08.558 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":66,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:16:08.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.559+0000 7f5457d96640 1 -- 172.21.15.76:0/2178675919 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5434074490 msgr2=0x7f54340769a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:08.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.559+0000 7f5457d96640 1 --2- 172.21.15.76:0/2178675919 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5434074490 0x7f54340769a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f54500cebe0 tx=0x7f544c007d70 comp rx=0 tx=0).stop 2024-04-25T19:16:08.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.559+0000 7f5457d96640 1 -- 172.21.15.76:0/2178675919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5450073440 msgr2=0x7f545014f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:08.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.559+0000 7f5457d96640 1 --2- 172.21.15.76:0/2178675919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5450073440 0x7f545014f1a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5448009960 tx=0x7f54480046c0 comp rx=0 tx=0).stop 2024-04-25T19:16:08.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.559+0000 7f5457d96640 1 -- 172.21.15.76:0/2178675919 shutdown_connections 2024-04-25T19:16:08.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.559+0000 7f5457d96640 1 --2- 172.21.15.76:0/2178675919 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5434074490 0x7f54340769a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:08.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.559+0000 7f5457d96640 1 --2- 172.21.15.76:0/2178675919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5450073440 0x7f545014f1a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:08.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.559+0000 7f5457d96640 1 --2- 172.21.15.76:0/2178675919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54500ce2a0 0x7f54501f0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:08.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.559+0000 7f5457d96640 1 --2- 172.21.15.76:0/2178675919 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54500c7750 0x7f545014ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:08.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.559+0000 7f5457d96640 1 -- 172.21.15.76:0/2178675919 >> 172.21.15.76:0/2178675919 conn(0x7f5450070800 msgr2=0x7f54500c5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:08.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.559+0000 7f5457d96640 1 -- 172.21.15.76:0/2178675919 shutdown_connections 2024-04-25T19:16:08.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:08.559+0000 7f5457d96640 1 -- 172.21.15.76:0/2178675919 wait complete. 2024-04-25T19:16:09.016 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-04-25T19:16:11.573 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:16:11.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.690+0000 7f473c2af640 1 Processor -- start 2024-04-25T19:16:11.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.690+0000 7f473c2af640 1 -- start start 2024-04-25T19:16:11.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.692+0000 7f473c2af640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f47340c96c0 0x7f473415cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:11.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.692+0000 7f473c2af640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4734074650 0x7f4734160a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:11.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.692+0000 7f473c2af640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f473406c5c0 0x7f473406c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:11.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.692+0000 7f473c2af640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f473415d1c0 con 0x7f47340c96c0 2024-04-25T19:16:11.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.692+0000 7f473c2af640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f473415d4a0 con 0x7f4734074650 2024-04-25T19:16:11.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.692+0000 7f473c2af640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f473415d780 con 0x7f473406c5c0 2024-04-25T19:16:11.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.692+0000 7f473a84c640 1 -- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f473406c5c0 msgr2=0x7f473406c9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3301/0 2024-04-25T19:16:11.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.692+0000 7f473a84c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f473406c5c0 0x7f473406c9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:11.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.692+0000 7f473a04b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f47340c96c0 0x7f473415cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:11.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.692+0000 7f473a04b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f47340c96c0 0x7f473415cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57500/0 (socket says 172.21.15.76:57500) 2024-04-25T19:16:11.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.692+0000 7f473a04b640 1 -- 172.21.15.76:0/463302702 learned_addr learned my addr 172.21.15.76:0/463302702 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:11.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.692+0000 7f473a04b640 1 -- 172.21.15.76:0/463302702 >> v2:172.21.15.76:3300/0 conn(0x7f47340c96c0 msgr2=0x7f473415cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T19:16:11.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.692+0000 7f473a04b640 1 -- 172.21.15.76:0/463302702 >> v2:172.21.15.76:3300/0 conn(0x7f47340c96c0 msgr2=0x7f473415cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:16:11.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.692+0000 7f473a04b640 1 --2- 172.21.15.76:0/463302702 >> v2:172.21.15.76:3300/0 conn(0x7f47340c96c0 0x7f473415cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:16:11.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.692+0000 7f473a04b640 1 --2- 172.21.15.76:0/463302702 >> v2:172.21.15.76:3300/0 conn(0x7f47340c96c0 0x7f473415cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:11.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.692+0000 7f473984a640 1 --2- 172.21.15.76:0/463302702 >> v2:172.21.15.105:3300/0 conn(0x7f4734074650 0x7f4734160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:11.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.692+0000 7f473984a640 1 -- 172.21.15.76:0/463302702 >> v2:172.21.15.105:3300/0 conn(0x7f4734074650 msgr2=0x7f4734160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T19:16:11.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.692+0000 7f473984a640 1 -- 172.21.15.76:0/463302702 >> v2:172.21.15.105:3300/0 conn(0x7f4734074650 msgr2=0x7f4734160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:16:11.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.692+0000 7f473984a640 1 --2- 172.21.15.76:0/463302702 >> v2:172.21.15.105:3300/0 conn(0x7f4734074650 0x7f4734160a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:16:11.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.692+0000 7f473984a640 1 --2- 172.21.15.76:0/463302702 >> v2:172.21.15.105:3300/0 conn(0x7f4734074650 0x7f4734160a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:11.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.893+0000 7f473a84c640 1 -- 172.21.15.76:0/463302702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f473406c5c0 msgr2=0x7f473406c9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3301/0 2024-04-25T19:16:11.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.893+0000 7f473a84c640 1 --2- 172.21.15.76:0/463302702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f473406c5c0 0x7f473406c9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T19:16:11.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.893+0000 7f473a04b640 1 --2- 172.21.15.76:0/463302702 >> v2:172.21.15.76:3300/0 conn(0x7f47340c96c0 0x7f473415cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:11.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.893+0000 7f473a04b640 1 -- 172.21.15.76:0/463302702 >> v2:172.21.15.76:3300/0 conn(0x7f47340c96c0 msgr2=0x7f473415cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T19:16:11.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.893+0000 7f473a04b640 1 -- 172.21.15.76:0/463302702 >> v2:172.21.15.76:3300/0 conn(0x7f47340c96c0 msgr2=0x7f473415cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:16:11.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.893+0000 7f473a04b640 1 --2- 172.21.15.76:0/463302702 >> v2:172.21.15.76:3300/0 conn(0x7f47340c96c0 0x7f473415cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:16:11.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.893+0000 7f473a04b640 1 --2- 172.21.15.76:0/463302702 >> v2:172.21.15.76:3300/0 conn(0x7f47340c96c0 0x7f473415cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T19:16:11.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.893+0000 7f473984a640 1 --2- 172.21.15.76:0/463302702 >> v2:172.21.15.105:3300/0 conn(0x7f4734074650 0x7f4734160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:11.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.893+0000 7f473984a640 1 -- 172.21.15.76:0/463302702 >> v2:172.21.15.105:3300/0 conn(0x7f4734074650 msgr2=0x7f4734160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T19:16:11.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.893+0000 7f473984a640 1 -- 172.21.15.76:0/463302702 >> v2:172.21.15.105:3300/0 conn(0x7f4734074650 msgr2=0x7f4734160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:16:11.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.893+0000 7f473984a640 1 --2- 172.21.15.76:0/463302702 >> v2:172.21.15.105:3300/0 conn(0x7f4734074650 0x7f4734160a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:16:11.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:11.893+0000 7f473984a640 1 --2- 172.21.15.76:0/463302702 >> v2:172.21.15.105:3300/0 conn(0x7f4734074650 0x7f4734160a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T19:16:12.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:12.294+0000 7f473a84c640 1 -- 172.21.15.76:0/463302702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f473406c5c0 msgr2=0x7f473406c9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3301/0 2024-04-25T19:16:12.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:12.294+0000 7f473a04b640 1 --2- 172.21.15.76:0/463302702 >> v2:172.21.15.76:3300/0 conn(0x7f47340c96c0 0x7f473415cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:12.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:12.294+0000 7f473a84c640 1 --2- 172.21.15.76:0/463302702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f473406c5c0 0x7f473406c9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T19:16:12.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:12.294+0000 7f473a04b640 1 -- 172.21.15.76:0/463302702 >> v2:172.21.15.76:3300/0 conn(0x7f47340c96c0 msgr2=0x7f473415cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T19:16:12.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:12.294+0000 7f473a04b640 1 -- 172.21.15.76:0/463302702 >> v2:172.21.15.76:3300/0 conn(0x7f47340c96c0 msgr2=0x7f473415cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:16:12.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:12.294+0000 7f473984a640 1 --2- 172.21.15.76:0/463302702 >> v2:172.21.15.105:3300/0 conn(0x7f4734074650 0x7f4734160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:12.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:12.294+0000 7f473a04b640 1 --2- 172.21.15.76:0/463302702 >> v2:172.21.15.76:3300/0 conn(0x7f47340c96c0 0x7f473415cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:16:12.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:12.294+0000 7f473a04b640 1 --2- 172.21.15.76:0/463302702 >> v2:172.21.15.76:3300/0 conn(0x7f47340c96c0 0x7f473415cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T19:16:12.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:12.294+0000 7f473984a640 1 -- 172.21.15.76:0/463302702 >> v2:172.21.15.105:3300/0 conn(0x7f4734074650 msgr2=0x7f4734160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T19:16:12.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:12.294+0000 7f473984a640 1 -- 172.21.15.76:0/463302702 >> v2:172.21.15.105:3300/0 conn(0x7f4734074650 msgr2=0x7f4734160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:16:12.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:12.294+0000 7f473984a640 1 --2- 172.21.15.76:0/463302702 >> v2:172.21.15.105:3300/0 conn(0x7f4734074650 0x7f4734160a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:16:12.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:12.294+0000 7f473984a640 1 --2- 172.21.15.76:0/463302702 >> v2:172.21.15.105:3300/0 conn(0x7f4734074650 0x7f4734160a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T19:16:13.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:13.095+0000 7f473a84c640 1 -- 172.21.15.76:0/463302702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f473406c5c0 msgr2=0x7f473406c9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3301/0 2024-04-25T19:16:13.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:13.095+0000 7f473a84c640 1 --2- 172.21.15.76:0/463302702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f473406c5c0 0x7f473406c9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T19:16:13.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:13.095+0000 7f473a04b640 1 --2- 172.21.15.76:0/463302702 >> v2:172.21.15.76:3300/0 conn(0x7f47340c96c0 0x7f473415cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:13.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:13.095+0000 7f473984a640 1 --2- 172.21.15.76:0/463302702 >> v2:172.21.15.105:3300/0 conn(0x7f4734074650 0x7f4734160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:13.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:13.095+0000 7f473a04b640 1 -- 172.21.15.76:0/463302702 >> v2:172.21.15.76:3300/0 conn(0x7f47340c96c0 msgr2=0x7f473415cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T19:16:13.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:13.095+0000 7f473a04b640 1 -- 172.21.15.76:0/463302702 >> v2:172.21.15.76:3300/0 conn(0x7f47340c96c0 msgr2=0x7f473415cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:16:13.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:13.095+0000 7f473a04b640 1 --2- 172.21.15.76:0/463302702 >> v2:172.21.15.76:3300/0 conn(0x7f47340c96c0 0x7f473415cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:16:13.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:13.095+0000 7f473a04b640 1 --2- 172.21.15.76:0/463302702 >> v2:172.21.15.76:3300/0 conn(0x7f47340c96c0 0x7f473415cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T19:16:13.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:13.095+0000 7f473984a640 1 -- 172.21.15.76:0/463302702 >> v2:172.21.15.105:3300/0 conn(0x7f4734074650 msgr2=0x7f4734160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T19:16:13.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:13.095+0000 7f473984a640 1 -- 172.21.15.76:0/463302702 >> v2:172.21.15.105:3300/0 conn(0x7f4734074650 msgr2=0x7f4734160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:16:13.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:13.095+0000 7f473984a640 1 --2- 172.21.15.76:0/463302702 >> v2:172.21.15.105:3300/0 conn(0x7f4734074650 0x7f4734160a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:16:13.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:13.095+0000 7f473984a640 1 --2- 172.21.15.76:0/463302702 >> v2:172.21.15.105:3300/0 conn(0x7f4734074650 0x7f4734160a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T19:16:14.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:14.692+0000 7f472bfff640 1 -- 172.21.15.76:0/463302702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f473406c5c0 msgr2=0x7f473406c9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:16:14.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:14.692+0000 7f472bfff640 1 --2- 172.21.15.76:0/463302702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f473406c5c0 0x7f473406c9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:14.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:14.692+0000 7f472bfff640 1 -- 172.21.15.76:0/463302702 >> v2:172.21.15.105:3300/0 conn(0x7f4734074650 msgr2=0x7f4734160a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:16:14.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:14.692+0000 7f472bfff640 1 --2- 172.21.15.76:0/463302702 >> v2:172.21.15.105:3300/0 conn(0x7f4734074650 0x7f4734160a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:14.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:14.692+0000 7f472bfff640 1 -- 172.21.15.76:0/463302702 >> v2:172.21.15.76:3300/0 conn(0x7f47340c96c0 msgr2=0x7f473415cb80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:16:14.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:14.692+0000 7f472bfff640 1 --2- 172.21.15.76:0/463302702 >> v2:172.21.15.76:3300/0 conn(0x7f47340c96c0 0x7f473415cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:14.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:14.692+0000 7f472bfff640 1 -- 172.21.15.76:0/463302702 --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4734160210 con 0x7f4718007400 2024-04-25T19:16:14.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:14.692+0000 7f472bfff640 1 -- 172.21.15.76:0/463302702 --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f471800c1c0 con 0x7f4718003690 2024-04-25T19:16:15.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.547+0000 7f4739049640 1 -- 172.21.15.76:0/463302702 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2669023291 0 0) 0x7f4734160210 con 0x7f4718007400 2024-04-25T19:16:15.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.547+0000 7f4739049640 1 -- 172.21.15.76:0/463302702 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4720003520 con 0x7f4718007400 2024-04-25T19:16:15.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.548+0000 7f4739049640 1 -- 172.21.15.76:0/463302702 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3903841561 0 0) 0x7f471800c1c0 con 0x7f4718003690 2024-04-25T19:16:15.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.548+0000 7f4739049640 1 -- 172.21.15.76:0/463302702 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4734160210 con 0x7f4718003690 2024-04-25T19:16:15.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.548+0000 7f4739049640 1 -- 172.21.15.76:0/463302702 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1388914505 0 0) 0x7f4720003520 con 0x7f4718007400 2024-04-25T19:16:15.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.548+0000 7f4739049640 1 -- 172.21.15.76:0/463302702 >> v1:172.21.15.105:6789/0 conn(0x7f4718003690 legacy=0x7f4718005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:15.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.548+0000 7f4739049640 1 -- 172.21.15.76:0/463302702 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f471800d4c0 con 0x7f4718007400 2024-04-25T19:16:15.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.548+0000 7f4739049640 1 -- 172.21.15.76:0/463302702 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f472400d580 con 0x7f4718007400 2024-04-25T19:16:15.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.548+0000 7f4739049640 1 -- 172.21.15.76:0/463302702 >> v1:172.21.15.76:6789/0 conn(0x7f4718007400 legacy=0x7f4718009910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:15.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.548+0000 7f473a84c640 1 -- 172.21.15.76:0/463302702 reap_dead start 2024-04-25T19:16:15.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.549+0000 7f4739049640 1 --2- 172.21.15.76:0/463302702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4720003b40 0x7f4720003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:15.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.549+0000 7f4739049640 1 --2- 172.21.15.76:0/463302702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4720004800 0x7f4720004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:15.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.549+0000 7f4739049640 1 --2- 172.21.15.76:0/463302702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47200053a0 0x7f472000ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:15.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.549+0000 7f4739049640 1 -- 172.21.15.76:0/463302702 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f472000c300 con 0x7f4720003b40 2024-04-25T19:16:15.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.549+0000 7f4739049640 1 -- 172.21.15.76:0/463302702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f472000c4b0 con 0x7f47200053a0 2024-04-25T19:16:15.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.549+0000 7f4739049640 1 -- 172.21.15.76:0/463302702 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f472000c6b0 con 0x7f4720004800 2024-04-25T19:16:15.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.549+0000 7f473984a640 1 -- 172.21.15.76:0/463302702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4720004800 msgr2=0x7f4720004c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3301/0 2024-04-25T19:16:15.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.549+0000 7f473984a640 1 --2- 172.21.15.76:0/463302702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4720004800 0x7f4720004c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:15.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.549+0000 7f473a04b640 1 --2- 172.21.15.76:0/463302702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4720003b40 0x7f4720003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:15.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.549+0000 7f473a84c640 1 --2- 172.21.15.76:0/463302702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47200053a0 0x7f472000ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:15.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.549+0000 7f473a04b640 1 -- 172.21.15.76:0/463302702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4720004800 msgr2=0x7f4720004c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:16:15.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.549+0000 7f473a04b640 1 --2- 172.21.15.76:0/463302702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4720004800 0x7f4720004c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:15.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.549+0000 7f473a04b640 1 -- 172.21.15.76:0/463302702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47200053a0 msgr2=0x7f472000ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:15.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.549+0000 7f473a04b640 1 --2- 172.21.15.76:0/463302702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47200053a0 0x7f472000ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:15.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.549+0000 7f473a04b640 1 -- 172.21.15.76:0/463302702 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4720003520 con 0x7f4720003b40 2024-04-25T19:16:15.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.550+0000 7f473a04b640 1 --2- 172.21.15.76:0/463302702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4720003b40 0x7f4720003f70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f47300096d0 tx=0x7f4720005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3df5cbd4b237a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:15.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.551+0000 7f4739049640 1 -- 172.21.15.76:0/463302702 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4730031070 con 0x7f4720003b40 2024-04-25T19:16:15.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.551+0000 7f4739049640 1 -- 172.21.15.76:0/463302702 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f473002b7e0 con 0x7f4720003b40 2024-04-25T19:16:15.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.551+0000 7f4739049640 1 -- 172.21.15.76:0/463302702 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4730009880 con 0x7f4720003b40 2024-04-25T19:16:15.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.552+0000 7f473c2af640 1 -- 172.21.15.76:0/463302702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4720003b40 msgr2=0x7f4720003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:15.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.552+0000 7f473c2af640 1 --2- 172.21.15.76:0/463302702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4720003b40 0x7f4720003f70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f47300096d0 tx=0x7f4720005830 comp rx=0 tx=0).stop 2024-04-25T19:16:15.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.553+0000 7f473c2af640 1 -- 172.21.15.76:0/463302702 shutdown_connections 2024-04-25T19:16:15.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.553+0000 7f473c2af640 1 --2- 172.21.15.76:0/463302702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f47200053a0 0x7f472000ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:15.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.553+0000 7f473c2af640 1 --2- 172.21.15.76:0/463302702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4720004800 0x7f4720004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:15.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.553+0000 7f473c2af640 1 --2- 172.21.15.76:0/463302702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4720003b40 0x7f4720003f70 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:15.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.553+0000 7f473c2af640 1 -- 172.21.15.76:0/463302702 >> 172.21.15.76:0/463302702 conn(0x7f47340cecc0 msgr2=0x7f47340cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:15.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.553+0000 7f473c2af640 1 -- 172.21.15.76:0/463302702 shutdown_connections 2024-04-25T19:16:15.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.553+0000 7f473c2af640 1 -- 172.21.15.76:0/463302702 wait complete. 2024-04-25T19:16:15.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.555+0000 7f473c2af640 1 Processor -- start 2024-04-25T19:16:15.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.555+0000 7f473c2af640 1 -- start start 2024-04-25T19:16:15.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.557+0000 7f473c2af640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4720003b40 0x7f47340ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:15.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.557+0000 7f473c2af640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4720004800 0x7f4734159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:15.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.557+0000 7f473c2af640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47200053a0 0x7f4734159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:15.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.557+0000 7f473c2af640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f47340ca060 con 0x7f4720003b40 2024-04-25T19:16:15.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.557+0000 7f473c2af640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f47340c7330 con 0x7f4720004800 2024-04-25T19:16:15.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.557+0000 7f473c2af640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f473415da60 con 0x7f47200053a0 2024-04-25T19:16:15.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.557+0000 7f473a84c640 1 -- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47200053a0 msgr2=0x7f4734159780 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3301/0 2024-04-25T19:16:15.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.557+0000 7f473a04b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4720003b40 0x7f47340ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:15.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.557+0000 7f473a84c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47200053a0 0x7f4734159780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:15.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.557+0000 7f473a04b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4720003b40 0x7f47340ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57552/0 (socket says 172.21.15.76:57552) 2024-04-25T19:16:15.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.557+0000 7f473a04b640 1 -- 172.21.15.76:0/1852175113 learned_addr learned my addr 172.21.15.76:0/1852175113 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:15.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.557+0000 7f473984a640 1 --2- 172.21.15.76:0/1852175113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4720004800 0x7f4734159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:15.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.557+0000 7f473a04b640 1 -- 172.21.15.76:0/1852175113 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47200053a0 msgr2=0x7f4734159780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:16:15.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.557+0000 7f473a04b640 1 --2- 172.21.15.76:0/1852175113 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47200053a0 0x7f4734159780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:15.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.557+0000 7f473a04b640 1 -- 172.21.15.76:0/1852175113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4720004800 msgr2=0x7f4734159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:15.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.557+0000 7f473a04b640 1 --2- 172.21.15.76:0/1852175113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4720004800 0x7f4734159240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:15.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.557+0000 7f473a04b640 1 -- 172.21.15.76:0/1852175113 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47340cae80 con 0x7f4720003b40 2024-04-25T19:16:15.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.557+0000 7f473a04b640 1 --2- 172.21.15.76:0/1852175113 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4720003b40 0x7f47340ccca0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f473002b4d0 tx=0x7f4730004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:15.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.558+0000 7f472affd640 1 -- 172.21.15.76:0/1852175113 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47300130b0 con 0x7f4720003b40 2024-04-25T19:16:15.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.558+0000 7f472affd640 1 -- 172.21.15.76:0/1852175113 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4730009860 con 0x7f4720003b40 2024-04-25T19:16:15.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.558+0000 7f473c2af640 1 -- 172.21.15.76:0/1852175113 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47341f34f0 con 0x7f4720003b40 2024-04-25T19:16:15.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.558+0000 7f472affd640 1 -- 172.21.15.76:0/1852175113 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473003f420 con 0x7f4720003b40 2024-04-25T19:16:15.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.558+0000 7f473c2af640 1 -- 172.21.15.76:0/1852175113 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f473415a4a0 con 0x7f4720003b40 2024-04-25T19:16:15.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.559+0000 7f473c2af640 1 -- 172.21.15.76:0/1852175113 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f473406a390 con 0x7f4720003b40 2024-04-25T19:16:15.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.559+0000 7f472affd640 1 -- 172.21.15.76:0/1852175113 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4730046020 con 0x7f4720003b40 2024-04-25T19:16:15.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.560+0000 7f472affd640 1 --2- 172.21.15.76:0/1852175113 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4718074d90 0x7f47180772a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:15.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.560+0000 7f472affd640 1 -- 172.21.15.76:0/1852175113 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f47300b5b30 con 0x7f4720003b40 2024-04-25T19:16:15.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.560+0000 7f473984a640 1 --2- 172.21.15.76:0/1852175113 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4718074d90 0x7f47180772a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:15.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.561+0000 7f473984a640 1 --2- 172.21.15.76:0/1852175113 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4718074d90 0x7f47180772a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f47340cae10 tx=0x7f4724008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:15.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.565+0000 7f472affd640 1 -- 172.21.15.76:0/1852175113 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f473004d020 con 0x7f4720003b40 2024-04-25T19:16:15.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.810+0000 7f473c2af640 1 -- 172.21.15.76:0/1852175113 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f4734074430 con 0x7f4720003b40 2024-04-25T19:16:15.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.811+0000 7f472affd640 1 -- 172.21.15.76:0/1852175113 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f47300802c0 con 0x7f4720003b40 2024-04-25T19:16:15.813 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":8,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:16:15.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.813+0000 7f473c2af640 1 -- 172.21.15.76:0/1852175113 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4718074d90 msgr2=0x7f47180772a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:15.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.813+0000 7f473c2af640 1 --2- 172.21.15.76:0/1852175113 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4718074d90 0x7f47180772a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f47340cae10 tx=0x7f4724008040 comp rx=0 tx=0).stop 2024-04-25T19:16:15.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.813+0000 7f473c2af640 1 -- 172.21.15.76:0/1852175113 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4720003b40 msgr2=0x7f47340ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:15.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.813+0000 7f473c2af640 1 --2- 172.21.15.76:0/1852175113 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4720003b40 0x7f47340ccca0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f473002b4d0 tx=0x7f4730004ab0 comp rx=0 tx=0).stop 2024-04-25T19:16:15.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.813+0000 7f473c2af640 1 -- 172.21.15.76:0/1852175113 shutdown_connections 2024-04-25T19:16:15.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.813+0000 7f473c2af640 1 --2- 172.21.15.76:0/1852175113 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4718074d90 0x7f47180772a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:15.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.813+0000 7f473c2af640 1 --2- 172.21.15.76:0/1852175113 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f47200053a0 0x7f4734159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:15.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.813+0000 7f473c2af640 1 --2- 172.21.15.76:0/1852175113 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4720004800 0x7f4734159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:15.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.813+0000 7f473c2af640 1 --2- 172.21.15.76:0/1852175113 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4720003b40 0x7f47340ccca0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:15.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.813+0000 7f473c2af640 1 -- 172.21.15.76:0/1852175113 >> 172.21.15.76:0/1852175113 conn(0x7f47340cecc0 msgr2=0x7f47340cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:15.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.813+0000 7f473c2af640 1 -- 172.21.15.76:0/1852175113 shutdown_connections 2024-04-25T19:16:15.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.813+0000 7f473c2af640 1 -- 172.21.15.76:0/1852175113 wait complete. 2024-04-25T19:16:15.826 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-04-25T19:16:15.826 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-04-25T19:16:15.826 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T19:16:15.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.933+0000 7fadc5da8640 1 Processor -- start 2024-04-25T19:16:15.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.933+0000 7fadc5da8640 1 -- start start 2024-04-25T19:16:15.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.935+0000 7fadc5da8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fadc00cc9e0 0x7fadc006d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:15.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.935+0000 7fadc5da8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fadc0156f10 0x7fadc015d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:15.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.935+0000 7fadc5da8640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc006e310 con 0x7fadc006de20 2024-04-25T19:16:15.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.935+0000 7fadc5da8640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fadc015d2f0 con 0x7fadc0156f10 2024-04-25T19:16:15.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.935+0000 7fadc5da8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fadc015dcc0 con 0x7fadc00cc9e0 2024-04-25T19:16:15.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.935+0000 7fadbffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fadc0156f10 0x7fadc015d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:15.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.935+0000 7fadbf7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fadc006de20 0x7fadc00cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36898/0 (socket says 172.21.15.76:36898) 2024-04-25T19:16:15.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.935+0000 7fadbf7fe640 1 -- 172.21.15.76:0/2180195857 learned_addr learned my addr 172.21.15.76:0/2180195857 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:16:15.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.935+0000 7fadbeffd640 1 -- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fadc00cc9e0 msgr2=0x7fadc006d6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3301/0 2024-04-25T19:16:15.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.935+0000 7fadbeffd640 1 --2- 172.21.15.76:0/2180195857 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fadc00cc9e0 0x7fadc006d6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:15.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.935+0000 7fadbffff640 1 -- 172.21.15.76:0/2180195857 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fadc00cc9e0 msgr2=0x7fadc006d6d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:16:15.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.935+0000 7fadbffff640 1 --2- 172.21.15.76:0/2180195857 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fadc00cc9e0 0x7fadc006d6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:15.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.935+0000 7fadbffff640 1 -- 172.21.15.76:0/2180195857 >> v1:172.21.15.76:6789/0 conn(0x7fadc006de20 legacy=0x7fadc00cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:15.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.935+0000 7fadbffff640 1 -- 172.21.15.76:0/2180195857 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc015ccc0 con 0x7fadc0156f10 2024-04-25T19:16:15.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.935+0000 7fadbe7fc640 1 -- 172.21.15.76:0/2180195857 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1664864711 0 0) 0x7fadc006e310 con 0x7fadc006de20 2024-04-25T19:16:15.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.936+0000 7fadbffff640 1 --2- 172.21.15.76:0/2180195857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadc0156f10 0x7fadc015d780 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009d90 tx=0x7fadb802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666515335312eb04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:15.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.936+0000 7fadbe7fc640 1 -- 172.21.15.76:0/2180195857 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8007740 con 0x7fadc0156f10 2024-04-25T19:16:15.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.936+0000 7fadbe7fc640 1 -- 172.21.15.76:0/2180195857 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb80078e0 con 0x7fadc0156f10 2024-04-25T19:16:15.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.936+0000 7fadbe7fc640 1 -- 172.21.15.76:0/2180195857 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8007cb0 con 0x7fadc0156f10 2024-04-25T19:16:15.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.937+0000 7fadc5da8640 1 -- 172.21.15.76:0/2180195857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadc0156f10 msgr2=0x7fadc015d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:15.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.937+0000 7fadc5da8640 1 --2- 172.21.15.76:0/2180195857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadc0156f10 0x7fadc015d780 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009d90 tx=0x7fadb802a5f0 comp rx=0 tx=0).stop 2024-04-25T19:16:15.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.937+0000 7fadc5da8640 1 -- 172.21.15.76:0/2180195857 shutdown_connections 2024-04-25T19:16:15.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.937+0000 7fadc5da8640 1 --2- 172.21.15.76:0/2180195857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadc0156f10 0x7fadc015d780 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:15.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.937+0000 7fadc5da8640 1 --2- 172.21.15.76:0/2180195857 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fadc00cc9e0 0x7fadc006d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:15.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.937+0000 7fadc5da8640 1 -- 172.21.15.76:0/2180195857 >> 172.21.15.76:0/2180195857 conn(0x7fadc00684e0 msgr2=0x7fadc00688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:15.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.937+0000 7fadc5da8640 1 -- 172.21.15.76:0/2180195857 shutdown_connections 2024-04-25T19:16:15.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.937+0000 7fadc5da8640 1 -- 172.21.15.76:0/2180195857 wait complete. 2024-04-25T19:16:15.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.938+0000 7fadc5da8640 1 Processor -- start 2024-04-25T19:16:15.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.939+0000 7fadc5da8640 1 -- start start 2024-04-25T19:16:15.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.940+0000 7fadc5da8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadc006de20 0x7fadc0157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:15.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.940+0000 7fadc5da8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fadc00cc9e0 0x7fadc00cf070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:15.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.940+0000 7fadc5da8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadc0156f10 0x7fadc014c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:15.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.940+0000 7fadc5da8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fadc015cf80 con 0x7fadc006de20 2024-04-25T19:16:15.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.940+0000 7fadc5da8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fadc00cb980 con 0x7fadc0156f10 2024-04-25T19:16:15.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.940+0000 7fadc5da8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fadc0157320 con 0x7fadc00cc9e0 2024-04-25T19:16:15.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.940+0000 7fadbf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadc006de20 0x7fadc0157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:15.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.940+0000 7fadbf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadc006de20 0x7fadc0157550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57578/0 (socket says 172.21.15.76:57578) 2024-04-25T19:16:15.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.940+0000 7fadbf7fe640 1 -- 172.21.15.76:0/1819964254 learned_addr learned my addr 172.21.15.76:0/1819964254 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:15.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.940+0000 7fadbeffd640 1 -- 172.21.15.76:0/1819964254 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fadc00cc9e0 msgr2=0x7fadc00cf070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3301/0 2024-04-25T19:16:15.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.940+0000 7fadbeffd640 1 --2- 172.21.15.76:0/1819964254 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fadc00cc9e0 0x7fadc00cf070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:15.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.940+0000 7fadbf7fe640 1 -- 172.21.15.76:0/1819964254 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fadc00cc9e0 msgr2=0x7fadc00cf070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:16:15.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.940+0000 7fadbf7fe640 1 --2- 172.21.15.76:0/1819964254 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fadc00cc9e0 0x7fadc00cf070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:15.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.940+0000 7fadbf7fe640 1 -- 172.21.15.76:0/1819964254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadc0156f10 msgr2=0x7fadc014c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:15.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.940+0000 7fadbf7fe640 1 --2- 172.21.15.76:0/1819964254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadc0156f10 0x7fadc014c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:15.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.940+0000 7fadbf7fe640 1 -- 172.21.15.76:0/1819964254 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc0158150 con 0x7fadc006de20 2024-04-25T19:16:15.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.940+0000 7fadbf7fe640 1 --2- 172.21.15.76:0/1819964254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadc006de20 0x7fadc0157550 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fadb000c430 tx=0x7fadb000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:15.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.941+0000 7fada7fff640 1 -- 172.21.15.76:0/1819964254 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb00074e0 con 0x7fadc006de20 2024-04-25T19:16:15.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.941+0000 7fada7fff640 1 -- 172.21.15.76:0/1819964254 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb00047e0 con 0x7fadc006de20 2024-04-25T19:16:15.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.941+0000 7fada7fff640 1 -- 172.21.15.76:0/1819964254 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb0004bd0 con 0x7fadc006de20 2024-04-25T19:16:15.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.941+0000 7fadc5da8640 1 -- 172.21.15.76:0/1819964254 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc01f2e20 con 0x7fadc006de20 2024-04-25T19:16:15.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.941+0000 7fadc5da8640 1 -- 172.21.15.76:0/1819964254 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc014dbf0 con 0x7fadc006de20 2024-04-25T19:16:15.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.942+0000 7fadc5da8640 1 --2- 172.21.15.76:0/1819964254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadc01f3900 0x7fadc014c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:15.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.942+0000 7fadc5da8640 1 -- 172.21.15.76:0/1819964254 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fadc014e170 con 0x7fadc01f3900 2024-04-25T19:16:15.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.942+0000 7fadbeffd640 1 --2- 172.21.15.76:0/1819964254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadc01f3900 0x7fadc014c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:15.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.942+0000 7fadbeffd640 1 -- 172.21.15.76:0/1819964254 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fadc0158430 con 0x7fadc01f3900 2024-04-25T19:16:15.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.942+0000 7fadbeffd640 1 --2- 172.21.15.76:0/1819964254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadc01f3900 0x7fadc014c860 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fadc00cf970 tx=0x7fadb400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:15.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.942+0000 7fada7fff640 1 -- 172.21.15.76:0/1819964254 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fadb0011070 con 0x7fadc006de20 2024-04-25T19:16:15.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.943+0000 7fada7fff640 1 --2- 172.21.15.76:0/1819964254 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fadac074490 0x7fadac0769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:15.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.943+0000 7fada7fff640 1 -- 172.21.15.76:0/1819964254 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fadb0094de0 con 0x7fadc006de20 2024-04-25T19:16:15.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.943+0000 7fada7fff640 1 -- 172.21.15.76:0/1819964254 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb4003e10 con 0x7fadc01f3900 2024-04-25T19:16:15.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.943+0000 7fada7fff640 1 -- 172.21.15.76:0/1819964254 <== mon.0 v2:172.21.15.76:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fadb4015070 con 0x7fadc01f3900 2024-04-25T19:16:15.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.943+0000 7fada7fff640 1 -- 172.21.15.76:0/1819964254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadc01f3900 msgr2=0x7fadc014c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:15.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.943+0000 7fada7fff640 1 --2- 172.21.15.76:0/1819964254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadc01f3900 0x7fadc014c860 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fadc00cf970 tx=0x7fadb400b8c0 comp rx=0 tx=0).stop 2024-04-25T19:16:15.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.943+0000 7fada7fff640 1 -- 172.21.15.76:0/1819964254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadc01f3900 msgr2=0x7fadc014c860 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:16:15.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.943+0000 7fada7fff640 1 --2- 172.21.15.76:0/1819964254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadc01f3900 0x7fadc014c860 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:15.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.943+0000 7fadbffff640 1 --2- 172.21.15.76:0/1819964254 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fadac074490 0x7fadac0769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:15.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.944+0000 7fadbffff640 1 --2- 172.21.15.76:0/1819964254 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fadac074490 0x7fadac0769a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fadb8034000 tx=0x7fadb802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:15.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.950+0000 7fadc5da8640 1 --2- 172.21.15.76:0/1819964254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadc0154980 0x7fadc01f61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:15.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.950+0000 7fadc5da8640 1 -- 172.21.15.76:0/1819964254 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fadc0157f00 con 0x7fadc0154980 2024-04-25T19:16:15.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.950+0000 7fadbeffd640 1 --2- 172.21.15.76:0/1819964254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadc0154980 0x7fadc01f61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:15.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.950+0000 7fadbeffd640 1 -- 172.21.15.76:0/1819964254 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fadc0069a50 con 0x7fadc0154980 2024-04-25T19:16:15.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.950+0000 7fadbeffd640 1 --2- 172.21.15.76:0/1819964254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadc0154980 0x7fadc01f61f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fadb4014000 tx=0x7fadb4005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:15.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.950+0000 7fada7fff640 1 -- 172.21.15.76:0/1819964254 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb4005820 con 0x7fadc0154980 2024-04-25T19:16:15.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.950+0000 7fada7fff640 1 -- 172.21.15.76:0/1819964254 <== mon.0 v2:172.21.15.76:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5054 (secure 0 0 0) 0x7fadb4005820 con 0x7fadc0154980 2024-04-25T19:16:15.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.950+0000 7fada7fff640 1 -- 172.21.15.76:0/1819964254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadc0154980 msgr2=0x7fadc01f61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:15.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.950+0000 7fada7fff640 1 --2- 172.21.15.76:0/1819964254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadc0154980 0x7fadc01f61f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fadb4014000 tx=0x7fadb4005a20 comp rx=0 tx=0).stop 2024-04-25T19:16:15.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.950+0000 7fada7fff640 1 -- 172.21.15.76:0/1819964254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadc0154980 msgr2=0x7fadc01f61f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:16:15.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.950+0000 7fada7fff640 1 --2- 172.21.15.76:0/1819964254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadc0154980 0x7fadc01f61f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:15.953 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:16:15.953 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:16:15.953 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:16:15.953 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "leader", 2024-04-25T19:16:15.953 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 8, 2024-04-25T19:16:15.953 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 74475, 2024-04-25T19:16:15.954 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:16:15.954 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:16:15.954 INFO:teuthology.orchestra.run.smithi076.stdout: 1 2024-04-25T19:16:15.954 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:15.954 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 0, 2024-04-25T19:16:15.954 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:16:15.954 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:16:15.954 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:16:15.954 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:15.954 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:15.954 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:15.954 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:15.954 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:15.955 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:15.955 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:15.955 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:15.955 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:15.955 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:15.955 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:15.955 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:15.955 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:16:15.955 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:16:15.955 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:15.955 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:15.955 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:15.955 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:15.955 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:15.956 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:15.956 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:15.956 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:15.956 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:15.956 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:15.956 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:15.956 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:15.956 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:15.956 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:16:15.956 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:16:15.956 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:16:15.956 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:16:15.956 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:16:15.956 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:16:15.957 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:16:15.957 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:16:15.957 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:16:15.957 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:16:15.957 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:16:15.957 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:16:15.957 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:16:15.957 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:16:15.957 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:16:15.957 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:16:15.957 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:16:15.957 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:15.957 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:15.957 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:15.957 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:15.958 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:15.958 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:15.958 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:15.958 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:15.958 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:15.958 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:15.958 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:15.958 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:15.958 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:16:15.958 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:15.958 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:16:15.958 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:15.959 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:16:15.959 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:16:15.959 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:15.959 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:15.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:15.959 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:15.959 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:16:15.959 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:15.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:15.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:15.959 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:15.959 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:16:15.959 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:15.959 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:15.959 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:15.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:15.960 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:16:15.960 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:16:15.960 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:15.960 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:15.960 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:15.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:15.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:15.960 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:16:15.960 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:16:15.960 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:15.960 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:15.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:15.961 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:15.961 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:16:15.961 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:15.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:15.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:15.961 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:15.961 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:16:15.961 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:15.961 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:15.961 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:15.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:15.961 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:16:15.961 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:16:15.961 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:15.962 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:15.962 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:15.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:15.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:15.962 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:16:15.962 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:16:15.962 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:15.962 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:15.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:15.962 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:15.962 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:16:15.962 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:15.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:15.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:15.963 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:15.963 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:16:15.963 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:15.963 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:15.963 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:15.963 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:15.963 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:16:15.963 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:16:15.963 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:15.963 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:15.963 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:15.963 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:15.963 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:15.963 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:15.964 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:16:15.964 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:16:15.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:15.964 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:15.964 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:15.964 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:16:15.964 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:15.964 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:15.964 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": [ 2024-04-25T19:16:15.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:15.964 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:15.964 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:15.964 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 6 2024-04-25T19:16:15.964 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:15.965 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:15.965 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": [ 2024-04-25T19:16:15.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:15.965 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:15.965 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:15.965 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 4 2024-04-25T19:16:15.965 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:15.965 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:15.965 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:16:15.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:15.965 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:15.965 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:15.965 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 5 2024-04-25T19:16:15.965 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:15.966 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:15.966 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": [ 2024-04-25T19:16:15.966 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:15.966 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:15.966 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:15.966 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:16:15.966 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:15.966 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:15.966 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:15.966 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:16:15.966 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:16:15.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.951+0000 7fadc5da8640 1 -- 172.21.15.76:0/1819964254 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fadac074490 msgr2=0x7fadac0769a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:15.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.951+0000 7fadc5da8640 1 --2- 172.21.15.76:0/1819964254 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fadac074490 0x7fadac0769a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fadb8034000 tx=0x7fadb802e000 comp rx=0 tx=0).stop 2024-04-25T19:16:15.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.951+0000 7fadc5da8640 1 -- 172.21.15.76:0/1819964254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadc006de20 msgr2=0x7fadc0157550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:15.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.951+0000 7fadc5da8640 1 --2- 172.21.15.76:0/1819964254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadc006de20 0x7fadc0157550 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fadb000c430 tx=0x7fadb000c8f0 comp rx=0 tx=0).stop 2024-04-25T19:16:15.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.951+0000 7fadbffff640 1 -- 172.21.15.76:0/1819964254 reap_dead start 2024-04-25T19:16:15.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.951+0000 7fadc5da8640 1 -- 172.21.15.76:0/1819964254 shutdown_connections 2024-04-25T19:16:15.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.951+0000 7fadc5da8640 1 --2- 172.21.15.76:0/1819964254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadc006de20 0x7fadc0157550 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:15.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.951+0000 7fadc5da8640 1 -- 172.21.15.76:0/1819964254 >> 172.21.15.76:0/1819964254 conn(0x7fadc00684e0 msgr2=0x7fadc0156230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:15.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.951+0000 7fadc5da8640 1 -- 172.21.15.76:0/1819964254 shutdown_connections 2024-04-25T19:16:15.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:15.951+0000 7fadc5da8640 1 -- 172.21.15.76:0/1819964254 wait complete. 2024-04-25T19:16:15.969 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T19:16:15.969 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-04-25T19:16:15.969 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T19:16:16.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.075+0000 7f1957ed8640 1 Processor -- start 2024-04-25T19:16:16.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.075+0000 7f1957ed8640 1 -- start start 2024-04-25T19:16:16.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.077+0000 7f1957ed8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f195015cd10 0x7f1950156c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:16.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.077+0000 7f1957ed8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f195006e0c0 0x7f195006e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:16.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.077+0000 7f1957ed8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19501571c0 con 0x7f19501609c0 2024-04-25T19:16:16.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.077+0000 7f1957ed8640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f195015d250 con 0x7f195006e0c0 2024-04-25T19:16:16.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.077+0000 7f1957ed8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f195015d510 con 0x7f195015cd10 2024-04-25T19:16:16.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.077+0000 7f1955c74640 1 -- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f195015cd10 msgr2=0x7f1950156c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3301/0 2024-04-25T19:16:16.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.077+0000 7f1955473640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f195006e0c0 0x7f195006e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:16.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.077+0000 7f1955c74640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f195015cd10 0x7f1950156c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:16.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.077+0000 7f1955473640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f195006e0c0 0x7f195006e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57598/0 (socket says 172.21.15.76:57598) 2024-04-25T19:16:16.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.077+0000 7f1955473640 1 -- 172.21.15.76:0/1237737408 learned_addr learned my addr 172.21.15.76:0/1237737408 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:16.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.077+0000 7f1955473640 1 -- 172.21.15.76:0/1237737408 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f195015cd10 msgr2=0x7f1950156c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:16:16.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.077+0000 7f1955473640 1 --2- 172.21.15.76:0/1237737408 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f195015cd10 0x7f1950156c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:16.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.077+0000 7f1955473640 1 -- 172.21.15.76:0/1237737408 >> v1:172.21.15.105:6789/0 conn(0x7f19501609c0 legacy=0x7f1950072ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:16.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.077+0000 7f1955473640 1 -- 172.21.15.76:0/1237737408 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f195015d7f0 con 0x7f195006e0c0 2024-04-25T19:16:16.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.078+0000 7f1955473640 1 --2- 172.21.15.76:0/1237737408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f195006e0c0 0x7f195006e4d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1940009870 tx=0x7f194002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6000f2743e1502 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:16.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.078+0000 7f1954c72640 1 -- 172.21.15.76:0/1237737408 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1940037070 con 0x7f195006e0c0 2024-04-25T19:16:16.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.078+0000 7f1954c72640 1 -- 172.21.15.76:0/1237737408 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1940032440 con 0x7f195006e0c0 2024-04-25T19:16:16.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.078+0000 7f1954c72640 1 -- 172.21.15.76:0/1237737408 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1940032810 con 0x7f195006e0c0 2024-04-25T19:16:16.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.079+0000 7f1957ed8640 1 -- 172.21.15.76:0/1237737408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f195006e0c0 msgr2=0x7f195006e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:16.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.079+0000 7f1957ed8640 1 --2- 172.21.15.76:0/1237737408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f195006e0c0 0x7f195006e4d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1940009870 tx=0x7f194002a270 comp rx=0 tx=0).stop 2024-04-25T19:16:16.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.079+0000 7f1957ed8640 1 -- 172.21.15.76:0/1237737408 shutdown_connections 2024-04-25T19:16:16.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.079+0000 7f1957ed8640 1 --2- 172.21.15.76:0/1237737408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f195006e0c0 0x7f195006e4d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:16.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.079+0000 7f1957ed8640 1 --2- 172.21.15.76:0/1237737408 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f195015cd10 0x7f1950156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:16.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.079+0000 7f1957ed8640 1 -- 172.21.15.76:0/1237737408 >> 172.21.15.76:0/1237737408 conn(0x7f1950070b00 msgr2=0x7f1950070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:16.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.079+0000 7f1957ed8640 1 -- 172.21.15.76:0/1237737408 shutdown_connections 2024-04-25T19:16:16.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.079+0000 7f1957ed8640 1 -- 172.21.15.76:0/1237737408 wait complete. 2024-04-25T19:16:16.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.081+0000 7f1957ed8640 1 Processor -- start 2024-04-25T19:16:16.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.081+0000 7f1957ed8640 1 -- start start 2024-04-25T19:16:16.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.082+0000 7f1957ed8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f195006e0c0 0x7f19501f3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:16.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.082+0000 7f1957ed8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f195015cd10 0x7f1950153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:16.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.082+0000 7f1957ed8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19501609c0 0x7f19501f6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:16.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.082+0000 7f1957ed8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f19500c5210 con 0x7f19501609c0 2024-04-25T19:16:16.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.082+0000 7f1957ed8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f19501604a0 con 0x7f195006e0c0 2024-04-25T19:16:16.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.082+0000 7f1957ed8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f19500c5390 con 0x7f195015cd10 2024-04-25T19:16:16.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.082+0000 7f1955473640 1 -- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f195015cd10 msgr2=0x7f1950153be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3301/0 2024-04-25T19:16:16.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.082+0000 7f1955473640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f195015cd10 0x7f1950153be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:16.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.082+0000 7f1956475640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19501609c0 0x7f19501f6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:16.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.082+0000 7f1956475640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19501609c0 0x7f19501f6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57600/0 (socket says 172.21.15.76:57600) 2024-04-25T19:16:16.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.082+0000 7f1956475640 1 -- 172.21.15.76:0/877530702 learned_addr learned my addr 172.21.15.76:0/877530702 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:16.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.082+0000 7f1955c74640 1 --2- 172.21.15.76:0/877530702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f195006e0c0 0x7f19501f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:16.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.082+0000 7f1956475640 1 -- 172.21.15.76:0/877530702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f195015cd10 msgr2=0x7f1950153be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:16:16.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.082+0000 7f1956475640 1 --2- 172.21.15.76:0/877530702 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f195015cd10 0x7f1950153be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:16.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.082+0000 7f1956475640 1 -- 172.21.15.76:0/877530702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f195006e0c0 msgr2=0x7f19501f3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:16.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.082+0000 7f1956475640 1 --2- 172.21.15.76:0/877530702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f195006e0c0 0x7f19501f3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:16.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.082+0000 7f1956475640 1 -- 172.21.15.76:0/877530702 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f195015a0e0 con 0x7f19501609c0 2024-04-25T19:16:16.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.083+0000 7f1956475640 1 --2- 172.21.15.76:0/877530702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19501609c0 0x7f19501f6200 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f194800c3c0 tx=0x7f194800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:16.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.083+0000 7f19467fc640 1 -- 172.21.15.76:0/877530702 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1948017070 con 0x7f19501609c0 2024-04-25T19:16:16.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.083+0000 7f19467fc640 1 -- 172.21.15.76:0/877530702 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19480052d0 con 0x7f19501609c0 2024-04-25T19:16:16.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.083+0000 7f19467fc640 1 -- 172.21.15.76:0/877530702 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1948003830 con 0x7f19501609c0 2024-04-25T19:16:16.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.083+0000 7f1957ed8640 1 -- 172.21.15.76:0/877530702 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19501f3bc0 con 0x7f19501609c0 2024-04-25T19:16:16.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.083+0000 7f1957ed8640 1 -- 172.21.15.76:0/877530702 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19501f2e80 con 0x7f19501609c0 2024-04-25T19:16:16.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.084+0000 7f1957ed8640 1 --2- 172.21.15.76:0/877530702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1950158180 0x7f19501f6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:16.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.084+0000 7f1957ed8640 1 -- 172.21.15.76:0/877530702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f195015a600 con 0x7f1950158180 2024-04-25T19:16:16.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.084+0000 7f1955c74640 1 --2- 172.21.15.76:0/877530702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1950158180 0x7f19501f6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:16.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.084+0000 7f1955c74640 1 -- 172.21.15.76:0/877530702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f19501f2c90 con 0x7f1950158180 2024-04-25T19:16:16.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.084+0000 7f1955c74640 1 --2- 172.21.15.76:0/877530702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1950158180 0x7f19501f6740 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f194c009f40 tx=0x7f194c007720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:16.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.084+0000 7f19467fc640 1 -- 172.21.15.76:0/877530702 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f194801a070 con 0x7f19501609c0 2024-04-25T19:16:16.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.085+0000 7f19467fc640 1 --2- 172.21.15.76:0/877530702 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1934074460 0x7f1934076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:16.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.085+0000 7f19467fc640 1 -- 172.21.15.76:0/877530702 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f19480944c0 con 0x7f19501609c0 2024-04-25T19:16:16.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.085+0000 7f1955473640 1 --2- 172.21.15.76:0/877530702 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1934074460 0x7f1934076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:16.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.085+0000 7f19467fc640 1 -- 172.21.15.76:0/877530702 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c01a070 con 0x7f1950158180 2024-04-25T19:16:16.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.085+0000 7f19467fc640 1 -- 172.21.15.76:0/877530702 <== mon.1 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f194c0054e0 con 0x7f1950158180 2024-04-25T19:16:16.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.085+0000 7f19467fc640 1 -- 172.21.15.76:0/877530702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1950158180 msgr2=0x7f19501f6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:16.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.085+0000 7f19467fc640 1 --2- 172.21.15.76:0/877530702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1950158180 0x7f19501f6740 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f194c009f40 tx=0x7f194c007720 comp rx=0 tx=0).stop 2024-04-25T19:16:16.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.085+0000 7f19467fc640 1 -- 172.21.15.76:0/877530702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1950158180 msgr2=0x7f19501f6740 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:16:16.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.085+0000 7f19467fc640 1 --2- 172.21.15.76:0/877530702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1950158180 0x7f19501f6740 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:16.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.086+0000 7f1955473640 1 --2- 172.21.15.76:0/877530702 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1934074460 0x7f1934076970 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1940034000 tx=0x7f194002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:16.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.092+0000 7f1957ed8640 1 --2- 172.21.15.76:0/877530702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19501f3050 0x7f19501f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:16.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.092+0000 7f1957ed8640 1 -- 172.21.15.76:0/877530702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f195006ea90 con 0x7f19501f3050 2024-04-25T19:16:16.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.092+0000 7f1955c74640 1 --2- 172.21.15.76:0/877530702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19501f3050 0x7f19501f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:16.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.092+0000 7f1955c74640 1 -- 172.21.15.76:0/877530702 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f19500cb430 con 0x7f19501f3050 2024-04-25T19:16:16.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.092+0000 7f1955c74640 1 --2- 172.21.15.76:0/877530702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19501f3050 0x7f19501f6c80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f194c009fd0 tx=0x7f194c005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:16.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.093+0000 7f19467fc640 1 -- 172.21.15.76:0/877530702 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c01a070 con 0x7f19501f3050 2024-04-25T19:16:16.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.093+0000 7f19467fc640 1 -- 172.21.15.76:0/877530702 <== mon.1 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4882 (secure 0 0 0) 0x7f194c021030 con 0x7f19501f3050 2024-04-25T19:16:16.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.093+0000 7f19467fc640 1 -- 172.21.15.76:0/877530702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19501f3050 msgr2=0x7f19501f6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:16.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.093+0000 7f19467fc640 1 --2- 172.21.15.76:0/877530702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19501f3050 0x7f19501f6c80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f194c009fd0 tx=0x7f194c005d50 comp rx=0 tx=0).stop 2024-04-25T19:16:16.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.093+0000 7f19467fc640 1 -- 172.21.15.76:0/877530702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19501f3050 msgr2=0x7f19501f6c80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:16:16.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.093+0000 7f19467fc640 1 --2- 172.21.15.76:0/877530702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19501f3050 0x7f19501f6c80 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f194c009fd0 tx=0x7f194c005d50 comp rx=0 tx=0).stop 2024-04-25T19:16:16.095 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:16:16.095 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:16:16.095 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:16:16.096 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "peon", 2024-04-25T19:16:16.096 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 8, 2024-04-25T19:16:16.096 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 74574, 2024-04-25T19:16:16.096 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:16:16.096 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:16:16.096 INFO:teuthology.orchestra.run.smithi076.stdout: 1 2024-04-25T19:16:16.096 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:16.096 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 0, 2024-04-25T19:16:16.096 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:16:16.096 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:16:16.096 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:16:16.096 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:16.096 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:16.096 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:16.097 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:16.097 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:16.097 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:16.097 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:16.097 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:16.097 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:16.097 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:16.097 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:16.097 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:16.097 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:16:16.097 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:16:16.097 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:16.097 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:16.097 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:16.097 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:16.098 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:16.098 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:16.098 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:16.098 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:16.098 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:16.098 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:16.098 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:16.098 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:16.098 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:16.098 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:16:16.098 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:16:16.098 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:16:16.098 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:16:16.099 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:16:16.099 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:16:16.099 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:16:16.099 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:16:16.099 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:16:16.099 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:16:16.099 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:16:16.099 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:16:16.099 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:16:16.099 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:16:16.099 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:16:16.099 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:16:16.099 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:16:16.099 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:16.100 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:16.100 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:16.100 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:16.100 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:16.100 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:16.100 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:16.100 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:16.100 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:16.100 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:16.100 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:16.100 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:16.100 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:16:16.100 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:16.100 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:16:16.101 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:16.101 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:16:16.101 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:16:16.101 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:16.101 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:16.101 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:16.101 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:16.101 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:16:16.101 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:16.101 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:16.101 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:16.101 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:16.101 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:16:16.101 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:16.102 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:16.102 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:16.102 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:16.102 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:16:16.102 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:16:16.102 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:16.102 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:16.102 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:16.102 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:16.102 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:16.102 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:16:16.102 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:16:16.102 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:16.102 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:16.103 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:16.103 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:16.103 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:16:16.103 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:16.103 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:16.103 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:16.103 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:16.103 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:16:16.103 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:16.103 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:16.103 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:16.103 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:16.103 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:16:16.103 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:16:16.104 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:16.104 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:16.104 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:16.104 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:16.104 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:16.104 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:16:16.104 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:16:16.104 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:16.104 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:16.104 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:16.104 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:16.104 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:16:16.104 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:16.104 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:16.105 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:16.105 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:16.105 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:16:16.105 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:16.105 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:16.105 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:16.105 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:16.105 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:16:16.105 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:16:16.105 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:16.105 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:16.105 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:16.105 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:16.105 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:16.106 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:16.106 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:16:16.106 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:16:16.106 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:16.106 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:16.106 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:16.106 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:16:16.106 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:16.106 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:16.106 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": [ 2024-04-25T19:16:16.106 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:16.106 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:16.106 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:16.106 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 4 2024-04-25T19:16:16.107 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:16.107 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:16.107 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:16:16.107 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:16.107 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:16.107 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:16.107 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 3 2024-04-25T19:16:16.107 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:16.107 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:16.107 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": [ 2024-04-25T19:16:16.107 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:16.107 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:16.107 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:16.107 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:16:16.107 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:16.108 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:16.108 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:16.108 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:16:16.108 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:16:16.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.093+0000 7f1957ed8640 1 -- 172.21.15.76:0/877530702 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1934074460 msgr2=0x7f1934076970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:16.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.093+0000 7f1957ed8640 1 --2- 172.21.15.76:0/877530702 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f1934074460 0x7f1934076970 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1940034000 tx=0x7f194002e000 comp rx=0 tx=0).stop 2024-04-25T19:16:16.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.093+0000 7f1957ed8640 1 -- 172.21.15.76:0/877530702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19501609c0 msgr2=0x7f19501f6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:16.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.093+0000 7f1957ed8640 1 --2- 172.21.15.76:0/877530702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19501609c0 0x7f19501f6200 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f194800c3c0 tx=0x7f194800c7c0 comp rx=0 tx=0).stop 2024-04-25T19:16:16.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.093+0000 7f1956475640 1 -- 172.21.15.76:0/877530702 reap_dead start 2024-04-25T19:16:16.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.093+0000 7f1957ed8640 1 -- 172.21.15.76:0/877530702 shutdown_connections 2024-04-25T19:16:16.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.093+0000 7f1957ed8640 1 --2- 172.21.15.76:0/877530702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19501609c0 0x7f19501f6200 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:16.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.093+0000 7f1957ed8640 1 -- 172.21.15.76:0/877530702 >> 172.21.15.76:0/877530702 conn(0x7f1950070b00 msgr2=0x7f19500ca6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:16.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.093+0000 7f1957ed8640 1 -- 172.21.15.76:0/877530702 shutdown_connections 2024-04-25T19:16:16.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:16.093+0000 7f1957ed8640 1 -- 172.21.15.76:0/877530702 wait complete. 2024-04-25T19:16:16.110 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T19:16:16.110 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-04-25T19:16:25.643 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:16:25.646 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:16:36.111 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-04-25T19:16:36.112 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-04-25T19:16:36.112 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T19:16:36.112 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T19:16:36.117 INFO:tasks.ceph.mon.c:Started 2024-04-25T19:16:36.117 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T19:16:36.118 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:16:36.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.234+0000 7f0902b97640 1 Processor -- start 2024-04-25T19:16:36.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.234+0000 7f0902b97640 1 -- start start 2024-04-25T19:16:36.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.236+0000 7f0902b97640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08fc0714c0 0x7f08fc06c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.236+0000 7f0902b97640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f08fc0c9730 0x7f08fc06cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.236+0000 7f0902b97640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08fc06d160 con 0x7f08fc156bf0 2024-04-25T19:16:36.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.236+0000 7f0902b97640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f08fc15d660 con 0x7f08fc0c9730 2024-04-25T19:16:36.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.236+0000 7f0902b97640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f08fc071fa0 con 0x7f08fc0714c0 2024-04-25T19:16:36.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.236+0000 7f0901134640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f08fc156bf0 0x7f08fc0c7710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40312/0 (socket says 172.21.15.76:40312) 2024-04-25T19:16:36.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.236+0000 7f0901134640 1 -- 172.21.15.76:0/3417948099 learned_addr learned my addr 172.21.15.76:0/3417948099 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:16:36.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.236+0000 7f0900933640 1 --2- 172.21.15.76:0/3417948099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08fc0714c0 0x7f08fc06c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.236+0000 7f08fbfff640 1 --2- 172.21.15.76:0/3417948099 >> v2:172.21.15.105:3300/0 conn(0x7f08fc0c9730 0x7f08fc06cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.236+0000 7f0900933640 1 -- 172.21.15.76:0/3417948099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08fc0714c0 msgr2=0x7f08fc06c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T19:16:36.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.236+0000 7f0900933640 1 -- 172.21.15.76:0/3417948099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08fc0714c0 msgr2=0x7f08fc06c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:16:36.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.236+0000 7f0900933640 1 --2- 172.21.15.76:0/3417948099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08fc0714c0 0x7f08fc06c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:16:36.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.236+0000 7f0900933640 1 --2- 172.21.15.76:0/3417948099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08fc0714c0 0x7f08fc06c5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:36.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.236+0000 7f08fb7fe640 1 -- 172.21.15.76:0/3417948099 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3883849460 0 0) 0x7f08fc06d160 con 0x7f08fc156bf0 2024-04-25T19:16:36.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.236+0000 7f08fb7fe640 1 -- 172.21.15.76:0/3417948099 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08ec003520 con 0x7f08fc156bf0 2024-04-25T19:16:36.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.236+0000 7f08fbfff640 1 -- 172.21.15.76:0/3417948099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08fc0714c0 msgr2=0x7f08fc06c5c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:16:36.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.236+0000 7f08fbfff640 1 --2- 172.21.15.76:0/3417948099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08fc0714c0 0x7f08fc06c5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.236+0000 7f08fbfff640 1 -- 172.21.15.76:0/3417948099 >> v1:172.21.15.76:6789/0 conn(0x7f08fc156bf0 legacy=0x7f08fc0c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:36.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.236+0000 7f08fbfff640 1 -- 172.21.15.76:0/3417948099 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08fc15d940 con 0x7f08fc0c9730 2024-04-25T19:16:36.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.237+0000 7f08fbfff640 1 --2- 172.21.15.76:0/3417948099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08fc0c9730 0x7f08fc06cb00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f08e80097a0 tx=0x7f08e802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af21f9810281e31f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:36.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.237+0000 7f08fb7fe640 1 -- 172.21.15.76:0/3417948099 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e8037070 con 0x7f08fc0c9730 2024-04-25T19:16:36.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.237+0000 7f08fb7fe640 1 -- 172.21.15.76:0/3417948099 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08e8032440 con 0x7f08fc0c9730 2024-04-25T19:16:36.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.237+0000 7f08fb7fe640 1 -- 172.21.15.76:0/3417948099 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e80327f0 con 0x7f08fc0c9730 2024-04-25T19:16:36.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.238+0000 7f0902b97640 1 -- 172.21.15.76:0/3417948099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08fc0c9730 msgr2=0x7f08fc06cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:36.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.238+0000 7f0902b97640 1 --2- 172.21.15.76:0/3417948099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08fc0c9730 0x7f08fc06cb00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f08e80097a0 tx=0x7f08e802a180 comp rx=0 tx=0).stop 2024-04-25T19:16:36.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.238+0000 7f0902b97640 1 -- 172.21.15.76:0/3417948099 shutdown_connections 2024-04-25T19:16:36.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.238+0000 7f0902b97640 1 --2- 172.21.15.76:0/3417948099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08fc0c9730 0x7f08fc06cb00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.238+0000 7f0902b97640 1 --2- 172.21.15.76:0/3417948099 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08fc0714c0 0x7f08fc06c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.238+0000 7f0902b97640 1 -- 172.21.15.76:0/3417948099 >> 172.21.15.76:0/3417948099 conn(0x7f08fc06d6d0 msgr2=0x7f08fc06dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:36.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.238+0000 7f0902b97640 1 -- 172.21.15.76:0/3417948099 shutdown_connections 2024-04-25T19:16:36.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.238+0000 7f0902b97640 1 -- 172.21.15.76:0/3417948099 wait complete. 2024-04-25T19:16:36.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.240+0000 7f0902b97640 1 Processor -- start 2024-04-25T19:16:36.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.240+0000 7f0902b97640 1 -- start start 2024-04-25T19:16:36.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.241+0000 7f0902b97640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08fc0c9730 0x7f08fc143240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.241+0000 7f0902b97640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08fc156bf0 0x7f08fc0cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.241+0000 7f0902b97640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08fc143990 0x7f08fc0ce150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.241+0000 7f0902b97640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc0cc6f0 con 0x7f08fc156bf0 2024-04-25T19:16:36.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.241+0000 7f0902b97640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc0cc570 con 0x7f08fc143990 2024-04-25T19:16:36.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.241+0000 7f0902b97640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f08fc0c7e20 con 0x7f08fc0c9730 2024-04-25T19:16:36.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.241+0000 7f08fbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08fc156bf0 0x7f08fc0cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.241+0000 7f08fbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08fc156bf0 0x7f08fc0cdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46072/0 (socket says 172.21.15.76:46072) 2024-04-25T19:16:36.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.241+0000 7f08fbfff640 1 -- 172.21.15.76:0/1172426761 learned_addr learned my addr 172.21.15.76:0/1172426761 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:36.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.241+0000 7f0901134640 1 --2- 172.21.15.76:0/1172426761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08fc143990 0x7f08fc0ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.241+0000 7f0900933640 1 --2- 172.21.15.76:0/1172426761 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08fc0c9730 0x7f08fc143240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.241+0000 7f0900933640 1 -- 172.21.15.76:0/1172426761 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08fc0c9730 msgr2=0x7f08fc143240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-25T19:16:36.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.241+0000 7f0900933640 1 -- 172.21.15.76:0/1172426761 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08fc0c9730 msgr2=0x7f08fc143240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T19:16:36.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.241+0000 7f0900933640 1 --2- 172.21.15.76:0/1172426761 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08fc0c9730 0x7f08fc143240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:16:36.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.241+0000 7f0900933640 1 --2- 172.21.15.76:0/1172426761 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08fc0c9730 0x7f08fc143240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:36.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.241+0000 7f08fbfff640 1 -- 172.21.15.76:0/1172426761 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08fc0c9730 msgr2=0x7f08fc143240 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:16:36.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.241+0000 7f08fbfff640 1 --2- 172.21.15.76:0/1172426761 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08fc0c9730 0x7f08fc143240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.241+0000 7f08fbfff640 1 -- 172.21.15.76:0/1172426761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08fc143990 msgr2=0x7f08fc0ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:36.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.241+0000 7f08fbfff640 1 --2- 172.21.15.76:0/1172426761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08fc143990 0x7f08fc0ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.241+0000 7f08fbfff640 1 -- 172.21.15.76:0/1172426761 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08fc0ca170 con 0x7f08fc156bf0 2024-04-25T19:16:36.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.242+0000 7f08fbfff640 1 --2- 172.21.15.76:0/1172426761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08fc156bf0 0x7f08fc0cdc10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f08e802a1c0 tx=0x7f08e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:36.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.242+0000 7f08f97fa640 1 -- 172.21.15.76:0/1172426761 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e8037040 con 0x7f08fc156bf0 2024-04-25T19:16:36.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.242+0000 7f08f97fa640 1 -- 172.21.15.76:0/1172426761 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08e8042070 con 0x7f08fc156bf0 2024-04-25T19:16:36.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.242+0000 7f08f97fa640 1 -- 172.21.15.76:0/1172426761 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e8030900 con 0x7f08fc156bf0 2024-04-25T19:16:36.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.242+0000 7f0902b97640 1 -- 172.21.15.76:0/1172426761 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08fc143780 con 0x7f08fc156bf0 2024-04-25T19:16:36.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.242+0000 7f0902b97640 1 -- 172.21.15.76:0/1172426761 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08fc15dd50 con 0x7f08fc156bf0 2024-04-25T19:16:36.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.243+0000 7f0902b97640 1 -- 172.21.15.76:0/1172426761 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08fc06a390 con 0x7f08fc156bf0 2024-04-25T19:16:36.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.243+0000 7f08f97fa640 1 -- 172.21.15.76:0/1172426761 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f08e8030aa0 con 0x7f08fc156bf0 2024-04-25T19:16:36.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.244+0000 7f08f97fa640 1 --2- 172.21.15.76:0/1172426761 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f08e00744e0 0x7f08e00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.244+0000 7f08f97fa640 1 -- 172.21.15.76:0/1172426761 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f08e80b5950 con 0x7f08fc156bf0 2024-04-25T19:16:36.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.244+0000 7f0900933640 1 --2- 172.21.15.76:0/1172426761 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f08e00744e0 0x7f08e00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.244+0000 7f0900933640 1 --2- 172.21.15.76:0/1172426761 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f08e00744e0 0x7f08e00769f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f08f0006fd0 tx=0x7f08f0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:36.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.249+0000 7f08f97fa640 1 -- 172.21.15.76:0/1172426761 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f08e80800f0 con 0x7f08fc156bf0 2024-04-25T19:16:36.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.492+0000 7f0902b97640 1 -- 172.21.15.76:0/1172426761 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f08fc160e60 con 0x7f08fc156bf0 2024-04-25T19:16:36.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.492+0000 7f08f97fa640 1 -- 172.21.15.76:0/1172426761 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7f08e807f9f0 con 0x7f08fc156bf0 2024-04-25T19:16:36.494 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:16:36.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.494+0000 7f0902b97640 1 -- 172.21.15.76:0/1172426761 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f08e00744e0 msgr2=0x7f08e00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:36.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.494+0000 7f0902b97640 1 --2- 172.21.15.76:0/1172426761 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f08e00744e0 0x7f08e00769f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f08f0006fd0 tx=0x7f08f0008040 comp rx=0 tx=0).stop 2024-04-25T19:16:36.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.494+0000 7f0902b97640 1 -- 172.21.15.76:0/1172426761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08fc156bf0 msgr2=0x7f08fc0cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:36.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.494+0000 7f0902b97640 1 --2- 172.21.15.76:0/1172426761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08fc156bf0 0x7f08fc0cdc10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f08e802a1c0 tx=0x7f08e80046c0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.495+0000 7f0902b97640 1 -- 172.21.15.76:0/1172426761 shutdown_connections 2024-04-25T19:16:36.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.495+0000 7f0902b97640 1 --2- 172.21.15.76:0/1172426761 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f08e00744e0 0x7f08e00769f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.495+0000 7f0902b97640 1 --2- 172.21.15.76:0/1172426761 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f08fc143990 0x7f08fc0ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.495+0000 7f0902b97640 1 --2- 172.21.15.76:0/1172426761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08fc156bf0 0x7f08fc0cdc10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.495+0000 7f0902b97640 1 --2- 172.21.15.76:0/1172426761 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f08fc0c9730 0x7f08fc143240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.495+0000 7f0902b97640 1 -- 172.21.15.76:0/1172426761 >> 172.21.15.76:0/1172426761 conn(0x7f08fc06d6d0 msgr2=0x7f08fc075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:36.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.495+0000 7f0902b97640 1 -- 172.21.15.76:0/1172426761 shutdown_connections 2024-04-25T19:16:36.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.495+0000 7f0902b97640 1 -- 172.21.15.76:0/1172426761 wait complete. 2024-04-25T19:16:36.507 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T19:16:36.508 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T19:16:36.508 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T19:16:36.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.616+0000 7fc7bc78f640 1 Processor -- start 2024-04-25T19:16:36.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.616+0000 7fc7bc78f640 1 -- start start 2024-04-25T19:16:36.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.617+0000 7fc7bc78f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7b415ac70 0x7fc7b415b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.617+0000 7fc7bc78f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7b4071ad0 con 0x7fc7b4073880 2024-04-25T19:16:36.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.617+0000 7fc7bc78f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7b4071d70 con 0x7fc7b4071520 2024-04-25T19:16:36.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.617+0000 7fc7bc78f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fc7b4074150 con 0x7fc7b415ac70 2024-04-25T19:16:36.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.617+0000 7fc7b9d2a640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc7b4073880 0x7fc7b40c83d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:40324/0 (socket says 172.21.15.76:40324) 2024-04-25T19:16:36.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.617+0000 7fc7b9d2a640 1 -- 172.21.15.76:0/3313497471 learned_addr learned my addr 172.21.15.76:0/3313497471 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:16:36.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.617+0000 7fc7bad2c640 1 --2- 172.21.15.76:0/3313497471 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7b415ac70 0x7fc7b415b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.617+0000 7fc7b9529640 1 -- 172.21.15.76:0/3313497471 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2943249905 0 0) 0x7fc7b4071ad0 con 0x7fc7b4073880 2024-04-25T19:16:36.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.617+0000 7fc7b9529640 1 -- 172.21.15.76:0/3313497471 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7a0003520 con 0x7fc7b4073880 2024-04-25T19:16:36.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.618+0000 7fc7bad2c640 1 -- 172.21.15.76:0/3313497471 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7b415ac70 msgr2=0x7fc7b415b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T19:16:36.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.618+0000 7fc7bad2c640 1 -- 172.21.15.76:0/3313497471 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7b415ac70 msgr2=0x7fc7b415b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:16:36.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.618+0000 7fc7bad2c640 1 --2- 172.21.15.76:0/3313497471 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7b415ac70 0x7fc7b415b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:16:36.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.618+0000 7fc7bad2c640 1 --2- 172.21.15.76:0/3313497471 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7b415ac70 0x7fc7b415b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:36.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.618+0000 7fc7b9529640 1 -- 172.21.15.76:0/3313497471 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2996727076 0 0) 0x7fc7a0003520 con 0x7fc7b4073880 2024-04-25T19:16:36.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.618+0000 7fc7b9529640 1 -- 172.21.15.76:0/3313497471 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7b415ac70 msgr2=0x7fc7b415b080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:16:36.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.618+0000 7fc7b9529640 1 --2- 172.21.15.76:0/3313497471 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7b415ac70 0x7fc7b415b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.618+0000 7fc7b9529640 1 -- 172.21.15.76:0/3313497471 >> v1:172.21.15.105:6789/0 conn(0x7fc7b4071520 legacy=0x7fc7b4151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:36.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.618+0000 7fc7b9529640 1 -- 172.21.15.76:0/3313497471 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b4074430 con 0x7fc7b4073880 2024-04-25T19:16:36.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.618+0000 7fc7b9529640 1 -- 172.21.15.76:0/3313497471 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7fc7a4001670 con 0x7fc7b4073880 2024-04-25T19:16:36.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.618+0000 7fc7b9529640 1 -- 172.21.15.76:0/3313497471 >> v1:172.21.15.76:6789/0 conn(0x7fc7b4073880 legacy=0x7fc7b40c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:36.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.618+0000 7fc7b9529640 1 --2- 172.21.15.76:0/3313497471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7a00039e0 0x7fc7a0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.618+0000 7fc7b9529640 1 --2- 172.21.15.76:0/3313497471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7a0004800 0x7fc7a0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.618+0000 7fc7b9529640 1 --2- 172.21.15.76:0/3313497471 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7b415ac70 0x7fc7a00052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.618+0000 7fc7b9529640 1 -- 172.21.15.76:0/3313497471 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b4074150 con 0x7fc7a00039e0 2024-04-25T19:16:36.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.618+0000 7fc7b9529640 1 -- 172.21.15.76:0/3313497471 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a000bb50 con 0x7fc7a0004800 2024-04-25T19:16:36.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.618+0000 7fc7b9529640 1 -- 172.21.15.76:0/3313497471 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a000bd50 con 0x7fc7b415ac70 2024-04-25T19:16:36.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.618+0000 7fc7ba52b640 1 --2- 172.21.15.76:0/3313497471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7a00039e0 0x7fc7a0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.618+0000 7fc7b9d2a640 1 --2- 172.21.15.76:0/3313497471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7a0004800 0x7fc7a0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.618+0000 7fc7bad2c640 1 --2- 172.21.15.76:0/3313497471 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7b415ac70 0x7fc7a00052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.618+0000 7fc7ba52b640 1 -- 172.21.15.76:0/3313497471 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7b415ac70 msgr2=0x7fc7a00052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:36.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.618+0000 7fc7ba52b640 1 --2- 172.21.15.76:0/3313497471 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7b415ac70 0x7fc7a00052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.618+0000 7fc7ba52b640 1 -- 172.21.15.76:0/3313497471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7a0004800 msgr2=0x7fc7a0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:36.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.618+0000 7fc7ba52b640 1 --2- 172.21.15.76:0/3313497471 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7a0004800 0x7fc7a0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.618+0000 7fc7ba52b640 1 -- 172.21.15.76:0/3313497471 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a0003520 con 0x7fc7a00039e0 2024-04-25T19:16:36.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.619+0000 7fc7ba52b640 1 --2- 172.21.15.76:0/3313497471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7a00039e0 0x7fc7a0003e10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0001f30 tx=0x7fc7a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d47c8ec7292e29d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:36.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.619+0000 7fc7b9529640 1 -- 172.21.15.76:0/3313497471 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0001b30 con 0x7fc7a00039e0 2024-04-25T19:16:36.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.619+0000 7fc7b9529640 1 -- 172.21.15.76:0/3313497471 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b0003b70 con 0x7fc7a00039e0 2024-04-25T19:16:36.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.619+0000 7fc7b9529640 1 -- 172.21.15.76:0/3313497471 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b00050d0 con 0x7fc7a00039e0 2024-04-25T19:16:36.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.620+0000 7fc7bc78f640 1 -- 172.21.15.76:0/3313497471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7a00039e0 msgr2=0x7fc7a0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:36.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.620+0000 7fc7bc78f640 1 --2- 172.21.15.76:0/3313497471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7a00039e0 0x7fc7a0003e10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0001f30 tx=0x7fc7a0005830 comp rx=0 tx=0).stop 2024-04-25T19:16:36.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.620+0000 7fc7bad2c640 1 -- 172.21.15.76:0/3313497471 reap_dead start 2024-04-25T19:16:36.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.620+0000 7fc7bc78f640 1 -- 172.21.15.76:0/3313497471 shutdown_connections 2024-04-25T19:16:36.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.620+0000 7fc7bc78f640 1 -- 172.21.15.76:0/3313497471 >> 172.21.15.76:0/3313497471 conn(0x7fc7b40cecc0 msgr2=0x7fc7b40cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:36.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.620+0000 7fc7bc78f640 1 -- 172.21.15.76:0/3313497471 shutdown_connections 2024-04-25T19:16:36.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.620+0000 7fc7bc78f640 1 -- 172.21.15.76:0/3313497471 wait complete. 2024-04-25T19:16:36.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.622+0000 7fc7bc78f640 1 Processor -- start 2024-04-25T19:16:36.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.622+0000 7fc7bc78f640 1 -- start start 2024-04-25T19:16:36.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.623+0000 7fc7bc78f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7a00039e0 0x7fc7b41f29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.623+0000 7fc7bc78f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7b414b680 0x7fc7b4149b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.623+0000 7fc7bc78f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7b4153f50 0x7fc7b41f5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.623+0000 7fc7bc78f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b4154fc0 con 0x7fc7a00039e0 2024-04-25T19:16:36.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.623+0000 7fc7bc78f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b40749f0 con 0x7fc7b4153f50 2024-04-25T19:16:36.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.623+0000 7fc7bc78f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fc7b41f33f0 con 0x7fc7b414b680 2024-04-25T19:16:36.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.623+0000 7fc7ba52b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7a00039e0 0x7fc7b41f29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.623+0000 7fc7ba52b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7a00039e0 0x7fc7b41f29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46080/0 (socket says 172.21.15.76:46080) 2024-04-25T19:16:36.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.623+0000 7fc7ba52b640 1 -- 172.21.15.76:0/263153059 learned_addr learned my addr 172.21.15.76:0/263153059 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:36.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.623+0000 7fc7bad2c640 1 --2- 172.21.15.76:0/263153059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7b4153f50 0x7fc7b41f5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.623+0000 7fc7b9d2a640 1 --2- 172.21.15.76:0/263153059 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7b414b680 0x7fc7b4149b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.623+0000 7fc7b9d2a640 1 -- 172.21.15.76:0/263153059 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7b414b680 msgr2=0x7fc7b4149b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T19:16:36.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.623+0000 7fc7b9d2a640 1 -- 172.21.15.76:0/263153059 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7b414b680 msgr2=0x7fc7b4149b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T19:16:36.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.623+0000 7fc7b9d2a640 1 --2- 172.21.15.76:0/263153059 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7b414b680 0x7fc7b4149b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:16:36.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.623+0000 7fc7b9d2a640 1 --2- 172.21.15.76:0/263153059 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7b414b680 0x7fc7b4149b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:36.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.623+0000 7fc7ba52b640 1 -- 172.21.15.76:0/263153059 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7b414b680 msgr2=0x7fc7b4149b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:16:36.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.623+0000 7fc7ba52b640 1 --2- 172.21.15.76:0/263153059 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc7b414b680 0x7fc7b4149b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.623+0000 7fc7ba52b640 1 -- 172.21.15.76:0/263153059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7b4153f50 msgr2=0x7fc7b41f5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:36.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.623+0000 7fc7ba52b640 1 --2- 172.21.15.76:0/263153059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7b4153f50 0x7fc7b41f5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.623+0000 7fc7ba52b640 1 -- 172.21.15.76:0/263153059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b41f2f20 con 0x7fc7a00039e0 2024-04-25T19:16:36.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.623+0000 7fc7bad2c640 1 --2- 172.21.15.76:0/263153059 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc7b4153f50 0x7fc7b41f5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:16:36.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.623+0000 7fc7ba52b640 1 --2- 172.21.15.76:0/263153059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7a00039e0 0x7fc7b41f29e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0033040 tx=0x7fc7b00015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:36.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.624+0000 7fc7aaffd640 1 -- 172.21.15.76:0/263153059 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0041070 con 0x7fc7a00039e0 2024-04-25T19:16:36.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.624+0000 7fc7aaffd640 1 -- 172.21.15.76:0/263153059 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b0006d50 con 0x7fc7a00039e0 2024-04-25T19:16:36.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.624+0000 7fc7bc78f640 1 -- 172.21.15.76:0/263153059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b40c6d00 con 0x7fc7a00039e0 2024-04-25T19:16:36.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.624+0000 7fc7aaffd640 1 -- 172.21.15.76:0/263153059 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0007240 con 0x7fc7a00039e0 2024-04-25T19:16:36.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.624+0000 7fc7bc78f640 1 -- 172.21.15.76:0/263153059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b40c7a80 con 0x7fc7a00039e0 2024-04-25T19:16:36.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.624+0000 7fc7bc78f640 1 --2- 172.21.15.76:0/263153059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7b40c7330 0x7fc7b41f62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.624+0000 7fc7bc78f640 1 -- 172.21.15.76:0/263153059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b4155140 con 0x7fc7b40c7330 2024-04-25T19:16:36.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.625+0000 7fc7b9d2a640 1 --2- 172.21.15.76:0/263153059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7b40c7330 0x7fc7b41f62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.625+0000 7fc7b9d2a640 1 -- 172.21.15.76:0/263153059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc7b4154de0 con 0x7fc7b40c7330 2024-04-25T19:16:36.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.625+0000 7fc7b9d2a640 1 --2- 172.21.15.76:0/263153059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7b40c7330 0x7fc7b41f62b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40ccb90 tx=0x7fc7a4007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:36.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.625+0000 7fc7aaffd640 1 -- 172.21.15.76:0/263153059 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a40117a0 con 0x7fc7b40c7330 2024-04-25T19:16:36.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.625+0000 7fc7aaffd640 1 -- 172.21.15.76:0/263153059 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc7b0003a30 con 0x7fc7a00039e0 2024-04-25T19:16:36.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.626+0000 7fc7aaffd640 1 --2- 172.21.15.76:0/263153059 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fc7980743b0 0x7fc7980768c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.626+0000 7fc7aaffd640 1 -- 172.21.15.76:0/263153059 <== mon.0 v2:172.21.15.76:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc7a401a070 con 0x7fc7b40c7330 2024-04-25T19:16:36.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.626+0000 7fc7aaffd640 1 -- 172.21.15.76:0/263153059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7b40c7330 msgr2=0x7fc7b41f62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:36.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.626+0000 7fc7aaffd640 1 --2- 172.21.15.76:0/263153059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7b40c7330 0x7fc7b41f62b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40ccb90 tx=0x7fc7a4007430 comp rx=0 tx=0).stop 2024-04-25T19:16:36.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.626+0000 7fc7aaffd640 1 -- 172.21.15.76:0/263153059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7b40c7330 msgr2=0x7fc7b41f62b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:16:36.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.626+0000 7fc7aaffd640 1 --2- 172.21.15.76:0/263153059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7b40c7330 0x7fc7b41f62b0 secure :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40ccb90 tx=0x7fc7a4007430 comp rx=0 tx=0).stop 2024-04-25T19:16:36.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.626+0000 7fc7aaffd640 1 -- 172.21.15.76:0/263153059 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fc7b00b5d90 con 0x7fc7a00039e0 2024-04-25T19:16:36.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.626+0000 7fc7bad2c640 1 --2- 172.21.15.76:0/263153059 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fc7980743b0 0x7fc7980768c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.626+0000 7fc7bad2c640 1 --2- 172.21.15.76:0/263153059 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fc7980743b0 0x7fc7980768c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40c6060 tx=0x7fc7ac008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:36.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.632+0000 7fc7bc78f640 1 --2- 172.21.15.76:0/263153059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7b41f67f0 0x7fc7b41473f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.632+0000 7fc7bc78f640 1 -- 172.21.15.76:0/263153059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b41f6c00 con 0x7fc7b41f67f0 2024-04-25T19:16:36.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.632+0000 7fc7b9d2a640 1 --2- 172.21.15.76:0/263153059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7b41f67f0 0x7fc7b41473f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.633+0000 7fc7b9d2a640 1 -- 172.21.15.76:0/263153059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc7b406a390 con 0x7fc7b41f67f0 2024-04-25T19:16:36.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.633+0000 7fc7b9d2a640 1 --2- 172.21.15.76:0/263153059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7b41f67f0 0x7fc7b41473f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40017e0 tx=0x7fc7b4074600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:36.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.633+0000 7fc7aaffd640 1 -- 172.21.15.76:0/263153059 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a401fe10 con 0x7fc7b41f67f0 2024-04-25T19:16:36.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.633+0000 7fc7aaffd640 1 -- 172.21.15.76:0/263153059 <== mon.0 v2:172.21.15.76:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5066 (secure 0 0 0) 0x7fc7a401fe10 con 0x7fc7b41f67f0 2024-04-25T19:16:36.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.633+0000 7fc7aaffd640 1 -- 172.21.15.76:0/263153059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7b41f67f0 msgr2=0x7fc7b41473f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:36.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.633+0000 7fc7aaffd640 1 --2- 172.21.15.76:0/263153059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7b41f67f0 0x7fc7b41473f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40017e0 tx=0x7fc7b4074600 comp rx=0 tx=0).stop 2024-04-25T19:16:36.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.633+0000 7fc7aaffd640 1 -- 172.21.15.76:0/263153059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7b41f67f0 msgr2=0x7fc7b41473f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:16:36.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.633+0000 7fc7aaffd640 1 --2- 172.21.15.76:0/263153059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7b41f67f0 0x7fc7b41473f0 secure :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40017e0 tx=0x7fc7b4074600 comp rx=0 tx=0).stop 2024-04-25T19:16:36.636 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:16:36.636 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:16:36.636 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:16:36.636 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "leader", 2024-04-25T19:16:36.636 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 10, 2024-04-25T19:16:36.636 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 95159, 2024-04-25T19:16:36.636 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:16:36.636 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:16:36.637 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-04-25T19:16:36.637 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:16:36.637 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:36.637 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 0, 2024-04-25T19:16:36.637 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:16:36.637 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:16:36.637 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:16:36.637 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:36.637 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:36.637 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:36.637 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:36.637 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:36.637 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:36.637 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:36.637 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:36.638 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:36.638 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:36.638 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:36.638 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:36.638 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:16:36.638 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:16:36.638 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:36.638 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:36.638 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:36.638 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:36.638 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:36.638 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:36.638 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:36.638 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:36.639 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:36.639 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:36.639 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:36.639 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:36.639 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:36.639 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:16:36.639 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:16:36.639 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:16:36.639 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:16:36.639 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:16:36.639 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:16:36.639 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:16:36.639 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:16:36.639 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:16:36.639 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:16:36.639 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:16:36.640 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:16:36.640 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:16:36.640 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:16:36.640 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:16:36.640 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:16:36.640 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:16:36.640 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:36.640 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:36.640 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:36.640 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:36.640 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:36.640 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:36.640 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:36.640 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:36.640 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:36.641 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:36.641 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:36.641 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:36.641 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:16:36.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:36.641 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:16:36.641 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.641 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:16:36.641 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:16:36.641 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:36.641 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:36.641 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.641 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:36.641 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:16:36.641 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:36.642 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:36.642 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.642 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:36.642 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:16:36.642 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:36.642 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:36.642 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:36.642 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:36.642 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:16:36.642 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:16:36.642 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:36.642 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:36.642 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:36.642 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:36.642 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.643 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:16:36.643 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:16:36.643 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:36.643 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:36.643 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.643 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:36.643 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:16:36.643 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:36.643 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:36.643 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.643 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:36.643 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:16:36.643 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:36.643 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:36.643 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:36.644 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:36.644 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:16:36.644 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:16:36.644 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:36.644 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:36.644 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:36.644 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:36.644 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.644 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:16:36.644 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:16:36.644 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:36.644 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:36.644 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.644 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:36.644 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:16:36.644 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:36.645 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:36.645 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.645 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:36.645 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:16:36.645 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:36.645 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:36.645 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:36.645 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:36.645 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:16:36.645 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:16:36.645 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:36.645 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:36.645 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:36.645 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:36.645 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:36.646 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:36.646 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:16:36.646 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:16:36.646 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.646 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:36.646 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:36.646 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:16:36.646 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:36.646 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:36.646 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": [ 2024-04-25T19:16:36.646 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.646 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:36.646 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:36.646 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 6 2024-04-25T19:16:36.647 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:36.647 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:36.647 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": [ 2024-04-25T19:16:36.647 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.647 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:36.647 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:36.647 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 4 2024-04-25T19:16:36.647 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:36.647 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:36.647 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:16:36.647 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.647 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:36.647 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:36.647 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 5 2024-04-25T19:16:36.647 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:36.648 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:36.648 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": [ 2024-04-25T19:16:36.648 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.648 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:36.648 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:36.648 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:16:36.648 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:36.648 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:36.648 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:36.648 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:16:36.648 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:16:36.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.634+0000 7fc7bc78f640 1 -- 172.21.15.76:0/263153059 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fc7980743b0 msgr2=0x7fc7980768c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:36.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.634+0000 7fc7bc78f640 1 --2- 172.21.15.76:0/263153059 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fc7980743b0 0x7fc7980768c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40c6060 tx=0x7fc7ac008040 comp rx=0 tx=0).stop 2024-04-25T19:16:36.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.634+0000 7fc7bc78f640 1 -- 172.21.15.76:0/263153059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7a00039e0 msgr2=0x7fc7b41f29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:36.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.634+0000 7fc7bc78f640 1 --2- 172.21.15.76:0/263153059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc7a00039e0 0x7fc7b41f29e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0033040 tx=0x7fc7b00015b0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.634+0000 7fc7bad2c640 1 -- 172.21.15.76:0/263153059 reap_dead start 2024-04-25T19:16:36.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.634+0000 7fc7bad2c640 1 -- 172.21.15.76:0/263153059 reap_dead start 2024-04-25T19:16:36.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.634+0000 7fc7bc78f640 1 -- 172.21.15.76:0/263153059 shutdown_connections 2024-04-25T19:16:36.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.634+0000 7fc7bc78f640 1 -- 172.21.15.76:0/263153059 >> 172.21.15.76:0/263153059 conn(0x7fc7b40cecc0 msgr2=0x7fc7b40cbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:36.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.634+0000 7fc7bc78f640 1 -- 172.21.15.76:0/263153059 shutdown_connections 2024-04-25T19:16:36.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.634+0000 7fc7bc78f640 1 -- 172.21.15.76:0/263153059 wait complete. 2024-04-25T19:16:36.650 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T19:16:36.651 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T19:16:36.651 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T19:16:36.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.758+0000 7ff7318a7640 1 Processor -- start 2024-04-25T19:16:36.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.758+0000 7ff7318a7640 1 -- start start 2024-04-25T19:16:36.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.760+0000 7ff7318a7640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff72c157200 0x7ff72c0d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.760+0000 7ff7318a7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff72c0cfd50 0x7ff72c0cee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.760+0000 7ff7318a7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff72c0d0160 0x7ff72c0cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.760+0000 7ff7318a7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff72c15dd00 con 0x7ff72c0cfd50 2024-04-25T19:16:36.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.760+0000 7ff7318a7640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7ff72c15cea0 con 0x7ff72c157200 2024-04-25T19:16:36.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.760+0000 7ff7318a7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff72c15d180 con 0x7ff72c0d0160 2024-04-25T19:16:36.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.760+0000 7ff72a7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff72c0cfd50 0x7ff72c0cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.760+0000 7ff72a7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff72c0cfd50 0x7ff72c0cee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46104/0 (socket says 172.21.15.76:46104) 2024-04-25T19:16:36.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.760+0000 7ff72a7fc640 1 -- 172.21.15.76:0/3395917538 learned_addr learned my addr 172.21.15.76:0/3395917538 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:36.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.760+0000 7ff72b7fe640 1 --2- 172.21.15.76:0/3395917538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff72c0d0160 0x7ff72c0cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.760+0000 7ff72affd640 1 --2- 172.21.15.76:0/3395917538 >> v2:172.21.15.105:3300/0 conn(0x7ff72c157200 0x7ff72c0d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.760+0000 7ff72b7fe640 1 -- 172.21.15.76:0/3395917538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff72c0d0160 msgr2=0x7ff72c0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T19:16:36.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.760+0000 7ff72b7fe640 1 -- 172.21.15.76:0/3395917538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff72c0d0160 msgr2=0x7ff72c0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:16:36.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.760+0000 7ff72b7fe640 1 --2- 172.21.15.76:0/3395917538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff72c0d0160 0x7ff72c0cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:16:36.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.760+0000 7ff72a7fc640 1 -- 172.21.15.76:0/3395917538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff72c0d0160 msgr2=0x7ff72c0cf3d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:16:36.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.760+0000 7ff72b7fe640 1 --2- 172.21.15.76:0/3395917538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff72c0d0160 0x7ff72c0cf3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:36.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.760+0000 7ff72a7fc640 1 --2- 172.21.15.76:0/3395917538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff72c0d0160 0x7ff72c0cf3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.760+0000 7ff72a7fc640 1 -- 172.21.15.76:0/3395917538 >> v2:172.21.15.105:3300/0 conn(0x7ff72c157200 msgr2=0x7ff72c0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:36.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.760+0000 7ff72a7fc640 1 --2- 172.21.15.76:0/3395917538 >> v2:172.21.15.105:3300/0 conn(0x7ff72c157200 0x7ff72c0d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.760+0000 7ff72a7fc640 1 -- 172.21.15.76:0/3395917538 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72c15d460 con 0x7ff72c0cfd50 2024-04-25T19:16:36.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.760+0000 7ff72affd640 1 --2- 172.21.15.76:0/3395917538 >> v2:172.21.15.105:3300/0 conn(0x7ff72c157200 0x7ff72c0d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:16:36.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.761+0000 7ff72a7fc640 1 --2- 172.21.15.76:0/3395917538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff72c0cfd50 0x7ff72c0cee90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0cfad0 tx=0x7ff72002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f716f3582b2c70ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:36.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.761+0000 7ff729ffb640 1 -- 172.21.15.76:0/3395917538 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720037070 con 0x7ff72c0cfd50 2024-04-25T19:16:36.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.761+0000 7ff729ffb640 1 -- 172.21.15.76:0/3395917538 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff720032440 con 0x7ff72c0cfd50 2024-04-25T19:16:36.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.761+0000 7ff729ffb640 1 -- 172.21.15.76:0/3395917538 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720032810 con 0x7ff72c0cfd50 2024-04-25T19:16:36.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.762+0000 7ff7318a7640 1 -- 172.21.15.76:0/3395917538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff72c0cfd50 msgr2=0x7ff72c0cee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:36.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.762+0000 7ff7318a7640 1 --2- 172.21.15.76:0/3395917538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff72c0cfd50 0x7ff72c0cee90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0cfad0 tx=0x7ff72002a270 comp rx=0 tx=0).stop 2024-04-25T19:16:36.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.762+0000 7ff7318a7640 1 -- 172.21.15.76:0/3395917538 shutdown_connections 2024-04-25T19:16:36.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.762+0000 7ff7318a7640 1 --2- 172.21.15.76:0/3395917538 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff72c0d0160 0x7ff72c0cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.762+0000 7ff7318a7640 1 --2- 172.21.15.76:0/3395917538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff72c0cfd50 0x7ff72c0cee90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.762+0000 7ff7318a7640 1 --2- 172.21.15.76:0/3395917538 >> v2:172.21.15.105:3300/0 conn(0x7ff72c157200 0x7ff72c0d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.762+0000 7ff7318a7640 1 -- 172.21.15.76:0/3395917538 >> 172.21.15.76:0/3395917538 conn(0x7ff72c0d2370 msgr2=0x7ff72c0d2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:36.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.762+0000 7ff7318a7640 1 -- 172.21.15.76:0/3395917538 shutdown_connections 2024-04-25T19:16:36.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.762+0000 7ff7318a7640 1 -- 172.21.15.76:0/3395917538 wait complete. 2024-04-25T19:16:36.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.763+0000 7ff7318a7640 1 Processor -- start 2024-04-25T19:16:36.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.764+0000 7ff7318a7640 1 -- start start 2024-04-25T19:16:36.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.765+0000 7ff7318a7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff72c0cfd50 0x7ff72c14e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.765+0000 7ff7318a7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff72c0d0160 0x7ff72c14c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.765+0000 7ff7318a7640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff72c157200 0x7ff72c14ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.765+0000 7ff7318a7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c0ccb70 con 0x7ff72c0d0160 2024-04-25T19:16:36.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.765+0000 7ff7318a7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c15da20 con 0x7ff72c0cfd50 2024-04-25T19:16:36.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.765+0000 7ff7318a7640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff72c0cccf0 con 0x7ff72c157200 2024-04-25T19:16:36.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.765+0000 7ff72a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff72c0d0160 0x7ff72c14c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.765+0000 7ff72a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff72c0d0160 0x7ff72c14c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46110/0 (socket says 172.21.15.76:46110) 2024-04-25T19:16:36.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.765+0000 7ff72a7fc640 1 -- 172.21.15.76:0/2440820142 learned_addr learned my addr 172.21.15.76:0/2440820142 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:36.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.765+0000 7ff72affd640 1 --2- 172.21.15.76:0/2440820142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff72c0cfd50 0x7ff72c14e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.765+0000 7ff72b7fe640 1 --2- 172.21.15.76:0/2440820142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff72c157200 0x7ff72c14ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.765+0000 7ff72a7fc640 1 -- 172.21.15.76:0/2440820142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff72c157200 msgr2=0x7ff72c14ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:36.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.765+0000 7ff72a7fc640 1 --2- 172.21.15.76:0/2440820142 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff72c157200 0x7ff72c14ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.765+0000 7ff72a7fc640 1 -- 172.21.15.76:0/2440820142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff72c0cfd50 msgr2=0x7ff72c14e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:36.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.765+0000 7ff72a7fc640 1 --2- 172.21.15.76:0/2440820142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff72c0cfd50 0x7ff72c14e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.765+0000 7ff72a7fc640 1 -- 172.21.15.76:0/2440820142 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72c0c8af0 con 0x7ff72c0d0160 2024-04-25T19:16:36.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.765+0000 7ff72a7fc640 1 --2- 172.21.15.76:0/2440820142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff72c0d0160 0x7ff72c14c4c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff72002a2b0 tx=0x7ff7200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:36.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.765+0000 7ff713fff640 1 -- 172.21.15.76:0/2440820142 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720041020 con 0x7ff72c0d0160 2024-04-25T19:16:36.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.765+0000 7ff713fff640 1 -- 172.21.15.76:0/2440820142 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7200306c0 con 0x7ff72c0d0160 2024-04-25T19:16:36.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.766+0000 7ff7318a7640 1 -- 172.21.15.76:0/2440820142 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff72c14e9a0 con 0x7ff72c0d0160 2024-04-25T19:16:36.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.766+0000 7ff713fff640 1 -- 172.21.15.76:0/2440820142 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720037040 con 0x7ff72c0d0160 2024-04-25T19:16:36.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.766+0000 7ff7318a7640 1 -- 172.21.15.76:0/2440820142 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff72c1f4290 con 0x7ff72c0d0160 2024-04-25T19:16:36.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.766+0000 7ff7318a7640 1 --2- 172.21.15.76:0/2440820142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff72c15c950 0x7ff72c1f3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.766+0000 7ff7318a7640 1 -- 172.21.15.76:0/2440820142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c14ee70 con 0x7ff72c15c950 2024-04-25T19:16:36.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.767+0000 7ff72affd640 1 --2- 172.21.15.76:0/2440820142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff72c15c950 0x7ff72c1f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.767+0000 7ff72affd640 1 -- 172.21.15.76:0/2440820142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff72c14e180 con 0x7ff72c15c950 2024-04-25T19:16:36.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.767+0000 7ff72affd640 1 --2- 172.21.15.76:0/2440820142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff72c15c950 0x7ff72c1f3960 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff71c009bc0 tx=0x7ff71c007700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:36.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.767+0000 7ff713fff640 1 -- 172.21.15.76:0/2440820142 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff720044020 con 0x7ff72c0d0160 2024-04-25T19:16:36.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.768+0000 7ff713fff640 1 --2- 172.21.15.76:0/2440820142 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff7180744b0 0x7ff7180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.768+0000 7ff713fff640 1 -- 172.21.15.76:0/2440820142 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7ff7200b5ac0 con 0x7ff72c0d0160 2024-04-25T19:16:36.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.768+0000 7ff72b7fe640 1 --2- 172.21.15.76:0/2440820142 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff7180744b0 0x7ff7180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.768+0000 7ff713fff640 1 -- 172.21.15.76:0/2440820142 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c014610 con 0x7ff72c15c950 2024-04-25T19:16:36.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.768+0000 7ff713fff640 1 -- 172.21.15.76:0/2440820142 <== mon.1 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff71c019070 con 0x7ff72c15c950 2024-04-25T19:16:36.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.768+0000 7ff713fff640 1 -- 172.21.15.76:0/2440820142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff72c15c950 msgr2=0x7ff72c1f3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:36.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.768+0000 7ff713fff640 1 --2- 172.21.15.76:0/2440820142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff72c15c950 0x7ff72c1f3960 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff71c009bc0 tx=0x7ff71c007700 comp rx=0 tx=0).stop 2024-04-25T19:16:36.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.768+0000 7ff713fff640 1 -- 172.21.15.76:0/2440820142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff72c15c950 msgr2=0x7ff72c1f3960 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:16:36.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.768+0000 7ff713fff640 1 --2- 172.21.15.76:0/2440820142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff72c15c950 0x7ff72c1f3960 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.768+0000 7ff72b7fe640 1 --2- 172.21.15.76:0/2440820142 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff7180744b0 0x7ff7180769c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0d0640 tx=0x7ff724008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:36.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.774+0000 7ff7318a7640 1 --2- 172.21.15.76:0/2440820142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff72c14faa0 0x7ff72c14feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.774+0000 7ff7318a7640 1 -- 172.21.15.76:0/2440820142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c0c8e70 con 0x7ff72c14faa0 2024-04-25T19:16:36.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.775+0000 7ff72affd640 1 --2- 172.21.15.76:0/2440820142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff72c14faa0 0x7ff72c14feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.775+0000 7ff72affd640 1 -- 172.21.15.76:0/2440820142 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff72c1f33f0 con 0x7ff72c14faa0 2024-04-25T19:16:36.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.775+0000 7ff72affd640 1 --2- 172.21.15.76:0/2440820142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff72c14faa0 0x7ff72c14feb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff71c018000 tx=0x7ff71c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:36.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.776+0000 7ff713fff640 1 -- 172.21.15.76:0/2440820142 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c028640 con 0x7ff72c14faa0 2024-04-25T19:16:36.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.776+0000 7ff713fff640 1 -- 172.21.15.76:0/2440820142 <== mon.1 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4894 (secure 0 0 0) 0x7ff71c028640 con 0x7ff72c14faa0 2024-04-25T19:16:36.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.776+0000 7ff713fff640 1 -- 172.21.15.76:0/2440820142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff72c14faa0 msgr2=0x7ff72c14feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:36.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.776+0000 7ff713fff640 1 --2- 172.21.15.76:0/2440820142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff72c14faa0 0x7ff72c14feb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff71c018000 tx=0x7ff71c004a40 comp rx=0 tx=0).stop 2024-04-25T19:16:36.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.776+0000 7ff713fff640 1 -- 172.21.15.76:0/2440820142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff72c14faa0 msgr2=0x7ff72c14feb0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:16:36.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.776+0000 7ff713fff640 1 --2- 172.21.15.76:0/2440820142 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff72c14faa0 0x7ff72c14feb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.778 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:16:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:16:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:16:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "peon", 2024-04-25T19:16:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 10, 2024-04-25T19:16:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 95258, 2024-04-25T19:16:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:16:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:16:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-04-25T19:16:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:16:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 0, 2024-04-25T19:16:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:16:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:16:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:16:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:16:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:16:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:16:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:16:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:16:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:16:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:16:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:16:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:16:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:16:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:16:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:16:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:16:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:16:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:16:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:16:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:16:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:16:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:16:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:16:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:16:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:16:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:16:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:16:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:16:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:16:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:16:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:16:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:16:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:16:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:16:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:16:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:16:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:16:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:16:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:16:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:16:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:16:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:16:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:16:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:16:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:16:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": [ 2024-04-25T19:16:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 4 2024-04-25T19:16:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:16:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 3 2024-04-25T19:16:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": [ 2024-04-25T19:16:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:16:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:16:36.790 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:16:36.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.776+0000 7ff7318a7640 1 -- 172.21.15.76:0/2440820142 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff7180744b0 msgr2=0x7ff7180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:36.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.776+0000 7ff7318a7640 1 --2- 172.21.15.76:0/2440820142 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff7180744b0 0x7ff7180769c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0d0640 tx=0x7ff724008040 comp rx=0 tx=0).stop 2024-04-25T19:16:36.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.776+0000 7ff7318a7640 1 -- 172.21.15.76:0/2440820142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff72c0d0160 msgr2=0x7ff72c14c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:36.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.776+0000 7ff7318a7640 1 --2- 172.21.15.76:0/2440820142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff72c0d0160 0x7ff72c14c4c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff72002a2b0 tx=0x7ff7200046c0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.776+0000 7ff72b7fe640 1 -- 172.21.15.76:0/2440820142 reap_dead start 2024-04-25T19:16:36.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.776+0000 7ff72b7fe640 1 -- 172.21.15.76:0/2440820142 reap_dead start 2024-04-25T19:16:36.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.776+0000 7ff7318a7640 1 -- 172.21.15.76:0/2440820142 shutdown_connections 2024-04-25T19:16:36.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.776+0000 7ff7318a7640 1 -- 172.21.15.76:0/2440820142 >> 172.21.15.76:0/2440820142 conn(0x7ff72c0d2370 msgr2=0x7ff72c1558d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:36.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.776+0000 7ff7318a7640 1 -- 172.21.15.76:0/2440820142 shutdown_connections 2024-04-25T19:16:36.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.776+0000 7ff7318a7640 1 -- 172.21.15.76:0/2440820142 wait complete. 2024-04-25T19:16:36.792 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T19:16:36.793 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T19:16:36.793 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T19:16:36.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.900+0000 7f5b2ef9d640 1 Processor -- start 2024-04-25T19:16:36.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.900+0000 7f5b2ef9d640 1 -- start start 2024-04-25T19:16:36.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.902+0000 7f5b2ef9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5b280b76d0 0x7f5b280cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.902+0000 7f5b2ef9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280ce040 0x7f5b280ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.902+0000 7f5b2ef9d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b280c9d90 con 0x7f5b280cd2d0 2024-04-25T19:16:36.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.902+0000 7f5b2ef9d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5b2815fe20 con 0x7f5b280b76d0 2024-04-25T19:16:36.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.902+0000 7f5b2ef9d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f5b28160100 con 0x7f5b280ce040 2024-04-25T19:16:36.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.902+0000 7f5b2d79a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5b280b76d0 0x7f5b280cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.902+0000 7f5b2d79a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5b280b76d0 0x7f5b280cdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46126/0 (socket says 172.21.15.76:46126) 2024-04-25T19:16:36.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.902+0000 7f5b2d79a640 1 -- 172.21.15.76:0/966392376 learned_addr learned my addr 172.21.15.76:0/966392376 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:36.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.902+0000 7f5b2e79c640 1 --2- 172.21.15.76:0/966392376 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280ce040 0x7f5b280ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.902+0000 7f5b2d79a640 1 -- 172.21.15.76:0/966392376 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280ce040 msgr2=0x7f5b280ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:36.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.902+0000 7f5b2d79a640 1 --2- 172.21.15.76:0/966392376 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280ce040 0x7f5b280ca2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.902+0000 7f5b2d79a640 1 -- 172.21.15.76:0/966392376 >> v1:172.21.15.105:6789/0 conn(0x7f5b280cd2d0 legacy=0x7f5b28153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:36.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.902+0000 7f5b2d79a640 1 -- 172.21.15.76:0/966392376 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b281603e0 con 0x7f5b280b76d0 2024-04-25T19:16:36.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.903+0000 7f5b2d79a640 1 --2- 172.21.15.76:0/966392376 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b280b76d0 0x7f5b280cdb00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009870 tx=0x7f5b1802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3cce37db1cb20b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:36.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.903+0000 7f5b2cf99640 1 -- 172.21.15.76:0/966392376 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b18037070 con 0x7f5b280b76d0 2024-04-25T19:16:36.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.903+0000 7f5b2cf99640 1 -- 172.21.15.76:0/966392376 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b1802ae50 con 0x7f5b280b76d0 2024-04-25T19:16:36.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.903+0000 7f5b2cf99640 1 -- 172.21.15.76:0/966392376 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b18032860 con 0x7f5b280b76d0 2024-04-25T19:16:36.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.904+0000 7f5b2ef9d640 1 -- 172.21.15.76:0/966392376 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b280b76d0 msgr2=0x7f5b280cdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:36.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.904+0000 7f5b2ef9d640 1 --2- 172.21.15.76:0/966392376 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b280b76d0 0x7f5b280cdb00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009870 tx=0x7f5b1802a300 comp rx=0 tx=0).stop 2024-04-25T19:16:36.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.904+0000 7f5b2ef9d640 1 -- 172.21.15.76:0/966392376 shutdown_connections 2024-04-25T19:16:36.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.904+0000 7f5b2ef9d640 1 --2- 172.21.15.76:0/966392376 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280ce040 0x7f5b280ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.904+0000 7f5b2ef9d640 1 --2- 172.21.15.76:0/966392376 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b280b76d0 0x7f5b280cdb00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.904+0000 7f5b2ef9d640 1 -- 172.21.15.76:0/966392376 >> 172.21.15.76:0/966392376 conn(0x7f5b280d1cc0 msgr2=0x7f5b280d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:36.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.904+0000 7f5b2ef9d640 1 -- 172.21.15.76:0/966392376 shutdown_connections 2024-04-25T19:16:36.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.904+0000 7f5b2ef9d640 1 -- 172.21.15.76:0/966392376 wait complete. 2024-04-25T19:16:36.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.905+0000 7f5b2ef9d640 1 Processor -- start 2024-04-25T19:16:36.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.905+0000 7f5b2ef9d640 1 -- start start 2024-04-25T19:16:36.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.907+0000 7f5b2ef9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b280b76d0 0x7f5b280d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.907+0000 7f5b2ef9d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cd2d0 0x7f5b281f6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.907+0000 7f5b2ef9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b280ce040 0x7f5b281f6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.907+0000 7f5b2ef9d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5b28160c80 con 0x7f5b280ce040 2024-04-25T19:16:36.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.907+0000 7f5b2ef9d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5b281609a0 con 0x7f5b280b76d0 2024-04-25T19:16:36.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.907+0000 7f5b2ef9d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f5b280cd6e0 con 0x7f5b280cd2d0 2024-04-25T19:16:36.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.907+0000 7f5b2e79c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b280ce040 0x7f5b281f6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.907+0000 7f5b2e79c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b280ce040 0x7f5b281f6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46132/0 (socket says 172.21.15.76:46132) 2024-04-25T19:16:36.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.907+0000 7f5b2e79c640 1 -- 172.21.15.76:0/789173891 learned_addr learned my addr 172.21.15.76:0/789173891 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:36.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.907+0000 7f5b2df9b640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b280b76d0 0x7f5b280d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.907+0000 7f5b2d79a640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cd2d0 0x7f5b281f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.907+0000 7f5b2d79a640 1 -- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cd2d0 msgr2=0x7f5b281f6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T19:16:36.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.907+0000 7f5b2d79a640 1 -- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cd2d0 msgr2=0x7f5b281f6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T19:16:36.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.907+0000 7f5b2d79a640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cd2d0 0x7f5b281f6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:16:36.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.907+0000 7f5b2d79a640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cd2d0 0x7f5b281f6570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:36.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.907+0000 7f5b2e79c640 1 -- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cd2d0 msgr2=0x7f5b281f6570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:16:36.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.907+0000 7f5b2e79c640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cd2d0 0x7f5b281f6570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.907+0000 7f5b2e79c640 1 -- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b280b76d0 msgr2=0x7f5b280d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:36.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.907+0000 7f5b2e79c640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b280b76d0 0x7f5b280d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:36.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.907+0000 7f5b2e79c640 1 -- 172.21.15.76:0/789173891 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b280d0b30 con 0x7f5b280ce040 2024-04-25T19:16:36.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.907+0000 7f5b2e79c640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b280ce040 0x7f5b281f6ab0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000d600 tx=0x7f5b2000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:36.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.907+0000 7f5b1e7fc640 1 -- 172.21.15.76:0/789173891 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b200125f0 con 0x7f5b280ce040 2024-04-25T19:16:36.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.907+0000 7f5b1e7fc640 1 -- 172.21.15.76:0/789173891 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b20019070 con 0x7f5b280ce040 2024-04-25T19:16:36.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.907+0000 7f5b2ef9d640 1 -- 172.21.15.76:0/789173891 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b2814ec20 con 0x7f5b280ce040 2024-04-25T19:16:36.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.907+0000 7f5b1e7fc640 1 -- 172.21.15.76:0/789173891 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b200129f0 con 0x7f5b280ce040 2024-04-25T19:16:36.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.907+0000 7f5b2ef9d640 1 -- 172.21.15.76:0/789173891 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b280cfdf0 con 0x7f5b280ce040 2024-04-25T19:16:36.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.908+0000 7f5b2ef9d640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 0x7f5b280c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.908+0000 7f5b2ef9d640 1 -- 172.21.15.76:0/789173891 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f5b281532a0 con 0x7f5b280cffc0 2024-04-25T19:16:36.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.909+0000 7f5b2df9b640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 0x7f5b280c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.909+0000 7f5b2df9b640 1 -- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 msgr2=0x7f5b280c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-04-25T19:16:36.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.909+0000 7f5b2df9b640 1 -- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 msgr2=0x7f5b280c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T19:16:36.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.909+0000 7f5b2df9b640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 0x7f5b280c4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:16:36.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.909+0000 7f5b2df9b640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 0x7f5b280c4cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:36.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.909+0000 7f5b1e7fc640 1 -- 172.21.15.76:0/789173891 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5b20012b90 con 0x7f5b280ce040 2024-04-25T19:16:36.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.910+0000 7f5b1e7fc640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5b0c0744e0 0x7f5b0c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:36.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.910+0000 7f5b1e7fc640 1 -- 172.21.15.76:0/789173891 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f5b20095560 con 0x7f5b280ce040 2024-04-25T19:16:36.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.910+0000 7f5b2d79a640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5b0c0744e0 0x7f5b0c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:36.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:36.910+0000 7f5b2d79a640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5b0c0744e0 0x7f5b0c0769f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5b18034000 tx=0x7f5b1802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:37.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:37.110+0000 7f5b2df9b640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 0x7f5b280c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:37.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:37.110+0000 7f5b2df9b640 1 -- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 msgr2=0x7f5b280c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-04-25T19:16:37.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:37.110+0000 7f5b2df9b640 1 -- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 msgr2=0x7f5b280c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T19:16:37.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:37.110+0000 7f5b2df9b640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 0x7f5b280c4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:16:37.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:37.110+0000 7f5b2df9b640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 0x7f5b280c4cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T19:16:37.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:37.511+0000 7f5b2df9b640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 0x7f5b280c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:37.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:37.511+0000 7f5b2df9b640 1 -- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 msgr2=0x7f5b280c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-04-25T19:16:37.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:37.511+0000 7f5b2df9b640 1 -- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 msgr2=0x7f5b280c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T19:16:37.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:37.511+0000 7f5b2df9b640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 0x7f5b280c4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:16:37.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:37.511+0000 7f5b2df9b640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 0x7f5b280c4cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T19:16:38.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:38.312+0000 7f5b2df9b640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 0x7f5b280c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:38.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:38.312+0000 7f5b2df9b640 1 -- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 msgr2=0x7f5b280c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-04-25T19:16:38.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:38.312+0000 7f5b2df9b640 1 -- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 msgr2=0x7f5b280c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T19:16:38.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:38.312+0000 7f5b2df9b640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 0x7f5b280c4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:16:38.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:38.312+0000 7f5b2df9b640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 0x7f5b280c4cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T19:16:39.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:39.914+0000 7f5b2df9b640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 0x7f5b280c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:39.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:39.915+0000 7f5b2df9b640 1 -- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 msgr2=0x7f5b280c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-04-25T19:16:39.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:39.915+0000 7f5b2df9b640 1 -- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 msgr2=0x7f5b280c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T19:16:39.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:39.915+0000 7f5b2df9b640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 0x7f5b280c4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:16:39.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:39.915+0000 7f5b2df9b640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 0x7f5b280c4cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T19:16:40.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:40.907+0000 7f5b1d7fa640 1 -- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 msgr2=0x7f5b280c4cb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:16:40.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:40.907+0000 7f5b1d7fa640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 0x7f5b280c4cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:40.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:40.907+0000 7f5b1d7fa640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b04003540 0x7f5b04005a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:40.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:40.907+0000 7f5b1d7fa640 1 -- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 msgr2=0x7f5b280c4cb0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:16:40.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:40.907+0000 7f5b1d7fa640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280cffc0 0x7f5b280c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:40.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:40.907+0000 7f5b1d7fa640 1 -- 172.21.15.76:0/789173891 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f5b040060d0 con 0x7f5b04003540 2024-04-25T19:16:40.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:40.908+0000 7f5b2df9b640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b04003540 0x7f5b04005a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:40.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:40.908+0000 7f5b2df9b640 1 -- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b04003540 msgr2=0x7f5b04005a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-04-25T19:16:40.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:40.908+0000 7f5b2df9b640 1 -- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b04003540 msgr2=0x7f5b04005a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T19:16:40.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:40.908+0000 7f5b2df9b640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b04003540 0x7f5b04005a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:16:40.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:40.908+0000 7f5b2df9b640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b04003540 0x7f5b04005a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:41.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.109+0000 7f5b2df9b640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b04003540 0x7f5b04005a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:41.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.109+0000 7f5b2df9b640 1 -- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b04003540 msgr2=0x7f5b04005a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-04-25T19:16:41.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.109+0000 7f5b2df9b640 1 -- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b04003540 msgr2=0x7f5b04005a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T19:16:41.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.109+0000 7f5b2df9b640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b04003540 0x7f5b04005a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:16:41.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.109+0000 7f5b2df9b640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b04003540 0x7f5b04005a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T19:16:41.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.510+0000 7f5b2df9b640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b04003540 0x7f5b04005a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:41.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.510+0000 7f5b2df9b640 1 -- 172.21.15.76:0/789173891 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5b04006280 con 0x7f5b04003540 2024-04-25T19:16:41.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.511+0000 7f5b2df9b640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b04003540 0x7f5b04005a20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5b24001ac0 tx=0x7f5b24001f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:41.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.511+0000 7f5b1e7fc640 1 -- 172.21.15.76:0/789173891 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b240020b0 con 0x7f5b04003540 2024-04-25T19:16:41.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.511+0000 7f5b1e7fc640 1 -- 172.21.15.76:0/789173891 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5b24017bc0 con 0x7f5b04003540 2024-04-25T19:16:41.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.511+0000 7f5b1e7fc640 1 -- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b04003540 msgr2=0x7f5b04005a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:41.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.511+0000 7f5b1e7fc640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b04003540 0x7f5b04005a20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5b24001ac0 tx=0x7f5b24001f80 comp rx=0 tx=0).stop 2024-04-25T19:16:41.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.511+0000 7f5b1e7fc640 1 -- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b04003540 msgr2=0x7f5b04005a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:16:41.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.511+0000 7f5b1e7fc640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b04003540 0x7f5b04005a20 secure :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5b24001ac0 tx=0x7f5b24001f80 comp rx=0 tx=0).stop 2024-04-25T19:16:41.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.518+0000 7f5b2ef9d640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280d0d00 0x7f5b2814a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:41.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.519+0000 7f5b2ef9d640 1 -- 172.21.15.76:0/789173891 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f5b28050740 con 0x7f5b280d0d00 2024-04-25T19:16:41.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.519+0000 7f5b2df9b640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280d0d00 0x7f5b2814a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:41.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.519+0000 7f5b2df9b640 1 -- 172.21.15.76:0/789173891 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5b280cb1b0 con 0x7f5b280d0d00 2024-04-25T19:16:41.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.520+0000 7f5b2df9b640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280d0d00 0x7f5b2814a760 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5b24001ac0 tx=0x7f5b24000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:41.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.520+0000 7f5b1e7fc640 1 -- 172.21.15.76:0/789173891 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b24017e50 con 0x7f5b280d0d00 2024-04-25T19:16:41.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.520+0000 7f5b1e7fc640 1 -- 172.21.15.76:0/789173891 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4553 (secure 0 0 0) 0x7f5b24021680 con 0x7f5b280d0d00 2024-04-25T19:16:41.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.520+0000 7f5b1e7fc640 1 -- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280d0d00 msgr2=0x7f5b2814a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:41.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.520+0000 7f5b1e7fc640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280d0d00 0x7f5b2814a760 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5b24001ac0 tx=0x7f5b24000f40 comp rx=0 tx=0).stop 2024-04-25T19:16:41.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.520+0000 7f5b1e7fc640 1 -- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280d0d00 msgr2=0x7f5b2814a760 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:16:41.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.520+0000 7f5b1e7fc640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5b280d0d00 0x7f5b2814a760 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:41.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.520+0000 7f5b2e79c640 1 -- 172.21.15.76:0/789173891 reap_dead start 2024-04-25T19:16:41.522 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:16:41.523 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:16:41.523 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:16:41.523 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "peon", 2024-04-25T19:16:41.523 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 10, 2024-04-25T19:16:41.523 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 5298, 2024-04-25T19:16:41.523 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:16:41.523 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:16:41.523 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-04-25T19:16:41.523 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:16:41.523 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:41.523 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 5, 2024-04-25T19:16:41.523 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:16:41.523 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:16:41.523 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:16:41.523 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:41.524 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:41.524 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:41.524 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:41.524 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:41.524 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:41.524 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:41.524 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:41.524 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:41.524 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:41.524 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:41.524 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:41.524 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:16:41.524 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:16:41.524 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:41.524 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:41.524 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:41.525 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:41.525 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:41.525 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:41.525 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:41.525 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:41.525 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:41.525 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:41.525 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:41.525 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:41.525 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:41.525 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:16:41.525 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:16:41.525 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:16:41.525 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:16:41.526 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:16:41.526 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:16:41.526 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:16:41.526 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:16:41.526 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:16:41.526 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:16:41.526 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:16:41.526 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:16:41.526 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:16:41.526 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:16:41.526 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:16:41.526 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:16:41.526 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:16:41.526 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:16:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:16:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:41.527 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:16:41.528 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:16:41.528 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:41.528 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:41.528 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:41.528 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:41.528 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:16:41.528 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:41.528 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:41.528 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:41.528 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:41.528 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:16:41.528 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:41.528 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:41.528 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:41.528 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:41.528 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:16:41.529 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:16:41.529 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:41.529 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:41.529 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:41.529 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:41.529 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:41.529 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:16:41.529 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:16:41.529 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:41.529 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:41.529 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:41.529 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:41.529 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:16:41.529 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:41.529 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:41.529 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:41.529 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:41.530 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:16:41.530 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:41.530 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:41.530 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:41.530 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:41.530 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:16:41.530 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:16:41.530 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:41.530 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:41.530 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:41.530 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:41.530 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:41.530 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:16:41.530 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:16:41.531 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:41.531 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:41.531 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:41.531 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:41.531 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:16:41.531 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:41.531 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:41.531 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:41.531 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:41.531 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:16:41.531 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:41.531 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:41.531 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:41.531 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:41.532 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:16:41.532 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:16:41.532 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:41.532 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:41.532 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:41.532 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:41.532 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:41.532 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:41.532 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:16:41.532 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:16:41.532 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:41.532 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:41.532 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:41.532 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:16:41.532 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:41.533 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:41.533 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:16:41.533 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:41.533 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:41.533 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:41.533 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:16:41.533 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:41.533 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:41.533 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:41.533 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:16:41.533 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:16:41.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.521+0000 7f5b2ef9d640 1 -- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5b0c0744e0 msgr2=0x7f5b0c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:41.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.521+0000 7f5b2ef9d640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5b0c0744e0 0x7f5b0c0769f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5b18034000 tx=0x7f5b1802e000 comp rx=0 tx=0).stop 2024-04-25T19:16:41.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.521+0000 7f5b2ef9d640 1 -- 172.21.15.76:0/789173891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b280ce040 msgr2=0x7f5b281f6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:41.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.521+0000 7f5b2ef9d640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b280ce040 0x7f5b281f6ab0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000d600 tx=0x7f5b2000dac0 comp rx=0 tx=0).stop 2024-04-25T19:16:41.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.521+0000 7f5b2ef9d640 1 -- 172.21.15.76:0/789173891 shutdown_connections 2024-04-25T19:16:41.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.521+0000 7f5b2ef9d640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5b0c0744e0 0x7f5b0c0769f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:41.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.521+0000 7f5b2ef9d640 1 --2- 172.21.15.76:0/789173891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b280ce040 0x7f5b281f6ab0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:41.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.521+0000 7f5b2ef9d640 1 -- 172.21.15.76:0/789173891 >> 172.21.15.76:0/789173891 conn(0x7f5b280d1cc0 msgr2=0x7f5b2806d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:41.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.521+0000 7f5b2ef9d640 1 -- 172.21.15.76:0/789173891 shutdown_connections 2024-04-25T19:16:41.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.521+0000 7f5b2ef9d640 1 -- 172.21.15.76:0/789173891 wait complete. 2024-04-25T19:16:41.537 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T19:16:41.537 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-25T19:16:41.537 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-25T19:16:41.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.655+0000 7f346a564640 1 Processor -- start 2024-04-25T19:16:41.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.655+0000 7f346a564640 1 -- start start 2024-04-25T19:16:41.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.656+0000 7f346a564640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f34640d0810 0x7f346406f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:41.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.656+0000 7f346a564640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34640cea70 0x7f34640cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:41.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.656+0000 7f346a564640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f346415ce80 con 0x7f346406e160 2024-04-25T19:16:41.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.656+0000 7f346a564640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f346415d160 con 0x7f34640d0810 2024-04-25T19:16:41.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.656+0000 7f346a564640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f346415d440 con 0x7f34640cea70 2024-04-25T19:16:41.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.656+0000 7f3463fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f34640d0810 0x7f346406f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:41.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.656+0000 7f3463fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f34640d0810 0x7f346406f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52808/0 (socket says 172.21.15.76:52808) 2024-04-25T19:16:41.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.656+0000 7f3463fff640 1 -- 172.21.15.76:0/3346767921 learned_addr learned my addr 172.21.15.76:0/3346767921 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:41.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.657+0000 7f34637fe640 1 --2- 172.21.15.76:0/3346767921 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34640cea70 0x7f34640cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:41.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.657+0000 7f3463fff640 1 -- 172.21.15.76:0/3346767921 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34640cea70 msgr2=0x7f34640cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:41.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.657+0000 7f3463fff640 1 --2- 172.21.15.76:0/3346767921 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34640cea70 0x7f34640cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:41.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.657+0000 7f3463fff640 1 -- 172.21.15.76:0/3346767921 >> v1:172.21.15.76:6789/0 conn(0x7f346406e160 legacy=0x7f346406e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:41.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.657+0000 7f3463fff640 1 -- 172.21.15.76:0/3346767921 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f346415d720 con 0x7f34640d0810 2024-04-25T19:16:41.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.658+0000 7f3463fff640 1 --2- 172.21.15.76:0/3346767921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34640d0810 0x7f346406f000 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3454009870 tx=0x7f345402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d27a344f60242f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:41.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.658+0000 7f3462ffd640 1 -- 172.21.15.76:0/3346767921 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454037070 con 0x7f34640d0810 2024-04-25T19:16:41.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.658+0000 7f3462ffd640 1 -- 172.21.15.76:0/3346767921 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3454032440 con 0x7f34640d0810 2024-04-25T19:16:41.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.658+0000 7f3462ffd640 1 -- 172.21.15.76:0/3346767921 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454032810 con 0x7f34640d0810 2024-04-25T19:16:41.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.658+0000 7f346a564640 1 -- 172.21.15.76:0/3346767921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34640d0810 msgr2=0x7f346406f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:41.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.658+0000 7f346a564640 1 --2- 172.21.15.76:0/3346767921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34640d0810 0x7f346406f000 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3454009870 tx=0x7f345402a270 comp rx=0 tx=0).stop 2024-04-25T19:16:41.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.658+0000 7f346a564640 1 -- 172.21.15.76:0/3346767921 shutdown_connections 2024-04-25T19:16:41.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.658+0000 7f346a564640 1 --2- 172.21.15.76:0/3346767921 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f34640cea70 0x7f34640cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:41.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.658+0000 7f346a564640 1 --2- 172.21.15.76:0/3346767921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34640d0810 0x7f346406f000 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:41.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.658+0000 7f346a564640 1 -- 172.21.15.76:0/3346767921 >> 172.21.15.76:0/3346767921 conn(0x7f34640b3c00 msgr2=0x7f34640b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:41.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.659+0000 7f346a564640 1 -- 172.21.15.76:0/3346767921 shutdown_connections 2024-04-25T19:16:41.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.659+0000 7f346a564640 1 -- 172.21.15.76:0/3346767921 wait complete. 2024-04-25T19:16:41.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.661+0000 7f346a564640 1 Processor -- start 2024-04-25T19:16:41.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.661+0000 7f346a564640 1 -- start start 2024-04-25T19:16:41.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.663+0000 7f346a564640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f346406e160 0x7f34641f6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:41.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.663+0000 7f346a564640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34640cea70 0x7f346414d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:41.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.663+0000 7f346a564640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34640d0810 0x7f34641e64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.663+0000 7f346a564640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f346415cb70 con 0x7f34640cea70 2024-04-25T19:16:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.663+0000 7f346a564640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f346406dc40 con 0x7f34640d0810 2024-04-25T19:16:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.663+0000 7f346a564640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f346406f540 con 0x7f346406e160 2024-04-25T19:16:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.663+0000 7f34637fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34640cea70 0x7f346414d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.663+0000 7f34637fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34640cea70 0x7f346414d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52820/0 (socket says 172.21.15.76:52820) 2024-04-25T19:16:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.663+0000 7f34637fe640 1 -- 172.21.15.76:0/4134178073 learned_addr learned my addr 172.21.15.76:0/4134178073 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.663+0000 7f3463fff640 1 --2- 172.21.15.76:0/4134178073 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f346406e160 0x7f34641f6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.663+0000 7f3468b01640 1 --2- 172.21.15.76:0/4134178073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34640d0810 0x7f34641e64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.663+0000 7f34637fe640 1 -- 172.21.15.76:0/4134178073 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f346406e160 msgr2=0x7f34641f6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.663+0000 7f34637fe640 1 --2- 172.21.15.76:0/4134178073 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f346406e160 0x7f34641f6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.663+0000 7f34637fe640 1 -- 172.21.15.76:0/4134178073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34640d0810 msgr2=0x7f34641e64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.663+0000 7f34637fe640 1 --2- 172.21.15.76:0/4134178073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34640d0810 0x7f34641e64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.663+0000 7f34637fe640 1 -- 172.21.15.76:0/4134178073 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3464160a90 con 0x7f34640cea70 2024-04-25T19:16:41.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.663+0000 7f3468b01640 1 --2- 172.21.15.76:0/4134178073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34640d0810 0x7f34641e64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:16:41.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.663+0000 7f34637fe640 1 --2- 172.21.15.76:0/4134178073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34640cea70 0x7f346414d750 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f345800e8a0 tx=0x7f345800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:41.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.663+0000 7f3460ff9640 1 -- 172.21.15.76:0/4134178073 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3458017070 con 0x7f34640cea70 2024-04-25T19:16:41.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.663+0000 7f3460ff9640 1 -- 172.21.15.76:0/4134178073 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f345800c8e0 con 0x7f34640cea70 2024-04-25T19:16:41.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.663+0000 7f346a564640 1 -- 172.21.15.76:0/4134178073 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3464160d70 con 0x7f34640cea70 2024-04-25T19:16:41.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.663+0000 7f3460ff9640 1 -- 172.21.15.76:0/4134178073 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345800ccd0 con 0x7f34640cea70 2024-04-25T19:16:41.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.664+0000 7f346a564640 1 -- 172.21.15.76:0/4134178073 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3464155f70 con 0x7f34640cea70 2024-04-25T19:16:41.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.664+0000 7f346a564640 1 -- 172.21.15.76:0/4134178073 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3464160800 con 0x7f34640cea70 2024-04-25T19:16:41.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.665+0000 7f3460ff9640 1 -- 172.21.15.76:0/4134178073 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f345801a070 con 0x7f34640cea70 2024-04-25T19:16:41.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.666+0000 7f3460ff9640 1 --2- 172.21.15.76:0/4134178073 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f34500744b0 0x7f34500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:41.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.666+0000 7f3460ff9640 1 -- 172.21.15.76:0/4134178073 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f34580950b0 con 0x7f34640cea70 2024-04-25T19:16:41.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.666+0000 7f3463fff640 1 --2- 172.21.15.76:0/4134178073 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f34500744b0 0x7f34500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:41.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.666+0000 7f3463fff640 1 --2- 172.21.15.76:0/4134178073 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f34500744b0 0x7f34500769c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3454034000 tx=0x7f345402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:41.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.675+0000 7f3460ff9640 1 -- 172.21.15.76:0/4134178073 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f345805f850 con 0x7f34640cea70 2024-04-25T19:16:41.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.939+0000 7f346a564640 1 -- 172.21.15.76:0/4134178073 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f346406a390 con 0x7f34640cea70 2024-04-25T19:16:41.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.940+0000 7f3460ff9640 1 -- 172.21.15.76:0/4134178073 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f345805f150 con 0x7f34640cea70 2024-04-25T19:16:41.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.943+0000 7f346a564640 1 -- 172.21.15.76:0/4134178073 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f34500744b0 msgr2=0x7f34500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:41.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.943+0000 7f346a564640 1 --2- 172.21.15.76:0/4134178073 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f34500744b0 0x7f34500769c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3454034000 tx=0x7f345402e000 comp rx=0 tx=0).stop 2024-04-25T19:16:41.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.943+0000 7f346a564640 1 -- 172.21.15.76:0/4134178073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34640cea70 msgr2=0x7f346414d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:41.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.943+0000 7f346a564640 1 --2- 172.21.15.76:0/4134178073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34640cea70 0x7f346414d750 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f345800e8a0 tx=0x7f345800ed60 comp rx=0 tx=0).stop 2024-04-25T19:16:41.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.943+0000 7f346a564640 1 -- 172.21.15.76:0/4134178073 shutdown_connections 2024-04-25T19:16:41.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.943+0000 7f346a564640 1 --2- 172.21.15.76:0/4134178073 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f34500744b0 0x7f34500769c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:41.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.943+0000 7f346a564640 1 --2- 172.21.15.76:0/4134178073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34640d0810 0x7f34641e64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:41.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.943+0000 7f346a564640 1 --2- 172.21.15.76:0/4134178073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34640cea70 0x7f346414d750 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:41.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.943+0000 7f346a564640 1 --2- 172.21.15.76:0/4134178073 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f346406e160 0x7f34641f6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:41.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.943+0000 7f346a564640 1 -- 172.21.15.76:0/4134178073 >> 172.21.15.76:0/4134178073 conn(0x7f34640b3c00 msgr2=0x7f34640ca430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:41.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.943+0000 7f346a564640 1 -- 172.21.15.76:0/4134178073 shutdown_connections 2024-04-25T19:16:41.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:41.943+0000 7f346a564640 1 -- 172.21.15.76:0/4134178073 wait complete. 2024-04-25T19:16:41.956 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-04-25T19:16:51.957 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T19:16:51.957 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:16:52.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.068+0000 7f8c09372640 1 Processor -- start 2024-04-25T19:16:52.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.068+0000 7f8c09372640 1 -- start start 2024-04-25T19:16:52.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.069+0000 7f8c09372640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c040c7750 0x7f8c04153590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.069+0000 7f8c09372640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c040c8020 con 0x7f8c040c7b60 2024-04-25T19:16:52.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.070+0000 7f8c09372640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c041598a0 con 0x7f8c0415cd80 2024-04-25T19:16:52.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.070+0000 7f8c09372640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8c04159b40 con 0x7f8c040c7750 2024-04-25T19:16:52.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.070+0000 7f8c027fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8c040c7b60 0x7f8c040cdd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:56760/0 (socket says 172.21.15.76:56760) 2024-04-25T19:16:52.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.070+0000 7f8c027fc640 1 -- 172.21.15.76:0/2756761808 learned_addr learned my addr 172.21.15.76:0/2756761808 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:16:52.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.070+0000 7f8c02ffd640 1 --2- 172.21.15.76:0/2756761808 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c040c7750 0x7f8c04153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.070+0000 7f8c01ffb640 1 -- 172.21.15.76:0/2756761808 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1513747269 0 0) 0x7f8c040c8020 con 0x7f8c040c7b60 2024-04-25T19:16:52.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.070+0000 7f8c01ffb640 1 -- 172.21.15.76:0/2756761808 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bec003520 con 0x7f8c040c7b60 2024-04-25T19:16:52.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.070+0000 7f8c01ffb640 1 -- 172.21.15.76:0/2756761808 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3099011992 0 0) 0x7f8c041598a0 con 0x7f8c0415cd80 2024-04-25T19:16:52.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.070+0000 7f8c01ffb640 1 -- 172.21.15.76:0/2756761808 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c040c8020 con 0x7f8c0415cd80 2024-04-25T19:16:52.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.070+0000 7f8c01ffb640 1 -- 172.21.15.76:0/2756761808 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4231684646 0 0) 0x7f8bec003520 con 0x7f8c040c7b60 2024-04-25T19:16:52.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.070+0000 7f8c01ffb640 1 -- 172.21.15.76:0/2756761808 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c040c7750 msgr2=0x7f8c04153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:52.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.070+0000 7f8c01ffb640 1 --2- 172.21.15.76:0/2756761808 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c040c7750 0x7f8c04153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.070+0000 7f8c01ffb640 1 -- 172.21.15.76:0/2756761808 >> v1:172.21.15.105:6789/0 conn(0x7f8c0415cd80 legacy=0x7f8c0415d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.070+0000 7f8c01ffb640 1 -- 172.21.15.76:0/2756761808 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c04159e20 con 0x7f8c040c7b60 2024-04-25T19:16:52.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.070+0000 7f8c01ffb640 1 -- 172.21.15.76:0/2756761808 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f8bf8001670 con 0x7f8c040c7b60 2024-04-25T19:16:52.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.070+0000 7f8c01ffb640 1 -- 172.21.15.76:0/2756761808 >> v1:172.21.15.76:6789/0 conn(0x7f8c040c7b60 legacy=0x7f8c040cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.070+0000 7f8c01ffb640 1 --2- 172.21.15.76:0/2756761808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bec003b70 0x7f8bec003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.070+0000 7f8c01ffb640 1 --2- 172.21.15.76:0/2756761808 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c040c7750 0x7f8bec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.070+0000 7f8c01ffb640 1 --2- 172.21.15.76:0/2756761808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bec004d40 0x7f8bec005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.070+0000 7f8c01ffb640 1 -- 172.21.15.76:0/2756761808 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8c04159b40 con 0x7f8bec003b70 2024-04-25T19:16:52.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.070+0000 7f8c01ffb640 1 -- 172.21.15.76:0/2756761808 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec00bac0 con 0x7f8bec004d40 2024-04-25T19:16:52.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.070+0000 7f8c01ffb640 1 -- 172.21.15.76:0/2756761808 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8bec00bcc0 con 0x7f8c040c7750 2024-04-25T19:16:52.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.070+0000 7f8c02ffd640 1 --2- 172.21.15.76:0/2756761808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bec003b70 0x7f8bec003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.071+0000 7f8c027fc640 1 --2- 172.21.15.76:0/2756761808 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c040c7750 0x7f8bec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.071+0000 7f8c037fe640 1 --2- 172.21.15.76:0/2756761808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bec004d40 0x7f8bec005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.071+0000 7f8c02ffd640 1 -- 172.21.15.76:0/2756761808 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c040c7750 msgr2=0x7f8bec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:52.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.071+0000 7f8c02ffd640 1 --2- 172.21.15.76:0/2756761808 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c040c7750 0x7f8bec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.071+0000 7f8c02ffd640 1 -- 172.21.15.76:0/2756761808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bec004d40 msgr2=0x7f8bec005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:52.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.071+0000 7f8c02ffd640 1 --2- 172.21.15.76:0/2756761808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bec004d40 0x7f8bec005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.071+0000 7f8c02ffd640 1 -- 172.21.15.76:0/2756761808 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bec003520 con 0x7f8bec003b70 2024-04-25T19:16:52.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.071+0000 7f8c037fe640 1 --2- 172.21.15.76:0/2756761808 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bec004d40 0x7f8bec005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:16:52.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.072+0000 7f8c02ffd640 1 --2- 172.21.15.76:0/2756761808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bec003b70 0x7f8bec003fc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009650 tx=0x7f8bec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60429245e41983a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:52.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.072+0000 7f8c01ffb640 1 -- 172.21.15.76:0/2756761808 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4038070 con 0x7f8bec003b70 2024-04-25T19:16:52.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.072+0000 7f8c01ffb640 1 -- 172.21.15.76:0/2756761808 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf4033400 con 0x7f8bec003b70 2024-04-25T19:16:52.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.072+0000 7f8c01ffb640 1 -- 172.21.15.76:0/2756761808 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4009690 con 0x7f8bec003b70 2024-04-25T19:16:52.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.073+0000 7f8c09372640 1 -- 172.21.15.76:0/2756761808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bec003b70 msgr2=0x7f8bec003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.073+0000 7f8c09372640 1 --2- 172.21.15.76:0/2756761808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bec003b70 0x7f8bec003fc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009650 tx=0x7f8bec005830 comp rx=0 tx=0).stop 2024-04-25T19:16:52.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.073+0000 7f8c037fe640 1 -- 172.21.15.76:0/2756761808 reap_dead start 2024-04-25T19:16:52.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.073+0000 7f8c09372640 1 -- 172.21.15.76:0/2756761808 shutdown_connections 2024-04-25T19:16:52.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.073+0000 7f8c09372640 1 -- 172.21.15.76:0/2756761808 >> 172.21.15.76:0/2756761808 conn(0x7f8c04070800 msgr2=0x7f8c04070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:52.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.073+0000 7f8c09372640 1 -- 172.21.15.76:0/2756761808 shutdown_connections 2024-04-25T19:16:52.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.073+0000 7f8c09372640 1 -- 172.21.15.76:0/2756761808 wait complete. 2024-04-25T19:16:52.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.076+0000 7f8c09372640 1 Processor -- start 2024-04-25T19:16:52.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.076+0000 7f8c09372640 1 -- start start 2024-04-25T19:16:52.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.078+0000 7f8c09372640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bec003b70 0x7f8c041f2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.078+0000 7f8c09372640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c04153ff0 0x7f8c0414e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.078+0000 7f8c09372640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c0414c3b0 0x7f8c0414c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.078+0000 7f8c09372640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8c040c8520 con 0x7f8c04153ff0 2024-04-25T19:16:52.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.078+0000 7f8c09372640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0415a3e0 con 0x7f8bec003b70 2024-04-25T19:16:52.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.078+0000 7f8c09372640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8c040c86a0 con 0x7f8c0414c3b0 2024-04-25T19:16:52.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.078+0000 7f8c027fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c04153ff0 0x7f8c0414e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.078+0000 7f8c027fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c04153ff0 0x7f8c0414e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46866/0 (socket says 172.21.15.76:46866) 2024-04-25T19:16:52.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.078+0000 7f8c027fc640 1 -- 172.21.15.76:0/4230210338 learned_addr learned my addr 172.21.15.76:0/4230210338 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:52.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.078+0000 7f8c02ffd640 1 --2- 172.21.15.76:0/4230210338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bec003b70 0x7f8c041f2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.078+0000 7f8c037fe640 1 --2- 172.21.15.76:0/4230210338 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c0414c3b0 0x7f8c0414c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.078+0000 7f8c027fc640 1 -- 172.21.15.76:0/4230210338 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c0414c3b0 msgr2=0x7f8c0414c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.078+0000 7f8c027fc640 1 --2- 172.21.15.76:0/4230210338 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c0414c3b0 0x7f8c0414c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.078+0000 7f8c027fc640 1 -- 172.21.15.76:0/4230210338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bec003b70 msgr2=0x7f8c041f2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.078+0000 7f8c027fc640 1 --2- 172.21.15.76:0/4230210338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bec003b70 0x7f8c041f2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.078+0000 7f8c027fc640 1 -- 172.21.15.76:0/4230210338 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c041f26a0 con 0x7f8c04153ff0 2024-04-25T19:16:52.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.078+0000 7f8c037fe640 1 --2- 172.21.15.76:0/4230210338 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c0414c3b0 0x7f8c0414c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:16:52.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.078+0000 7f8c027fc640 1 --2- 172.21.15.76:0/4230210338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c04153ff0 0x7f8c0414e6b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8c0406fd10 tx=0x7f8bf80077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:52.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.079+0000 7f8beb7fe640 1 -- 172.21.15.76:0/4230210338 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf800f960 con 0x7f8c04153ff0 2024-04-25T19:16:52.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.079+0000 7f8beb7fe640 1 -- 172.21.15.76:0/4230210338 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf8018070 con 0x7f8c04153ff0 2024-04-25T19:16:52.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.079+0000 7f8c09372640 1 -- 172.21.15.76:0/4230210338 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c040736a0 con 0x7f8c04153ff0 2024-04-25T19:16:52.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.079+0000 7f8beb7fe640 1 -- 172.21.15.76:0/4230210338 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf800fd20 con 0x7f8c04153ff0 2024-04-25T19:16:52.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.079+0000 7f8c09372640 1 -- 172.21.15.76:0/4230210338 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c04154df0 con 0x7f8c04153ff0 2024-04-25T19:16:52.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.080+0000 7f8c09372640 1 -- 172.21.15.76:0/4230210338 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c0406a390 con 0x7f8c04153ff0 2024-04-25T19:16:52.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.081+0000 7f8beb7fe640 1 -- 172.21.15.76:0/4230210338 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8bf801d460 con 0x7f8c04153ff0 2024-04-25T19:16:52.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.081+0000 7f8beb7fe640 1 --2- 172.21.15.76:0/4230210338 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8bf00744e0 0x7f8bf00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.081+0000 7f8beb7fe640 1 -- 172.21.15.76:0/4230210338 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f8bf8095140 con 0x7f8c04153ff0 2024-04-25T19:16:52.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.081+0000 7f8c02ffd640 1 --2- 172.21.15.76:0/4230210338 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8bf00744e0 0x7f8bf00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.082+0000 7f8c02ffd640 1 --2- 172.21.15.76:0/4230210338 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8bf00744e0 0x7f8bf00769f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4035000 tx=0x7f8bf4008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:52.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.086+0000 7f8beb7fe640 1 -- 172.21.15.76:0/4230210338 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8bf805f8e0 con 0x7f8c04153ff0 2024-04-25T19:16:52.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.343+0000 7f8c09372640 1 -- 172.21.15.76:0/4230210338 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f8c040ccac0 con 0x7f8c04153ff0 2024-04-25T19:16:52.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.343+0000 7f8beb7fe640 1 -- 172.21.15.76:0/4230210338 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7f8bf805f1e0 con 0x7f8c04153ff0 2024-04-25T19:16:52.345 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:16:52.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.345+0000 7f8c09372640 1 -- 172.21.15.76:0/4230210338 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8bf00744e0 msgr2=0x7f8bf00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.345+0000 7f8c09372640 1 --2- 172.21.15.76:0/4230210338 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8bf00744e0 0x7f8bf00769f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4035000 tx=0x7f8bf4008030 comp rx=0 tx=0).stop 2024-04-25T19:16:52.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.345+0000 7f8c09372640 1 -- 172.21.15.76:0/4230210338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c04153ff0 msgr2=0x7f8c0414e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.345+0000 7f8c09372640 1 --2- 172.21.15.76:0/4230210338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c04153ff0 0x7f8c0414e6b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8c0406fd10 tx=0x7f8bf80077a0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.345+0000 7f8c09372640 1 -- 172.21.15.76:0/4230210338 shutdown_connections 2024-04-25T19:16:52.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.345+0000 7f8c09372640 1 --2- 172.21.15.76:0/4230210338 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8bf00744e0 0x7f8bf00769f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.345+0000 7f8c09372640 1 --2- 172.21.15.76:0/4230210338 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8c0414c3b0 0x7f8c0414c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.345+0000 7f8c09372640 1 --2- 172.21.15.76:0/4230210338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c04153ff0 0x7f8c0414e6b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.345+0000 7f8c09372640 1 --2- 172.21.15.76:0/4230210338 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bec003b70 0x7f8c041f2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.345+0000 7f8c09372640 1 -- 172.21.15.76:0/4230210338 >> 172.21.15.76:0/4230210338 conn(0x7f8c04070800 msgr2=0x7f8c041526b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:52.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.345+0000 7f8c09372640 1 -- 172.21.15.76:0/4230210338 shutdown_connections 2024-04-25T19:16:52.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.345+0000 7f8c09372640 1 -- 172.21.15.76:0/4230210338 wait complete. 2024-04-25T19:16:52.358 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T19:16:52.358 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-25T19:16:52.358 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T19:16:52.359 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T19:16:52.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.467+0000 7eff4e338640 1 Processor -- start 2024-04-25T19:16:52.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.467+0000 7eff4e338640 1 -- start start 2024-04-25T19:16:52.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.469+0000 7eff4e338640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7eff48159840 0x7eff48153b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.469+0000 7eff4e338640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7eff4815d880 0x7eff480c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.469+0000 7eff4e338640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff480715d0 con 0x7eff480c5b20 2024-04-25T19:16:52.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.469+0000 7eff4e338640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7eff4815ce30 con 0x7eff48159840 2024-04-25T19:16:52.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.469+0000 7eff4e338640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7eff480718b0 con 0x7eff4815d880 2024-04-25T19:16:52.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.469+0000 7eff47fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7eff48159840 0x7eff48153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.469+0000 7eff47fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7eff48159840 0x7eff48153b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46868/0 (socket says 172.21.15.76:46868) 2024-04-25T19:16:52.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.469+0000 7eff47fff640 1 -- 172.21.15.76:0/4177426089 learned_addr learned my addr 172.21.15.76:0/4177426089 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:52.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.469+0000 7eff4c8d5640 1 --2- 172.21.15.76:0/4177426089 >> v2:172.21.15.105:3300/0 conn(0x7eff4815d880 0x7eff480c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.469+0000 7eff47fff640 1 -- 172.21.15.76:0/4177426089 >> v2:172.21.15.105:3300/0 conn(0x7eff4815d880 msgr2=0x7eff480c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:52.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.469+0000 7eff47fff640 1 --2- 172.21.15.76:0/4177426089 >> v2:172.21.15.105:3300/0 conn(0x7eff4815d880 0x7eff480c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.469+0000 7eff47fff640 1 -- 172.21.15.76:0/4177426089 >> v1:172.21.15.105:6789/0 conn(0x7eff480c5b20 legacy=0x7eff480c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:52.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.469+0000 7eff47fff640 1 -- 172.21.15.76:0/4177426089 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff48071b90 con 0x7eff48159840 2024-04-25T19:16:52.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.470+0000 7eff47fff640 1 --2- 172.21.15.76:0/4177426089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff48159840 0x7eff48153b70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7eff480c8810 tx=0x7eff3802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5077ab49ff3fba87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:52.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.470+0000 7eff46ffd640 1 -- 172.21.15.76:0/4177426089 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff38037070 con 0x7eff48159840 2024-04-25T19:16:52.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.470+0000 7eff46ffd640 1 -- 172.21.15.76:0/4177426089 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff38032440 con 0x7eff48159840 2024-04-25T19:16:52.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.470+0000 7eff46ffd640 1 -- 172.21.15.76:0/4177426089 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff38032810 con 0x7eff48159840 2024-04-25T19:16:52.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.471+0000 7eff4e338640 1 -- 172.21.15.76:0/4177426089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff48159840 msgr2=0x7eff48153b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.471+0000 7eff4e338640 1 --2- 172.21.15.76:0/4177426089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff48159840 0x7eff48153b70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7eff480c8810 tx=0x7eff3802a230 comp rx=0 tx=0).stop 2024-04-25T19:16:52.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.471+0000 7eff4e338640 1 -- 172.21.15.76:0/4177426089 shutdown_connections 2024-04-25T19:16:52.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.471+0000 7eff4e338640 1 --2- 172.21.15.76:0/4177426089 >> v2:172.21.15.105:3300/0 conn(0x7eff4815d880 0x7eff480c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.471+0000 7eff4e338640 1 --2- 172.21.15.76:0/4177426089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff48159840 0x7eff48153b70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.471+0000 7eff4e338640 1 -- 172.21.15.76:0/4177426089 >> 172.21.15.76:0/4177426089 conn(0x7eff480cecc0 msgr2=0x7eff480cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:52.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.471+0000 7eff4e338640 1 -- 172.21.15.76:0/4177426089 shutdown_connections 2024-04-25T19:16:52.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.471+0000 7eff4e338640 1 -- 172.21.15.76:0/4177426089 wait complete. 2024-04-25T19:16:52.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.473+0000 7eff4e338640 1 Processor -- start 2024-04-25T19:16:52.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.473+0000 7eff4e338640 1 -- start start 2024-04-25T19:16:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.474+0000 7eff4e338640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff480c5b20 0x7eff48142b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.474+0000 7eff4e338640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff48159840 0x7eff4814f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.474+0000 7eff4e338640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff4815d880 0x7eff4814fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.474+0000 7eff4e338640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7eff4815dc90 con 0x7eff4815d880 2024-04-25T19:16:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.474+0000 7eff4e338640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7eff48071270 con 0x7eff48159840 2024-04-25T19:16:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.474+0000 7eff4e338640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7eff480c8e30 con 0x7eff480c5b20 2024-04-25T19:16:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.474+0000 7eff4c8d5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff4815d880 0x7eff4814fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.474+0000 7eff4c8d5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff4815d880 0x7eff4814fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46884/0 (socket says 172.21.15.76:46884) 2024-04-25T19:16:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.474+0000 7eff4c8d5640 1 -- 172.21.15.76:0/1793358009 learned_addr learned my addr 172.21.15.76:0/1793358009 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.474+0000 7eff477fe640 1 --2- 172.21.15.76:0/1793358009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff48159840 0x7eff4814f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.474+0000 7eff47fff640 1 --2- 172.21.15.76:0/1793358009 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff480c5b20 0x7eff48142b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.475+0000 7eff4c8d5640 1 -- 172.21.15.76:0/1793358009 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff480c5b20 msgr2=0x7eff48142b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.475+0000 7eff4c8d5640 1 --2- 172.21.15.76:0/1793358009 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7eff480c5b20 0x7eff48142b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.475+0000 7eff4c8d5640 1 -- 172.21.15.76:0/1793358009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff48159840 msgr2=0x7eff4814f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.475+0000 7eff4c8d5640 1 --2- 172.21.15.76:0/1793358009 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7eff48159840 0x7eff4814f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.475+0000 7eff4c8d5640 1 -- 172.21.15.76:0/1793358009 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff4814eee0 con 0x7eff4815d880 2024-04-25T19:16:52.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.475+0000 7eff4c8d5640 1 --2- 172.21.15.76:0/1793358009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff4815d880 0x7eff4814fc70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7eff40007d80 tx=0x7eff4000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:52.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.475+0000 7eff44ff9640 1 -- 172.21.15.76:0/1793358009 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff40014420 con 0x7eff4815d880 2024-04-25T19:16:52.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.475+0000 7eff44ff9640 1 -- 172.21.15.76:0/1793358009 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff40019070 con 0x7eff4815d880 2024-04-25T19:16:52.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.475+0000 7eff4e338640 1 -- 172.21.15.76:0/1793358009 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff4814f1c0 con 0x7eff4815d880 2024-04-25T19:16:52.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.475+0000 7eff44ff9640 1 -- 172.21.15.76:0/1793358009 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff40014800 con 0x7eff4815d880 2024-04-25T19:16:52.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.475+0000 7eff4e338640 1 -- 172.21.15.76:0/1793358009 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff481522b0 con 0x7eff4815d880 2024-04-25T19:16:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.476+0000 7eff4e338640 1 --2- 172.21.15.76:0/1793358009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff48150580 0x7eff481e2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.476+0000 7eff4e338640 1 -- 172.21.15.76:0/1793358009 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7eff48152480 con 0x7eff48150580 2024-04-25T19:16:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.476+0000 7eff47fff640 1 --2- 172.21.15.76:0/1793358009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff48150580 0x7eff481e2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.476+0000 7eff47fff640 1 -- 172.21.15.76:0/1793358009 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7eff4806a390 con 0x7eff48150580 2024-04-25T19:16:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.477+0000 7eff47fff640 1 --2- 172.21.15.76:0/1793358009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff48150580 0x7eff481e2b20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7eff4806ef90 tx=0x7eff380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.477+0000 7eff44ff9640 1 -- 172.21.15.76:0/1793358009 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7eff400149a0 con 0x7eff4815d880 2024-04-25T19:16:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.477+0000 7eff44ff9640 1 --2- 172.21.15.76:0/1793358009 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7eff340744b0 0x7eff340769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.477+0000 7eff44ff9640 1 -- 172.21.15.76:0/1793358009 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7eff400958f0 con 0x7eff4815d880 2024-04-25T19:16:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.477+0000 7eff44ff9640 1 -- 172.21.15.76:0/1793358009 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff38042070 con 0x7eff48150580 2024-04-25T19:16:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.477+0000 7eff44ff9640 1 -- 172.21.15.76:0/1793358009 <== mon.0 v2:172.21.15.76:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7eff3803e660 con 0x7eff48150580 2024-04-25T19:16:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.477+0000 7eff44ff9640 1 -- 172.21.15.76:0/1793358009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff48150580 msgr2=0x7eff481e2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.477+0000 7eff44ff9640 1 --2- 172.21.15.76:0/1793358009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff48150580 0x7eff481e2b20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7eff4806ef90 tx=0x7eff380046c0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.477+0000 7eff44ff9640 1 -- 172.21.15.76:0/1793358009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff48150580 msgr2=0x7eff481e2b20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:16:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.477+0000 7eff44ff9640 1 --2- 172.21.15.76:0/1793358009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff48150580 0x7eff481e2b20 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.478+0000 7eff477fe640 1 --2- 172.21.15.76:0/1793358009 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7eff340744b0 0x7eff340769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.478+0000 7eff477fe640 1 --2- 172.21.15.76:0/1793358009 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7eff340744b0 0x7eff340769c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7eff48076310 tx=0x7eff3c009060 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:52.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.484+0000 7eff4e338640 1 --2- 172.21.15.76:0/1793358009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff481e46d0 0x7eff481e4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.484+0000 7eff4e338640 1 -- 172.21.15.76:0/1793358009 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7eff4814e9d0 con 0x7eff481e46d0 2024-04-25T19:16:52.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.485+0000 7eff47fff640 1 --2- 172.21.15.76:0/1793358009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff481e46d0 0x7eff481e4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.485+0000 7eff47fff640 1 -- 172.21.15.76:0/1793358009 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7eff480cbf90 con 0x7eff481e46d0 2024-04-25T19:16:52.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.485+0000 7eff47fff640 1 --2- 172.21.15.76:0/1793358009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff481e46d0 0x7eff481e4e30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7eff48050990 tx=0x7eff38008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:52.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.485+0000 7eff44ff9640 1 -- 172.21.15.76:0/1793358009 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff38037040 con 0x7eff481e46d0 2024-04-25T19:16:52.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.485+0000 7eff44ff9640 1 -- 172.21.15.76:0/1793358009 <== mon.0 v2:172.21.15.76:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5068 (secure 0 0 0) 0x7eff38048a80 con 0x7eff481e46d0 2024-04-25T19:16:52.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.485+0000 7eff44ff9640 1 -- 172.21.15.76:0/1793358009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff481e46d0 msgr2=0x7eff481e4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.485+0000 7eff44ff9640 1 --2- 172.21.15.76:0/1793358009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff481e46d0 0x7eff481e4e30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7eff48050990 tx=0x7eff38008030 comp rx=0 tx=0).stop 2024-04-25T19:16:52.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.485+0000 7eff44ff9640 1 -- 172.21.15.76:0/1793358009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff481e46d0 msgr2=0x7eff481e4e30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:16:52.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.485+0000 7eff44ff9640 1 --2- 172.21.15.76:0/1793358009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff481e46d0 0x7eff481e4e30 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.488 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:16:52.488 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:16:52.488 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:16:52.488 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "leader", 2024-04-25T19:16:52.488 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 10, 2024-04-25T19:16:52.488 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 111010, 2024-04-25T19:16:52.488 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:16:52.488 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:16:52.488 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-04-25T19:16:52.488 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:16:52.488 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:52.488 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 16, 2024-04-25T19:16:52.488 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:16:52.489 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:16:52.489 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:16:52.489 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:52.489 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:52.489 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:52.489 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:52.489 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:52.489 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:52.489 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:52.489 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:52.489 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:52.489 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:52.489 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:52.489 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:52.489 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:16:52.490 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:16:52.490 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:52.490 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:52.490 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:52.490 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:52.490 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:52.490 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:52.490 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:52.490 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:52.490 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:52.490 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:52.490 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:52.490 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:52.491 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.491 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:16:52.491 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:16:52.491 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:16:52.491 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:16:52.491 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:16:52.491 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:16:52.491 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:16:52.491 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:16:52.491 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:16:52.491 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:16:52.491 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:16:52.491 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:16:52.491 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:16:52.491 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:16:52.491 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:16:52.492 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:16:52.492 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:16:52.492 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:52.492 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:52.492 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:52.492 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:52.492 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:52.492 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:52.492 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:52.492 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:52.492 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:52.492 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:52.492 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:52.492 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:52.492 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:16:52.492 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.492 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:16:52.493 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.493 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:16:52.493 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:16:52.493 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:52.493 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:52.493 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.493 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:52.493 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:16:52.493 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:52.493 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.493 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.493 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:52.493 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:16:52.493 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:52.493 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:52.493 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:52.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.494 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:16:52.494 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:16:52.494 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:52.494 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:52.494 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:52.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.494 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.494 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:16:52.494 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:16:52.494 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:52.494 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:52.494 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.494 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:52.494 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:16:52.494 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:52.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.495 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:52.495 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:16:52.495 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:52.495 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:52.495 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:52.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.495 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:16:52.495 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:16:52.495 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:52.495 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:52.495 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:52.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.495 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:16:52.496 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:16:52.496 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:52.496 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:52.496 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.496 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:52.496 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:16:52.496 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:52.496 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.496 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.496 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:52.496 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:16:52.496 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:52.496 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:52.496 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:52.496 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.496 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:16:52.497 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:16:52.497 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:52.497 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:52.497 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:52.497 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:52.497 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:52.497 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.497 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:16:52.497 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:16:52.497 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.497 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:52.497 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:52.497 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:16:52.497 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:52.497 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:52.497 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": [ 2024-04-25T19:16:52.498 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.498 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:52.498 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:52.498 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 6 2024-04-25T19:16:52.498 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:52.498 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:52.498 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": [ 2024-04-25T19:16:52.498 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.498 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:52.498 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:52.498 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 4 2024-04-25T19:16:52.498 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:52.498 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:52.498 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:16:52.498 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.499 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:52.499 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:52.499 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 5 2024-04-25T19:16:52.499 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:52.499 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:52.499 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": [ 2024-04-25T19:16:52.499 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.499 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:52.499 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:52.499 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:16:52.499 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:52.499 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:52.499 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.499 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:16:52.499 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:16:52.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.486+0000 7eff4e338640 1 -- 172.21.15.76:0/1793358009 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7eff340744b0 msgr2=0x7eff340769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.486+0000 7eff4e338640 1 --2- 172.21.15.76:0/1793358009 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7eff340744b0 0x7eff340769c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7eff48076310 tx=0x7eff3c009060 comp rx=0 tx=0).stop 2024-04-25T19:16:52.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.486+0000 7eff4e338640 1 -- 172.21.15.76:0/1793358009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff4815d880 msgr2=0x7eff4814fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.486+0000 7eff4e338640 1 --2- 172.21.15.76:0/1793358009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff4815d880 0x7eff4814fc70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7eff40007d80 tx=0x7eff4000e4b0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.486+0000 7eff4c8d5640 1 -- 172.21.15.76:0/1793358009 reap_dead start 2024-04-25T19:16:52.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.486+0000 7eff4c8d5640 1 -- 172.21.15.76:0/1793358009 reap_dead start 2024-04-25T19:16:52.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.486+0000 7eff4e338640 1 -- 172.21.15.76:0/1793358009 shutdown_connections 2024-04-25T19:16:52.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.486+0000 7eff4e338640 1 -- 172.21.15.76:0/1793358009 >> 172.21.15.76:0/1793358009 conn(0x7eff480cecc0 msgr2=0x7eff480cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:52.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.486+0000 7eff4e338640 1 -- 172.21.15.76:0/1793358009 shutdown_connections 2024-04-25T19:16:52.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.486+0000 7eff4e338640 1 -- 172.21.15.76:0/1793358009 wait complete. 2024-04-25T19:16:52.502 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T19:16:52.502 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T19:16:52.502 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T19:16:52.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.610+0000 7fdec864c640 1 Processor -- start 2024-04-25T19:16:52.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.611+0000 7fdec864c640 1 -- start start 2024-04-25T19:16:52.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.612+0000 7fdec864c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdec0156f60 0x7fdec00cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.612+0000 7fdec864c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdec00c95c0 0x7fdec00cf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.612+0000 7fdec864c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdec0073c30 con 0x7fdec00c9ea0 2024-04-25T19:16:52.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.612+0000 7fdec864c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fdec015d730 con 0x7fdec0156f60 2024-04-25T19:16:52.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.612+0000 7fdec864c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fdec00746b0 con 0x7fdec00c95c0 2024-04-25T19:16:52.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.612+0000 7fdec63e8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdec0156f60 0x7fdec00cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.612+0000 7fdec63e8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdec0156f60 0x7fdec00cefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46906/0 (socket says 172.21.15.76:46906) 2024-04-25T19:16:52.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.612+0000 7fdec63e8640 1 -- 172.21.15.76:0/2656006869 learned_addr learned my addr 172.21.15.76:0/2656006869 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:52.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.612+0000 7fdec5be7640 1 --2- 172.21.15.76:0/2656006869 >> v2:172.21.15.105:3300/0 conn(0x7fdec00c95c0 0x7fdec00cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.613+0000 7fdec63e8640 1 -- 172.21.15.76:0/2656006869 >> v2:172.21.15.105:3300/0 conn(0x7fdec00c95c0 msgr2=0x7fdec00cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:52.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.613+0000 7fdec63e8640 1 --2- 172.21.15.76:0/2656006869 >> v2:172.21.15.105:3300/0 conn(0x7fdec00c95c0 0x7fdec00cf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.613+0000 7fdec63e8640 1 -- 172.21.15.76:0/2656006869 >> v1:172.21.15.105:6789/0 conn(0x7fdec00c9ea0 legacy=0x7fdec00c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.613+0000 7fdec63e8640 1 -- 172.21.15.76:0/2656006869 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec0073940 con 0x7fdec0156f60 2024-04-25T19:16:52.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.613+0000 7fdec63e8640 1 --2- 172.21.15.76:0/2656006869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdec0156f60 0x7fdec00cefe0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdec00c5bf0 tx=0x7fdebc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daff8aa391d32ee3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:52.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.614+0000 7fdec53e6640 1 -- 172.21.15.76:0/2656006869 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc037070 con 0x7fdec0156f60 2024-04-25T19:16:52.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.614+0000 7fdec53e6640 1 -- 172.21.15.76:0/2656006869 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdebc032440 con 0x7fdec0156f60 2024-04-25T19:16:52.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.614+0000 7fdec53e6640 1 -- 172.21.15.76:0/2656006869 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc032810 con 0x7fdec0156f60 2024-04-25T19:16:52.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.614+0000 7fdec864c640 1 -- 172.21.15.76:0/2656006869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdec0156f60 msgr2=0x7fdec00cefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.614+0000 7fdec864c640 1 --2- 172.21.15.76:0/2656006869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdec0156f60 0x7fdec00cefe0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdec00c5bf0 tx=0x7fdebc02a270 comp rx=0 tx=0).stop 2024-04-25T19:16:52.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.615+0000 7fdec864c640 1 -- 172.21.15.76:0/2656006869 shutdown_connections 2024-04-25T19:16:52.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.615+0000 7fdec864c640 1 --2- 172.21.15.76:0/2656006869 >> v2:172.21.15.105:3300/0 conn(0x7fdec00c95c0 0x7fdec00cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.615+0000 7fdec864c640 1 --2- 172.21.15.76:0/2656006869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdec0156f60 0x7fdec00cefe0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.615+0000 7fdec864c640 1 -- 172.21.15.76:0/2656006869 >> 172.21.15.76:0/2656006869 conn(0x7fdec0071a30 msgr2=0x7fdec0071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:52.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.615+0000 7fdec864c640 1 -- 172.21.15.76:0/2656006869 shutdown_connections 2024-04-25T19:16:52.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.615+0000 7fdec864c640 1 -- 172.21.15.76:0/2656006869 wait complete. 2024-04-25T19:16:52.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.616+0000 7fdec864c640 1 Processor -- start 2024-04-25T19:16:52.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.616+0000 7fdec864c640 1 -- start start 2024-04-25T19:16:52.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.618+0000 7fdec864c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdec00c95c0 0x7fdec014ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.618+0000 7fdec864c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdec00c9ea0 0x7fdec015c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.618+0000 7fdec864c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec0156f60 0x7fdec00ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.618+0000 7fdec864c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdec00740e0 con 0x7fdec00c9ea0 2024-04-25T19:16:52.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.618+0000 7fdec864c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdec015da10 con 0x7fdec0156f60 2024-04-25T19:16:52.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.618+0000 7fdec864c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fdec00c7ea0 con 0x7fdec00c95c0 2024-04-25T19:16:52.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.618+0000 7fdec5be7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdec00c9ea0 0x7fdec015c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.618+0000 7fdec5be7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdec00c9ea0 0x7fdec015c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46910/0 (socket says 172.21.15.76:46910) 2024-04-25T19:16:52.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.618+0000 7fdec5be7640 1 -- 172.21.15.76:0/911589416 learned_addr learned my addr 172.21.15.76:0/911589416 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:52.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.618+0000 7fdec63e8640 1 --2- 172.21.15.76:0/911589416 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdec00c95c0 0x7fdec014ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.618+0000 7fdec6be9640 1 --2- 172.21.15.76:0/911589416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec0156f60 0x7fdec00ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.618+0000 7fdec5be7640 1 -- 172.21.15.76:0/911589416 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdec00c95c0 msgr2=0x7fdec014ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.618+0000 7fdec5be7640 1 --2- 172.21.15.76:0/911589416 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdec00c95c0 0x7fdec014ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.618+0000 7fdec5be7640 1 -- 172.21.15.76:0/911589416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec0156f60 msgr2=0x7fdec00ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.618+0000 7fdec5be7640 1 --2- 172.21.15.76:0/911589416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec0156f60 0x7fdec00ca750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.618+0000 7fdec5be7640 1 -- 172.21.15.76:0/911589416 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec01f3ad0 con 0x7fdec00c9ea0 2024-04-25T19:16:52.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.618+0000 7fdec5be7640 1 --2- 172.21.15.76:0/911589416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdec00c9ea0 0x7fdec015c3f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000e8a0 tx=0x7fdeb000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:52.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.620+0000 7fdeb6ffd640 1 -- 172.21.15.76:0/911589416 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0017070 con 0x7fdec00c9ea0 2024-04-25T19:16:52.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.620+0000 7fdeb6ffd640 1 -- 172.21.15.76:0/911589416 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeb000ee90 con 0x7fdec00c9ea0 2024-04-25T19:16:52.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.620+0000 7fdec864c640 1 -- 172.21.15.76:0/911589416 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec015c930 con 0x7fdec00c9ea0 2024-04-25T19:16:52.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.620+0000 7fdeb6ffd640 1 -- 172.21.15.76:0/911589416 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb000cb40 con 0x7fdec00c9ea0 2024-04-25T19:16:52.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.620+0000 7fdec864c640 1 -- 172.21.15.76:0/911589416 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec01f3d50 con 0x7fdec00c9ea0 2024-04-25T19:16:52.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.621+0000 7fdec864c640 1 --2- 172.21.15.76:0/911589416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec00cac90 0x7fdec00cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.621+0000 7fdec864c640 1 -- 172.21.15.76:0/911589416 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdec00cc9c0 con 0x7fdec00cac90 2024-04-25T19:16:52.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.622+0000 7fdec63e8640 1 --2- 172.21.15.76:0/911589416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec00cac90 0x7fdec00cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.622+0000 7fdeb6ffd640 1 -- 172.21.15.76:0/911589416 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdeb001a070 con 0x7fdec00c9ea0 2024-04-25T19:16:52.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.622+0000 7fdec63e8640 1 -- 172.21.15.76:0/911589416 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdec006a390 con 0x7fdec00cac90 2024-04-25T19:16:52.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.622+0000 7fdeb6ffd640 1 --2- 172.21.15.76:0/911589416 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fdea40744b0 0x7fdea40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.622+0000 7fdeb6ffd640 1 -- 172.21.15.76:0/911589416 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fdeb0094f80 con 0x7fdec00c9ea0 2024-04-25T19:16:52.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.622+0000 7fdec63e8640 1 --2- 172.21.15.76:0/911589416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec00cac90 0x7fdec00cb0a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdec006c5c0 tx=0x7fdebc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:52.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.623+0000 7fdec6be9640 1 --2- 172.21.15.76:0/911589416 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fdea40744b0 0x7fdea40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.623+0000 7fdeb6ffd640 1 -- 172.21.15.76:0/911589416 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc042070 con 0x7fdec00cac90 2024-04-25T19:16:52.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.623+0000 7fdec6be9640 1 --2- 172.21.15.76:0/911589416 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fdea40744b0 0x7fdea40769c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80012e0 tx=0x7fdeb8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:52.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.623+0000 7fdeb6ffd640 1 -- 172.21.15.76:0/911589416 <== mon.1 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fdebc03e660 con 0x7fdec00cac90 2024-04-25T19:16:52.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.623+0000 7fdeb6ffd640 1 -- 172.21.15.76:0/911589416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec00cac90 msgr2=0x7fdec00cb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.623+0000 7fdeb6ffd640 1 --2- 172.21.15.76:0/911589416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec00cac90 0x7fdec00cb0a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdec006c5c0 tx=0x7fdebc0046c0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.623+0000 7fdeb6ffd640 1 -- 172.21.15.76:0/911589416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec00cac90 msgr2=0x7fdec00cb0a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:16:52.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.623+0000 7fdeb6ffd640 1 --2- 172.21.15.76:0/911589416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec00cac90 0x7fdec00cb0a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.630+0000 7fdec864c640 1 --2- 172.21.15.76:0/911589416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec01f65a0 0x7fdec01f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.630+0000 7fdec864c640 1 -- 172.21.15.76:0/911589416 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdec00cb680 con 0x7fdec01f65a0 2024-04-25T19:16:52.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.631+0000 7fdec63e8640 1 --2- 172.21.15.76:0/911589416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec01f65a0 0x7fdec01f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.631+0000 7fdec63e8640 1 -- 172.21.15.76:0/911589416 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fdec00c9a70 con 0x7fdec01f65a0 2024-04-25T19:16:52.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.631+0000 7fdec63e8640 1 --2- 172.21.15.76:0/911589416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec01f65a0 0x7fdec01f69b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdebc000c00 tx=0x7fdebc008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:52.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.632+0000 7fdeb6ffd640 1 -- 172.21.15.76:0/911589416 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc037040 con 0x7fdec01f65a0 2024-04-25T19:16:52.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.632+0000 7fdeb6ffd640 1 -- 172.21.15.76:0/911589416 <== mon.1 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4896 (secure 0 0 0) 0x7fdebc03ee10 con 0x7fdec01f65a0 2024-04-25T19:16:52.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.632+0000 7fdeb6ffd640 1 -- 172.21.15.76:0/911589416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec01f65a0 msgr2=0x7fdec01f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.632+0000 7fdeb6ffd640 1 --2- 172.21.15.76:0/911589416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec01f65a0 0x7fdec01f69b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdebc000c00 tx=0x7fdebc008030 comp rx=0 tx=0).stop 2024-04-25T19:16:52.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.632+0000 7fdeb6ffd640 1 -- 172.21.15.76:0/911589416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec01f65a0 msgr2=0x7fdec01f69b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:16:52.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.632+0000 7fdeb6ffd640 1 --2- 172.21.15.76:0/911589416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdec01f65a0 0x7fdec01f69b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.634 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:16:52.634 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:16:52.634 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:16:52.634 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "peon", 2024-04-25T19:16:52.634 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 10, 2024-04-25T19:16:52.634 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 111113, 2024-04-25T19:16:52.634 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:16:52.634 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:16:52.635 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-04-25T19:16:52.635 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:16:52.635 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:52.635 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 16, 2024-04-25T19:16:52.635 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:16:52.635 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:16:52.635 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:16:52.635 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:52.635 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:52.635 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:52.635 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:52.635 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:52.635 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:52.635 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:52.635 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:52.636 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:52.636 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:52.636 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:52.636 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:52.636 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:16:52.636 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:16:52.636 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:52.636 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:52.636 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:52.636 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:52.636 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:52.636 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:52.636 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:52.637 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:52.637 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:52.637 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:52.637 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:52.637 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:52.637 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.637 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:16:52.637 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:16:52.637 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:16:52.637 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:16:52.637 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:16:52.637 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:16:52.637 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:16:52.637 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:16:52.637 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:16:52.637 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:16:52.638 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:16:52.638 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:16:52.638 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:16:52.638 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:16:52.638 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:16:52.638 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:16:52.638 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:16:52.638 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:52.638 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:52.638 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:52.638 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:52.638 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:52.638 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:52.638 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:52.639 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:52.639 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:52.639 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:52.639 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:52.639 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:52.639 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:16:52.639 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.639 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:16:52.639 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.639 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:16:52.639 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:16:52.639 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:52.639 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:52.639 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.639 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:52.639 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:16:52.640 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:52.640 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.640 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.640 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:52.640 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:16:52.640 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:52.640 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:52.640 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:52.640 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.640 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:16:52.640 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:16:52.640 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:52.640 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:52.640 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:52.640 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.641 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.641 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:16:52.641 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:16:52.641 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:52.641 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:52.641 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.641 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:52.641 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:16:52.641 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:52.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.641 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.641 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:52.641 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:16:52.641 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:52.641 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:52.641 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:52.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.642 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:16:52.642 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:16:52.642 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:52.642 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:52.642 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:52.642 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.642 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.642 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:16:52.642 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:16:52.642 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:52.642 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:52.642 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.642 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:52.642 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:16:52.642 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:52.643 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.643 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.643 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:52.643 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:16:52.643 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:52.643 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:52.643 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:52.643 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.643 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:16:52.643 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:16:52.643 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:52.643 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:52.643 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:52.643 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:52.643 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:52.643 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.644 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:16:52.644 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:16:52.644 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.644 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:52.644 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:52.644 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:16:52.644 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:52.644 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:52.644 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": [ 2024-04-25T19:16:52.644 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.644 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:52.644 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:52.644 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 4 2024-04-25T19:16:52.644 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:52.644 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:52.645 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:16:52.645 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.645 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:52.645 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:52.645 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 3 2024-04-25T19:16:52.645 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:52.645 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:52.645 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": [ 2024-04-25T19:16:52.645 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.645 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:52.645 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:52.645 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:16:52.645 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:52.645 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:52.645 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.645 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:16:52.646 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:16:52.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.632+0000 7fdec864c640 1 -- 172.21.15.76:0/911589416 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fdea40744b0 msgr2=0x7fdea40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.632+0000 7fdec864c640 1 --2- 172.21.15.76:0/911589416 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fdea40744b0 0x7fdea40769c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80012e0 tx=0x7fdeb8009040 comp rx=0 tx=0).stop 2024-04-25T19:16:52.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.632+0000 7fdec864c640 1 -- 172.21.15.76:0/911589416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdec00c9ea0 msgr2=0x7fdec015c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.632+0000 7fdec864c640 1 --2- 172.21.15.76:0/911589416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdec00c9ea0 0x7fdec015c3f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000e8a0 tx=0x7fdeb000ed60 comp rx=0 tx=0).stop 2024-04-25T19:16:52.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.632+0000 7fdec6be9640 1 -- 172.21.15.76:0/911589416 reap_dead start 2024-04-25T19:16:52.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.632+0000 7fdec864c640 1 -- 172.21.15.76:0/911589416 shutdown_connections 2024-04-25T19:16:52.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.632+0000 7fdec864c640 1 --2- 172.21.15.76:0/911589416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdec00c9ea0 0x7fdec015c3f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.632+0000 7fdec864c640 1 -- 172.21.15.76:0/911589416 >> 172.21.15.76:0/911589416 conn(0x7fdec0071a30 msgr2=0x7fdec0075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:52.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.632+0000 7fdec864c640 1 -- 172.21.15.76:0/911589416 shutdown_connections 2024-04-25T19:16:52.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.632+0000 7fdec864c640 1 -- 172.21.15.76:0/911589416 wait complete. 2024-04-25T19:16:52.648 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T19:16:52.648 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T19:16:52.648 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T19:16:52.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.757+0000 7f3315830640 1 Processor -- start 2024-04-25T19:16:52.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.757+0000 7f3315830640 1 -- start start 2024-04-25T19:16:52.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.758+0000 7f3315830640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3310071520 0x7f3310151350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.758+0000 7f3315830640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f331015ac70 0x7f331015b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.758+0000 7f3315830640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3310151890 con 0x7f3310073880 2024-04-25T19:16:52.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.758+0000 7f3315830640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f3310071ad0 con 0x7f3310071520 2024-04-25T19:16:52.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.758+0000 7f3315830640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f3310071d70 con 0x7f331015ac70 2024-04-25T19:16:52.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.758+0000 7f330effd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3310071520 0x7f3310151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.758+0000 7f330effd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f3310071520 0x7f3310151350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:47414/0 (socket says 172.21.15.76:47414) 2024-04-25T19:16:52.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.758+0000 7f330f7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f331015ac70 0x7f331015b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.758+0000 7f330effd640 1 -- 172.21.15.76:0/1323837782 learned_addr learned my addr 172.21.15.76:0/1323837782 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:52.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.759+0000 7f330effd640 1 -- 172.21.15.76:0/1323837782 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f331015ac70 msgr2=0x7f331015b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:52.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.759+0000 7f330effd640 1 --2- 172.21.15.76:0/1323837782 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f331015ac70 0x7f331015b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.759+0000 7f330effd640 1 -- 172.21.15.76:0/1323837782 >> v1:172.21.15.105:6789/0 conn(0x7f3310073880 legacy=0x7f33100c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.759+0000 7f330effd640 1 -- 172.21.15.76:0/1323837782 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3310074150 con 0x7f3310071520 2024-04-25T19:16:52.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.759+0000 7f330effd640 1 --2- 172.21.15.76:0/1323837782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3310071520 0x7f3310151350 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3300009870 tx=0x7f330002a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfd76304abf1006b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:52.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.759+0000 7f330dffb640 1 -- 172.21.15.76:0/1323837782 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300037070 con 0x7f3310071520 2024-04-25T19:16:52.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.759+0000 7f330dffb640 1 -- 172.21.15.76:0/1323837782 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f330002ae50 con 0x7f3310071520 2024-04-25T19:16:52.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.760+0000 7f330dffb640 1 -- 172.21.15.76:0/1323837782 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300032860 con 0x7f3310071520 2024-04-25T19:16:52.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.760+0000 7f3315830640 1 -- 172.21.15.76:0/1323837782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3310071520 msgr2=0x7f3310151350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.760+0000 7f3315830640 1 --2- 172.21.15.76:0/1323837782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3310071520 0x7f3310151350 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3300009870 tx=0x7f330002a300 comp rx=0 tx=0).stop 2024-04-25T19:16:52.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.760+0000 7f3315830640 1 -- 172.21.15.76:0/1323837782 shutdown_connections 2024-04-25T19:16:52.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.760+0000 7f3315830640 1 --2- 172.21.15.76:0/1323837782 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f331015ac70 0x7f331015b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.760+0000 7f3315830640 1 --2- 172.21.15.76:0/1323837782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3310071520 0x7f3310151350 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.760+0000 7f3315830640 1 -- 172.21.15.76:0/1323837782 >> 172.21.15.76:0/1323837782 conn(0x7f33100cecc0 msgr2=0x7f33100cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:52.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.760+0000 7f3315830640 1 -- 172.21.15.76:0/1323837782 shutdown_connections 2024-04-25T19:16:52.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.760+0000 7f3315830640 1 -- 172.21.15.76:0/1323837782 wait complete. 2024-04-25T19:16:52.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.762+0000 7f3315830640 1 Processor -- start 2024-04-25T19:16:52.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.762+0000 7f3315830640 1 -- start start 2024-04-25T19:16:52.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.763+0000 7f3315830640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3310071520 0x7f331014cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.763+0000 7f3315830640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3310073880 0x7f33100cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.763+0000 7f3315830640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f331015ac70 0x7f3310146610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.764+0000 7f3315830640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f33100749f0 con 0x7f3310073880 2024-04-25T19:16:52.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.764+0000 7f3315830640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3310074710 con 0x7f331015ac70 2024-04-25T19:16:52.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.764+0000 7f3315830640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f331015b5c0 con 0x7f3310071520 2024-04-25T19:16:52.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.764+0000 7f330e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3310073880 0x7f33100cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.764+0000 7f330e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3310073880 0x7f33100cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46918/0 (socket says 172.21.15.76:46918) 2024-04-25T19:16:52.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.764+0000 7f330e7fc640 1 -- 172.21.15.76:0/576462416 learned_addr learned my addr 172.21.15.76:0/576462416 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:52.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.764+0000 7f330effd640 1 --2- 172.21.15.76:0/576462416 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3310071520 0x7f331014cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.764+0000 7f330f7fe640 1 --2- 172.21.15.76:0/576462416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f331015ac70 0x7f3310146610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.764+0000 7f330e7fc640 1 -- 172.21.15.76:0/576462416 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3310071520 msgr2=0x7f331014cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.764+0000 7f330e7fc640 1 --2- 172.21.15.76:0/576462416 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3310071520 0x7f331014cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.764+0000 7f330e7fc640 1 -- 172.21.15.76:0/576462416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f331015ac70 msgr2=0x7f3310146610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.764+0000 7f330e7fc640 1 --2- 172.21.15.76:0/576462416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f331015ac70 0x7f3310146610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.764+0000 7f330e7fc640 1 -- 172.21.15.76:0/576462416 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f331014d290 con 0x7f3310073880 2024-04-25T19:16:52.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.764+0000 7f330f7fe640 1 --2- 172.21.15.76:0/576462416 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f331015ac70 0x7f3310146610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:16:52.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.764+0000 7f330e7fc640 1 --2- 172.21.15.76:0/576462416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3310073880 0x7f33100cddf0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f330400c4a0 tx=0x7f330400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:52.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.764+0000 7f32f7fff640 1 -- 172.21.15.76:0/576462416 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33040052d0 con 0x7f3310073880 2024-04-25T19:16:52.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.764+0000 7f32f7fff640 1 -- 172.21.15.76:0/576462416 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3304005470 con 0x7f3310073880 2024-04-25T19:16:52.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.764+0000 7f32f7fff640 1 -- 172.21.15.76:0/576462416 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304007260 con 0x7f3310073880 2024-04-25T19:16:52.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.764+0000 7f3315830640 1 -- 172.21.15.76:0/576462416 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33101f4320 con 0x7f3310073880 2024-04-25T19:16:52.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.764+0000 7f3315830640 1 -- 172.21.15.76:0/576462416 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33101f48e0 con 0x7f3310073880 2024-04-25T19:16:52.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.765+0000 7f3315830640 1 --2- 172.21.15.76:0/576462416 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33101f4ab0 0x7f3310146b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.765+0000 7f3315830640 1 -- 172.21.15.76:0/576462416 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f331014d760 con 0x7f33101f4ab0 2024-04-25T19:16:52.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.766+0000 7f330effd640 1 --2- 172.21.15.76:0/576462416 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33101f4ab0 0x7f3310146b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.766+0000 7f330effd640 1 -- 172.21.15.76:0/576462416 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f331006a390 con 0x7f33101f4ab0 2024-04-25T19:16:52.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.766+0000 7f330effd640 1 --2- 172.21.15.76:0/576462416 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33101f4ab0 0x7f3310146b50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3310073540 tx=0x7f33000046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:52.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.766+0000 7f32f7fff640 1 -- 172.21.15.76:0/576462416 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300041070 con 0x7f33101f4ab0 2024-04-25T19:16:52.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.766+0000 7f32f7fff640 1 -- 172.21.15.76:0/576462416 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3304007400 con 0x7f3310073880 2024-04-25T19:16:52.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.767+0000 7f32f7fff640 1 --2- 172.21.15.76:0/576462416 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f32fc0743e0 0x7f32fc0768f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.767+0000 7f32f7fff640 1 -- 172.21.15.76:0/576462416 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f3304094660 con 0x7f3310073880 2024-04-25T19:16:52.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.767+0000 7f32f7fff640 1 -- 172.21.15.76:0/576462416 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f330002ae50 con 0x7f33101f4ab0 2024-04-25T19:16:52.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.767+0000 7f32f7fff640 1 -- 172.21.15.76:0/576462416 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33101f4ab0 msgr2=0x7f3310146b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.767+0000 7f32f7fff640 1 --2- 172.21.15.76:0/576462416 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33101f4ab0 0x7f3310146b50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3310073540 tx=0x7f33000046c0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.767+0000 7f32f7fff640 1 -- 172.21.15.76:0/576462416 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33101f4ab0 msgr2=0x7f3310146b50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:16:52.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.767+0000 7f32f7fff640 1 --2- 172.21.15.76:0/576462416 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33101f4ab0 0x7f3310146b50 secure :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3310073540 tx=0x7f33000046c0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.767+0000 7f330f7fe640 1 --2- 172.21.15.76:0/576462416 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f32fc0743e0 0x7f32fc0768f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.768+0000 7f330f7fe640 1 --2- 172.21.15.76:0/576462416 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f32fc0743e0 0x7f32fc0768f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3310159520 tx=0x7f330800b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:52.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.774+0000 7f3315830640 1 --2- 172.21.15.76:0/576462416 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33100c5b20 0x7f331014ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.774+0000 7f3315830640 1 -- 172.21.15.76:0/576462416 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f331014c5e0 con 0x7f33100c5b20 2024-04-25T19:16:52.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.774+0000 7f330effd640 1 --2- 172.21.15.76:0/576462416 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33100c5b20 0x7f331014ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.774+0000 7f330effd640 1 -- 172.21.15.76:0/576462416 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3310147090 con 0x7f33100c5b20 2024-04-25T19:16:52.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.774+0000 7f330effd640 1 --2- 172.21.15.76:0/576462416 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33100c5b20 0x7f331014ff00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f330002a340 tx=0x7f3300008030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:52.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.775+0000 7f32f7fff640 1 -- 172.21.15.76:0/576462416 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300037040 con 0x7f33100c5b20 2024-04-25T19:16:52.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.775+0000 7f32f7fff640 1 -- 172.21.15.76:0/576462416 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4555 (secure 0 0 0) 0x7f330003d3f0 con 0x7f33100c5b20 2024-04-25T19:16:52.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.775+0000 7f32f7fff640 1 -- 172.21.15.76:0/576462416 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33100c5b20 msgr2=0x7f331014ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.775+0000 7f32f7fff640 1 --2- 172.21.15.76:0/576462416 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33100c5b20 0x7f331014ff00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f330002a340 tx=0x7f3300008030 comp rx=0 tx=0).stop 2024-04-25T19:16:52.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.775+0000 7f32f7fff640 1 -- 172.21.15.76:0/576462416 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33100c5b20 msgr2=0x7f331014ff00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:16:52.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.775+0000 7f32f7fff640 1 --2- 172.21.15.76:0/576462416 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f33100c5b20 0x7f331014ff00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.777 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:16:52.777 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:16:52.777 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:16:52.777 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "peon", 2024-04-25T19:16:52.777 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 10, 2024-04-25T19:16:52.778 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 16552, 2024-04-25T19:16:52.778 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:16:52.778 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:16:52.778 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-04-25T19:16:52.778 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:16:52.778 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:52.778 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 16, 2024-04-25T19:16:52.778 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:16:52.778 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:16:52.778 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:16:52.778 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:52.778 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:52.778 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:52.778 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:52.778 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:52.779 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:52.779 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:52.779 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:52.779 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:52.779 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:52.779 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:52.779 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:52.779 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:16:52.779 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:16:52.779 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:52.779 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:52.779 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:52.779 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:52.779 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:52.779 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:52.780 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:52.780 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:52.780 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:52.780 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:52.780 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:52.780 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:52.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.780 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:16:52.780 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:16:52.780 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:16:52.780 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:16:52.780 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:16:52.780 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:16:52.780 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:16:52.780 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:16:52.780 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:16:52.781 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:16:52.781 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:16:52.781 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:16:52.781 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:16:52.781 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:16:52.781 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:16:52.781 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:16:52.781 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:16:52.781 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:16:52.781 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:16:52.781 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:16:52.781 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:16:52.781 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:16:52.781 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:16:52.781 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:16:52.781 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:16:52.782 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:16:52.782 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:16:52.782 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:16:52.782 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:52.782 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:16:52.782 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.782 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:16:52.782 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.782 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:16:52.782 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:16:52.782 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:52.782 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:52.782 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.782 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:52.783 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:16:52.783 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:52.783 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.783 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.783 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:52.783 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:16:52.783 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:52.783 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:52.783 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:52.783 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.783 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:16:52.783 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:16:52.783 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:52.783 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:52.783 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:52.784 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.784 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.784 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:16:52.784 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:16:52.784 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:52.784 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:52.784 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.784 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:52.784 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:16:52.784 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:52.784 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.784 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.784 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:52.784 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:16:52.784 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:52.785 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:52.785 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:52.785 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.785 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:16:52.785 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:16:52.785 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:52.785 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:52.785 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:52.785 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.785 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.785 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:16:52.785 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:16:52.785 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:16:52.785 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:16:52.785 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.785 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:16:52.785 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:16:52.786 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:52.786 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.786 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.786 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:16:52.786 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:16:52.786 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:16:52.786 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:52.786 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:52.786 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.786 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:16:52.786 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:16:52.786 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:16:52.786 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:16:52.786 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:16:52.786 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:52.787 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:52.787 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.787 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:16:52.787 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:16:52.787 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.787 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:52.787 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:52.787 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:16:52.787 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:52.787 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:16:52.787 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:16:52.787 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:16:52.787 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:16:52.788 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:16:52.788 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:16:52.788 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:16:52.788 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:16:52.788 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:16:52.788 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:16:52.788 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:16:52.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.775+0000 7f3315830640 1 -- 172.21.15.76:0/576462416 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f32fc0743e0 msgr2=0x7f32fc0768f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.775+0000 7f3315830640 1 --2- 172.21.15.76:0/576462416 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f32fc0743e0 0x7f32fc0768f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3310159520 tx=0x7f330800b040 comp rx=0 tx=0).stop 2024-04-25T19:16:52.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.775+0000 7f3315830640 1 -- 172.21.15.76:0/576462416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3310073880 msgr2=0x7f33100cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.775+0000 7f3315830640 1 --2- 172.21.15.76:0/576462416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3310073880 0x7f33100cddf0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f330400c4a0 tx=0x7f330400c960 comp rx=0 tx=0).stop 2024-04-25T19:16:52.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.775+0000 7f330f7fe640 1 -- 172.21.15.76:0/576462416 reap_dead start 2024-04-25T19:16:52.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.775+0000 7f330f7fe640 1 -- 172.21.15.76:0/576462416 reap_dead start 2024-04-25T19:16:52.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.775+0000 7f3315830640 1 -- 172.21.15.76:0/576462416 shutdown_connections 2024-04-25T19:16:52.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.775+0000 7f3315830640 1 -- 172.21.15.76:0/576462416 >> 172.21.15.76:0/576462416 conn(0x7f33100cecc0 msgr2=0x7f3310159e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:52.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.775+0000 7f3315830640 1 -- 172.21.15.76:0/576462416 shutdown_connections 2024-04-25T19:16:52.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.775+0000 7f3315830640 1 -- 172.21.15.76:0/576462416 wait complete. 2024-04-25T19:16:52.790 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T19:16:52.790 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-04-25T19:16:52.790 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-25T19:16:52.791 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-04-25T19:16:52.791 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-04-25T19:16:52.791 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T19:16:52.791 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:16:52.807 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T19:16:52.807 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-04-25T19:16:52.807 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:16:52.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.914+0000 7f9afe66e640 1 Processor -- start 2024-04-25T19:16:52.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.914+0000 7f9afe66e640 1 -- start start 2024-04-25T19:16:52.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.915+0000 7f9afe66e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9af80d0810 0x7f9af806f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.915+0000 7f9afe66e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9af80cea70 0x7f9af80cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.915+0000 7f9afe66e640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9af815ce80 con 0x7f9af806e160 2024-04-25T19:16:52.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.915+0000 7f9afe66e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f9af815d160 con 0x7f9af80d0810 2024-04-25T19:16:52.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.915+0000 7f9afe66e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f9af815d440 con 0x7f9af80cea70 2024-04-25T19:16:52.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.915+0000 7f9afcc0b640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f9af806e160 legacy=0x7f9af806e570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-04-25T19:16:52.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.915+0000 7f9af7fff640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f9af80d0810 msgr2=0x7f9af806f000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-04-25T19:16:52.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.915+0000 7f9af7fff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9af80d0810 0x7f9af806f000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:52.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.915+0000 7f9af77fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9af80cea70 0x7f9af80cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.916+0000 7f9af77fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9af80cea70 0x7f9af80cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:42868/0 (socket says 172.21.15.76:42868) 2024-04-25T19:16:52.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.916+0000 7f9af77fe640 1 -- 172.21.15.76:0/3775136016 learned_addr learned my addr 172.21.15.76:0/3775136016 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:52.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.916+0000 7f9af77fe640 1 -- 172.21.15.76:0/3775136016 >> v2:172.21.15.105:3300/0 conn(0x7f9af80d0810 msgr2=0x7f9af806f000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:16:52.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.916+0000 7f9af77fe640 1 --2- 172.21.15.76:0/3775136016 >> v2:172.21.15.105:3300/0 conn(0x7f9af80d0810 0x7f9af806f000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.916+0000 7f9af77fe640 1 -- 172.21.15.76:0/3775136016 >> v1:172.21.15.105:6789/0 conn(0x7f9af806e160 legacy=0x7f9af806e570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:16:52.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.916+0000 7f9af77fe640 1 -- 172.21.15.76:0/3775136016 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af815d720 con 0x7f9af80cea70 2024-04-25T19:16:52.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.917+0000 7f9af77fe640 1 --2- 172.21.15.76:0/3775136016 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9af80cea70 0x7f9af80cee80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9af808fb10 tx=0x7f9aec02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78eaf45b2ef83fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:52.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.917+0000 7f9af6ffd640 1 -- 172.21.15.76:0/3775136016 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec037070 con 0x7f9af80cea70 2024-04-25T19:16:52.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.917+0000 7f9af6ffd640 1 -- 172.21.15.76:0/3775136016 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aec032440 con 0x7f9af80cea70 2024-04-25T19:16:52.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.917+0000 7f9af6ffd640 1 -- 172.21.15.76:0/3775136016 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec032810 con 0x7f9af80cea70 2024-04-25T19:16:52.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.917+0000 7f9afe66e640 1 -- 172.21.15.76:0/3775136016 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9af80cea70 msgr2=0x7f9af80cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.917+0000 7f9afe66e640 1 --2- 172.21.15.76:0/3775136016 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9af80cea70 0x7f9af80cee80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9af808fb10 tx=0x7f9aec02a210 comp rx=0 tx=0).stop 2024-04-25T19:16:52.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.918+0000 7f9afe66e640 1 -- 172.21.15.76:0/3775136016 shutdown_connections 2024-04-25T19:16:52.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.918+0000 7f9afe66e640 1 --2- 172.21.15.76:0/3775136016 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9af80cea70 0x7f9af80cee80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.918+0000 7f9afe66e640 1 --2- 172.21.15.76:0/3775136016 >> v2:172.21.15.105:3300/0 conn(0x7f9af80d0810 0x7f9af806f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.918+0000 7f9afe66e640 1 -- 172.21.15.76:0/3775136016 >> 172.21.15.76:0/3775136016 conn(0x7f9af80b3c00 msgr2=0x7f9af80b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:52.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.918+0000 7f9afe66e640 1 -- 172.21.15.76:0/3775136016 shutdown_connections 2024-04-25T19:16:52.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.918+0000 7f9afe66e640 1 -- 172.21.15.76:0/3775136016 wait complete. 2024-04-25T19:16:52.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.919+0000 7f9afe66e640 1 Processor -- start 2024-04-25T19:16:52.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.919+0000 7f9afe66e640 1 -- start start 2024-04-25T19:16:52.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.921+0000 7f9afe66e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9af806e160 0x7f9af81f3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.921+0000 7f9afe66e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9af80cea70 0x7f9af814ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.921+0000 7f9afe66e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9af80d0810 0x7f9af80d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.921+0000 7f9afe66e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9af815cb70 con 0x7f9af806e160 2024-04-25T19:16:52.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.921+0000 7f9afe66e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9af806dc40 con 0x7f9af80d0810 2024-04-25T19:16:52.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.921+0000 7f9afe66e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f9af806f540 con 0x7f9af80cea70 2024-04-25T19:16:52.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.921+0000 7f9af7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9af806e160 0x7f9af81f3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.921+0000 7f9afcc0b640 1 -- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9af80d0810 msgr2=0x7f9af80d0cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3300/0 2024-04-25T19:16:52.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.921+0000 7f9afcc0b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9af80d0810 0x7f9af80d0cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:52.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.921+0000 7f9af7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9af806e160 0x7f9af81f3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46932/0 (socket says 172.21.15.76:46932) 2024-04-25T19:16:52.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.921+0000 7f9af7fff640 1 -- 172.21.15.76:0/3505746119 learned_addr learned my addr 172.21.15.76:0/3505746119 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:52.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.921+0000 7f9af77fe640 1 --2- 172.21.15.76:0/3505746119 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9af80cea70 0x7f9af814ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.921+0000 7f9af7fff640 1 -- 172.21.15.76:0/3505746119 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9af80cea70 msgr2=0x7f9af814ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:52.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.921+0000 7f9af7fff640 1 --2- 172.21.15.76:0/3505746119 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9af80cea70 0x7f9af814ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.921+0000 7f9af7fff640 1 -- 172.21.15.76:0/3505746119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9af80d0810 msgr2=0x7f9af80d0cb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:16:52.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.921+0000 7f9af7fff640 1 --2- 172.21.15.76:0/3505746119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9af80d0810 0x7f9af80d0cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:52.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.921+0000 7f9af7fff640 1 -- 172.21.15.76:0/3505746119 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af81f3750 con 0x7f9af806e160 2024-04-25T19:16:52.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.921+0000 7f9af7fff640 1 --2- 172.21.15.76:0/3505746119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9af806e160 0x7f9af81f3210 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800b550 tx=0x7f9ae800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:52.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.921+0000 7f9af4ff9640 1 -- 172.21.15.76:0/3505746119 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8003cf0 con 0x7f9af806e160 2024-04-25T19:16:52.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.921+0000 7f9af4ff9640 1 -- 172.21.15.76:0/3505746119 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ae8015070 con 0x7f9af806e160 2024-04-25T19:16:52.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.922+0000 7f9afe66e640 1 -- 172.21.15.76:0/3505746119 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af81601f0 con 0x7f9af806e160 2024-04-25T19:16:52.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.922+0000 7f9af4ff9640 1 -- 172.21.15.76:0/3505746119 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae80052b0 con 0x7f9af806e160 2024-04-25T19:16:52.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.922+0000 7f9afe66e640 1 -- 172.21.15.76:0/3505746119 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af81604d0 con 0x7f9af806e160 2024-04-25T19:16:52.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.923+0000 7f9afe66e640 1 -- 172.21.15.76:0/3505746119 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9af806a390 con 0x7f9af806e160 2024-04-25T19:16:52.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.923+0000 7f9af4ff9640 1 -- 172.21.15.76:0/3505746119 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9ae8005450 con 0x7f9af806e160 2024-04-25T19:16:52.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.924+0000 7f9af4ff9640 1 --2- 172.21.15.76:0/3505746119 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9ae4074460 0x7f9ae4076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:52.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.924+0000 7f9af4ff9640 1 -- 172.21.15.76:0/3505746119 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f9ae8094f50 con 0x7f9af806e160 2024-04-25T19:16:52.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.924+0000 7f9af77fe640 1 --2- 172.21.15.76:0/3505746119 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9ae4074460 0x7f9ae4076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:52.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.924+0000 7f9af77fe640 1 --2- 172.21.15.76:0/3505746119 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9ae4074460 0x7f9ae4076970 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9aec034000 tx=0x7f9aec02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:52.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:52.929+0000 7f9af4ff9640 1 -- 172.21.15.76:0/3505746119 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ae805f6f0 con 0x7f9af806e160 2024-04-25T19:16:53.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:53.182+0000 7f9afe66e640 1 -- 172.21.15.76:0/3505746119 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f9af80d01a0 con 0x7f9af806e160 2024-04-25T19:16:53.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:53.182+0000 7f9af4ff9640 1 -- 172.21.15.76:0/3505746119 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7f9ae805eff0 con 0x7f9af806e160 2024-04-25T19:16:53.184 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:16:53.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:53.185+0000 7f9afe66e640 1 -- 172.21.15.76:0/3505746119 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9ae4074460 msgr2=0x7f9ae4076970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:53.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:53.185+0000 7f9afe66e640 1 --2- 172.21.15.76:0/3505746119 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9ae4074460 0x7f9ae4076970 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9aec034000 tx=0x7f9aec02e000 comp rx=0 tx=0).stop 2024-04-25T19:16:53.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:53.185+0000 7f9afe66e640 1 -- 172.21.15.76:0/3505746119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9af806e160 msgr2=0x7f9af81f3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:53.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:53.185+0000 7f9afe66e640 1 --2- 172.21.15.76:0/3505746119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9af806e160 0x7f9af81f3210 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800b550 tx=0x7f9ae800ba10 comp rx=0 tx=0).stop 2024-04-25T19:16:53.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:53.185+0000 7f9afe66e640 1 -- 172.21.15.76:0/3505746119 shutdown_connections 2024-04-25T19:16:53.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:53.185+0000 7f9afe66e640 1 --2- 172.21.15.76:0/3505746119 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9ae4074460 0x7f9ae4076970 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:53.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:53.185+0000 7f9afe66e640 1 --2- 172.21.15.76:0/3505746119 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9af80d0810 0x7f9af80d0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:53.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:53.185+0000 7f9afe66e640 1 --2- 172.21.15.76:0/3505746119 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9af80cea70 0x7f9af814ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:53.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:53.185+0000 7f9afe66e640 1 --2- 172.21.15.76:0/3505746119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9af806e160 0x7f9af81f3210 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:53.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:53.185+0000 7f9afe66e640 1 -- 172.21.15.76:0/3505746119 >> 172.21.15.76:0/3505746119 conn(0x7f9af80b3c00 msgr2=0x7f9af80ca550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:53.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:53.185+0000 7f9afe66e640 1 -- 172.21.15.76:0/3505746119 shutdown_connections 2024-04-25T19:16:53.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:53.185+0000 7f9afe66e640 1 -- 172.21.15.76:0/3505746119 wait complete. 2024-04-25T19:16:55.807 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:16:55.809 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:16:56.199 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:16:56.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.306+0000 7f0bfd64c640 1 Processor -- start 2024-04-25T19:16:56.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.306+0000 7f0bfd64c640 1 -- start start 2024-04-25T19:16:56.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.308+0000 7f0bfd64c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0bf80cea70 0x7f0bf80cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:56.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.308+0000 7f0bfd64c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bf806e160 0x7f0bf806e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:56.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.308+0000 7f0bfd64c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bf815ce80 con 0x7f0bf80d0810 2024-04-25T19:16:56.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.308+0000 7f0bfd64c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f0bf815d160 con 0x7f0bf80cea70 2024-04-25T19:16:56.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.308+0000 7f0bfd64c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f0bf815d440 con 0x7f0bf806e160 2024-04-25T19:16:56.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.308+0000 7f0bf67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0bf80cea70 0x7f0bf80cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:56.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.308+0000 7f0bf6ffd640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f0bf80d0810 legacy=0x7f0bf806f000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-04-25T19:16:56.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.308+0000 7f0bf67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0bf80cea70 0x7f0bf80cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46944/0 (socket says 172.21.15.76:46944) 2024-04-25T19:16:56.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.308+0000 7f0bf67fc640 1 -- 172.21.15.76:0/1644031364 learned_addr learned my addr 172.21.15.76:0/1644031364 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:56.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.308+0000 7f0bf77fe640 1 --2- 172.21.15.76:0/1644031364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bf806e160 0x7f0bf806e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:56.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.308+0000 7f0bf67fc640 1 -- 172.21.15.76:0/1644031364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bf806e160 msgr2=0x7f0bf806e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:56.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.308+0000 7f0bf67fc640 1 --2- 172.21.15.76:0/1644031364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bf806e160 0x7f0bf806e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:56.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.308+0000 7f0bf67fc640 1 -- 172.21.15.76:0/1644031364 >> v1:172.21.15.105:6789/0 conn(0x7f0bf80d0810 legacy=0x7f0bf806f000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:16:56.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.308+0000 7f0bf67fc640 1 -- 172.21.15.76:0/1644031364 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf815d720 con 0x7f0bf80cea70 2024-04-25T19:16:56.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.309+0000 7f0bf67fc640 1 --2- 172.21.15.76:0/1644031364 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bf80cea70 0x7f0bf80cee80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009870 tx=0x7f0bec02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ffd5ce485d82de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:56.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.309+0000 7f0bf5ffb640 1 -- 172.21.15.76:0/1644031364 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec037070 con 0x7f0bf80cea70 2024-04-25T19:16:56.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.309+0000 7f0bf5ffb640 1 -- 172.21.15.76:0/1644031364 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bec032440 con 0x7f0bf80cea70 2024-04-25T19:16:56.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.309+0000 7f0bf5ffb640 1 -- 172.21.15.76:0/1644031364 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec032810 con 0x7f0bf80cea70 2024-04-25T19:16:56.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.310+0000 7f0bfd64c640 1 -- 172.21.15.76:0/1644031364 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bf80cea70 msgr2=0x7f0bf80cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:56.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.310+0000 7f0bfd64c640 1 --2- 172.21.15.76:0/1644031364 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bf80cea70 0x7f0bf80cee80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009870 tx=0x7f0bec02a210 comp rx=0 tx=0).stop 2024-04-25T19:16:56.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.310+0000 7f0bfd64c640 1 -- 172.21.15.76:0/1644031364 shutdown_connections 2024-04-25T19:16:56.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.310+0000 7f0bfd64c640 1 --2- 172.21.15.76:0/1644031364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bf806e160 0x7f0bf806e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:56.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.310+0000 7f0bfd64c640 1 --2- 172.21.15.76:0/1644031364 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bf80cea70 0x7f0bf80cee80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:56.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.310+0000 7f0bfd64c640 1 -- 172.21.15.76:0/1644031364 >> 172.21.15.76:0/1644031364 conn(0x7f0bf80b3c00 msgr2=0x7f0bf80b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:56.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.310+0000 7f0bfd64c640 1 -- 172.21.15.76:0/1644031364 shutdown_connections 2024-04-25T19:16:56.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.310+0000 7f0bfd64c640 1 -- 172.21.15.76:0/1644031364 wait complete. 2024-04-25T19:16:56.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.313+0000 7f0bfd64c640 1 Processor -- start 2024-04-25T19:16:56.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.313+0000 7f0bfd64c640 1 -- start start 2024-04-25T19:16:56.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.315+0000 7f0bfd64c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bf806e160 0x7f0bf8151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:56.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.315+0000 7f0bfd64c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bf80cea70 0x7f0bf81f3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:56.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.315+0000 7f0bfd64c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bf80d0810 0x7f0bf8160340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:56.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.315+0000 7f0bfd64c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf806eab0 con 0x7f0bf80cea70 2024-04-25T19:16:56.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.315+0000 7f0bfd64c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf806dc40 con 0x7f0bf806e160 2024-04-25T19:16:56.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.315+0000 7f0bfd64c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f0bf815cb70 con 0x7f0bf80d0810 2024-04-25T19:16:56.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.315+0000 7f0bf6ffd640 1 -- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bf806e160 msgr2=0x7f0bf8151dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3300/0 2024-04-25T19:16:56.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.315+0000 7f0bf6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bf806e160 0x7f0bf8151dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:56.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.315+0000 7f0bf67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bf80cea70 0x7f0bf81f3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:56.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.315+0000 7f0bf67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bf80cea70 0x7f0bf81f3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46948/0 (socket says 172.21.15.76:46948) 2024-04-25T19:16:56.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.315+0000 7f0bf67fc640 1 -- 172.21.15.76:0/1489305802 learned_addr learned my addr 172.21.15.76:0/1489305802 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:56.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.315+0000 7f0bf77fe640 1 --2- 172.21.15.76:0/1489305802 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bf80d0810 0x7f0bf8160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:56.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.316+0000 7f0bf67fc640 1 -- 172.21.15.76:0/1489305802 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bf80d0810 msgr2=0x7f0bf8160340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:56.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.316+0000 7f0bf67fc640 1 --2- 172.21.15.76:0/1489305802 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bf80d0810 0x7f0bf8160340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:56.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.316+0000 7f0bf67fc640 1 -- 172.21.15.76:0/1489305802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bf806e160 msgr2=0x7f0bf8151dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:16:56.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.316+0000 7f0bf67fc640 1 --2- 172.21.15.76:0/1489305802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bf806e160 0x7f0bf8151dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:56.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.316+0000 7f0bf67fc640 1 -- 172.21.15.76:0/1489305802 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf81524d0 con 0x7f0bf80cea70 2024-04-25T19:16:56.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.316+0000 7f0bf77fe640 1 --2- 172.21.15.76:0/1489305802 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bf80d0810 0x7f0bf8160340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:16:56.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.316+0000 7f0bf67fc640 1 --2- 172.21.15.76:0/1489305802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bf80cea70 0x7f0bf81f3800 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0bec000c00 tx=0x7f0bec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:56.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.316+0000 7f0bdf7fe640 1 -- 172.21.15.76:0/1489305802 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec037040 con 0x7f0bf80cea70 2024-04-25T19:16:56.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.316+0000 7f0bdf7fe640 1 -- 172.21.15.76:0/1489305802 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bec042070 con 0x7f0bf80cea70 2024-04-25T19:16:56.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.316+0000 7f0bdf7fe640 1 -- 172.21.15.76:0/1489305802 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec030880 con 0x7f0bf80cea70 2024-04-25T19:16:56.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.316+0000 7f0bfd64c640 1 -- 172.21.15.76:0/1489305802 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf8152790 con 0x7f0bf80cea70 2024-04-25T19:16:56.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.316+0000 7f0bfd64c640 1 -- 172.21.15.76:0/1489305802 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf81f6820 con 0x7f0bf80cea70 2024-04-25T19:16:56.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.318+0000 7f0bfd64c640 1 -- 172.21.15.76:0/1489305802 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bf806a390 con 0x7f0bf80cea70 2024-04-25T19:16:56.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.318+0000 7f0bdf7fe640 1 -- 172.21.15.76:0/1489305802 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0bec030a20 con 0x7f0bf80cea70 2024-04-25T19:16:56.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.319+0000 7f0bdf7fe640 1 --2- 172.21.15.76:0/1489305802 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0be4074460 0x7f0be4076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:56.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.319+0000 7f0bdf7fe640 1 -- 172.21.15.76:0/1489305802 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f0bec0b5950 con 0x7f0bf80cea70 2024-04-25T19:16:56.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.319+0000 7f0bf6ffd640 1 --2- 172.21.15.76:0/1489305802 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0be4074460 0x7f0be4076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:56.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.320+0000 7f0bf6ffd640 1 --2- 172.21.15.76:0/1489305802 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0be4074460 0x7f0be4076970 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0be8006fd0 tx=0x7f0be8006d80 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:56.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.327+0000 7f0bdf7fe640 1 -- 172.21.15.76:0/1489305802 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0bec080180 con 0x7f0bf80cea70 2024-04-25T19:16:56.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.628+0000 7f0bfd64c640 1 -- 172.21.15.76:0/1489305802 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0bf80b8070 con 0x7f0bf80cea70 2024-04-25T19:16:56.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.629+0000 7f0bdf7fe640 1 -- 172.21.15.76:0/1489305802 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7f0bec07fa80 con 0x7f0bf80cea70 2024-04-25T19:16:56.631 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:16:56.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.631+0000 7f0bfd64c640 1 -- 172.21.15.76:0/1489305802 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0be4074460 msgr2=0x7f0be4076970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:56.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.631+0000 7f0bfd64c640 1 --2- 172.21.15.76:0/1489305802 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0be4074460 0x7f0be4076970 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0be8006fd0 tx=0x7f0be8006d80 comp rx=0 tx=0).stop 2024-04-25T19:16:56.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.631+0000 7f0bfd64c640 1 -- 172.21.15.76:0/1489305802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bf80cea70 msgr2=0x7f0bf81f3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:56.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.631+0000 7f0bfd64c640 1 --2- 172.21.15.76:0/1489305802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bf80cea70 0x7f0bf81f3800 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0bec000c00 tx=0x7f0bec0046c0 comp rx=0 tx=0).stop 2024-04-25T19:16:56.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.631+0000 7f0bfd64c640 1 -- 172.21.15.76:0/1489305802 shutdown_connections 2024-04-25T19:16:56.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.631+0000 7f0bfd64c640 1 --2- 172.21.15.76:0/1489305802 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0be4074460 0x7f0be4076970 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:56.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.631+0000 7f0bfd64c640 1 --2- 172.21.15.76:0/1489305802 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0bf80d0810 0x7f0bf8160340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:56.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.631+0000 7f0bfd64c640 1 --2- 172.21.15.76:0/1489305802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bf80cea70 0x7f0bf81f3800 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:56.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.631+0000 7f0bfd64c640 1 --2- 172.21.15.76:0/1489305802 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bf806e160 0x7f0bf8151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:56.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.631+0000 7f0bfd64c640 1 -- 172.21.15.76:0/1489305802 >> 172.21.15.76:0/1489305802 conn(0x7f0bf80b3c00 msgr2=0x7f0bf80cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:56.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.631+0000 7f0bfd64c640 1 -- 172.21.15.76:0/1489305802 shutdown_connections 2024-04-25T19:16:56.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:56.631+0000 7f0bfd64c640 1 -- 172.21.15.76:0/1489305802 wait complete. 2024-04-25T19:16:59.648 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:16:59.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.754+0000 7fcb6a692640 1 Processor -- start 2024-04-25T19:16:59.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.754+0000 7fcb6a692640 1 -- start start 2024-04-25T19:16:59.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.756+0000 7fcb6a692640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcb640cea70 0x7fcb640cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:59.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.756+0000 7fcb6a692640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcb6406e160 0x7fcb6406e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:59.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.756+0000 7fcb6a692640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb6415ce80 con 0x7fcb640d0810 2024-04-25T19:16:59.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.756+0000 7fcb6a692640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fcb6415d160 con 0x7fcb6406e160 2024-04-25T19:16:59.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.756+0000 7fcb6a692640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fcb6415d440 con 0x7fcb640cea70 2024-04-25T19:16:59.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.756+0000 7fcb63fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fcb640d0810 0x7fcb6406f000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33876/0 (socket says 172.21.15.76:33876) 2024-04-25T19:16:59.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.756+0000 7fcb63fff640 1 -- 172.21.15.76:0/4079760131 learned_addr learned my addr 172.21.15.76:0/4079760131 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:16:59.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.756+0000 7fcb68c2f640 1 --2- 172.21.15.76:0/4079760131 >> v2:172.21.15.76:3300/0 conn(0x7fcb6406e160 0x7fcb6406e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:59.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.756+0000 7fcb637fe640 1 --2- 172.21.15.76:0/4079760131 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcb640cea70 0x7fcb640cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:59.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.757+0000 7fcb68c2f640 1 -- 172.21.15.76:0/4079760131 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcb640cea70 msgr2=0x7fcb640cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:16:59.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.757+0000 7fcb68c2f640 1 --2- 172.21.15.76:0/4079760131 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcb640cea70 0x7fcb640cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:59.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.757+0000 7fcb68c2f640 1 -- 172.21.15.76:0/4079760131 >> v1:172.21.15.76:6789/0 conn(0x7fcb640d0810 legacy=0x7fcb6406f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:59.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.757+0000 7fcb68c2f640 1 -- 172.21.15.76:0/4079760131 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6415d720 con 0x7fcb6406e160 2024-04-25T19:16:59.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.758+0000 7fcb68c2f640 1 --2- 172.21.15.76:0/4079760131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb6406e160 0x7fcb6406e570 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009d90 tx=0x7fcb5c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163628482244d53f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:59.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.758+0000 7fcb62ffd640 1 -- 172.21.15.76:0/4079760131 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c007740 con 0x7fcb6406e160 2024-04-25T19:16:59.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.758+0000 7fcb62ffd640 1 -- 172.21.15.76:0/4079760131 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb5c0078e0 con 0x7fcb6406e160 2024-04-25T19:16:59.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.758+0000 7fcb62ffd640 1 -- 172.21.15.76:0/4079760131 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c007cb0 con 0x7fcb6406e160 2024-04-25T19:16:59.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.759+0000 7fcb6a692640 1 -- 172.21.15.76:0/4079760131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb6406e160 msgr2=0x7fcb6406e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:59.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.759+0000 7fcb6a692640 1 --2- 172.21.15.76:0/4079760131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb6406e160 0x7fcb6406e570 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009d90 tx=0x7fcb5c02a660 comp rx=0 tx=0).stop 2024-04-25T19:16:59.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.759+0000 7fcb6a692640 1 -- 172.21.15.76:0/4079760131 shutdown_connections 2024-04-25T19:16:59.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.759+0000 7fcb6a692640 1 --2- 172.21.15.76:0/4079760131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb6406e160 0x7fcb6406e570 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:59.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.759+0000 7fcb6a692640 1 --2- 172.21.15.76:0/4079760131 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcb640cea70 0x7fcb640cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:59.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.759+0000 7fcb6a692640 1 -- 172.21.15.76:0/4079760131 >> 172.21.15.76:0/4079760131 conn(0x7fcb640b3c00 msgr2=0x7fcb640b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:16:59.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.759+0000 7fcb6a692640 1 -- 172.21.15.76:0/4079760131 shutdown_connections 2024-04-25T19:16:59.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.759+0000 7fcb6a692640 1 -- 172.21.15.76:0/4079760131 wait complete. 2024-04-25T19:16:59.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.761+0000 7fcb6a692640 1 Processor -- start 2024-04-25T19:16:59.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.762+0000 7fcb6a692640 1 -- start start 2024-04-25T19:16:59.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.764+0000 7fcb6a692640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcb6406e160 0x7fcb641f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:59.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.764+0000 7fcb6a692640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb640cea70 0x7fcb64155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:59.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.764+0000 7fcb6a692640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb640d0810 0x7fcb640b6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:59.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.764+0000 7fcb6a692640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6406eab0 con 0x7fcb640cea70 2024-04-25T19:16:59.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.764+0000 7fcb637fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb640cea70 0x7fcb64155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:59.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.764+0000 7fcb68c2f640 1 -- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb640d0810 msgr2=0x7fcb640b6b40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3300/0 2024-04-25T19:16:59.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.764+0000 7fcb637fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb640cea70 0x7fcb64155ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33890/0 (socket says 172.21.15.76:33890) 2024-04-25T19:16:59.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.764+0000 7fcb637fe640 1 -- 172.21.15.76:0/3366844064 learned_addr learned my addr 172.21.15.76:0/3366844064 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:16:59.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.764+0000 7fcb68c2f640 1 --2- 172.21.15.76:0/3366844064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb640d0810 0x7fcb640b6b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:16:59.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.764+0000 7fcb6a692640 1 -- 172.21.15.76:0/3366844064 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6406dc40 con 0x7fcb640d0810 2024-04-25T19:16:59.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.764+0000 7fcb6a692640 1 -- 172.21.15.76:0/3366844064 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fcb6415cb70 con 0x7fcb6406e160 2024-04-25T19:16:59.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.764+0000 7fcb63fff640 1 --2- 172.21.15.76:0/3366844064 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcb6406e160 0x7fcb641f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:59.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.764+0000 7fcb63fff640 1 -- 172.21.15.76:0/3366844064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb640d0810 msgr2=0x7fcb640b6b40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:16:59.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.764+0000 7fcb63fff640 1 --2- 172.21.15.76:0/3366844064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb640d0810 0x7fcb640b6b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:59.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.764+0000 7fcb63fff640 1 -- 172.21.15.76:0/3366844064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb640cea70 msgr2=0x7fcb64155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:16:59.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.764+0000 7fcb63fff640 1 --2- 172.21.15.76:0/3366844064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb640cea70 0x7fcb64155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:16:59.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.765+0000 7fcb63fff640 1 -- 172.21.15.76:0/3366844064 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb640b7db0 con 0x7fcb6406e160 2024-04-25T19:16:59.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.765+0000 7fcb637fe640 1 --2- 172.21.15.76:0/3366844064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb640cea70 0x7fcb64155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:16:59.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.765+0000 7fcb63fff640 1 --2- 172.21.15.76:0/3366844064 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcb6406e160 0x7fcb641f6e10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcb6408f3f0 tx=0x7fcb5400c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:59.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.765+0000 7fcb60ff9640 1 -- 172.21.15.76:0/3366844064 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54017070 con 0x7fcb6406e160 2024-04-25T19:16:59.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.765+0000 7fcb60ff9640 1 -- 172.21.15.76:0/3366844064 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb54005d00 con 0x7fcb6406e160 2024-04-25T19:16:59.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.765+0000 7fcb60ff9640 1 -- 172.21.15.76:0/3366844064 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb54007310 con 0x7fcb6406e160 2024-04-25T19:16:59.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.766+0000 7fcb6a692640 1 -- 172.21.15.76:0/3366844064 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb640b5090 con 0x7fcb6406e160 2024-04-25T19:16:59.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.766+0000 7fcb6a692640 1 -- 172.21.15.76:0/3366844064 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb641f6610 con 0x7fcb6406e160 2024-04-25T19:16:59.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.767+0000 7fcb6a692640 1 -- 172.21.15.76:0/3366844064 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb641f6400 con 0x7fcb6406e160 2024-04-25T19:16:59.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.768+0000 7fcb60ff9640 1 -- 172.21.15.76:0/3366844064 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcb5401a070 con 0x7fcb6406e160 2024-04-25T19:16:59.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.769+0000 7fcb60ff9640 1 --2- 172.21.15.76:0/3366844064 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fcb50074490 0x7fcb500769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:16:59.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.769+0000 7fcb60ff9640 1 -- 172.21.15.76:0/3366844064 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fcb540943a0 con 0x7fcb6406e160 2024-04-25T19:16:59.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.769+0000 7fcb637fe640 1 --2- 172.21.15.76:0/3366844064 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fcb50074490 0x7fcb500769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:16:59.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.770+0000 7fcb637fe640 1 --2- 172.21.15.76:0/3366844064 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fcb50074490 0x7fcb500769a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcb6415e3b0 tx=0x7fcb58009190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:16:59.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:16:59.775+0000 7fcb60ff9640 1 -- 172.21.15.76:0/3366844064 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcb5405e4f0 con 0x7fcb6406e160 2024-04-25T19:17:00.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:00.218+0000 7fcb6a692640 1 -- 172.21.15.76:0/3366844064 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fcb6406a390 con 0x7fcb6406e160 2024-04-25T19:17:00.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:00.218+0000 7fcb60ff9640 1 -- 172.21.15.76:0/3366844064 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7fcb540627f0 con 0x7fcb6406e160 2024-04-25T19:17:00.221 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:17:00.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:00.221+0000 7fcb6a692640 1 -- 172.21.15.76:0/3366844064 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fcb50074490 msgr2=0x7fcb500769a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:00.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:00.221+0000 7fcb6a692640 1 --2- 172.21.15.76:0/3366844064 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fcb50074490 0x7fcb500769a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcb6415e3b0 tx=0x7fcb58009190 comp rx=0 tx=0).stop 2024-04-25T19:17:00.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:00.221+0000 7fcb6a692640 1 -- 172.21.15.76:0/3366844064 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcb6406e160 msgr2=0x7fcb641f6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:00.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:00.221+0000 7fcb6a692640 1 --2- 172.21.15.76:0/3366844064 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcb6406e160 0x7fcb641f6e10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcb6408f3f0 tx=0x7fcb5400c760 comp rx=0 tx=0).stop 2024-04-25T19:17:00.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:00.221+0000 7fcb6a692640 1 -- 172.21.15.76:0/3366844064 shutdown_connections 2024-04-25T19:17:00.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:00.221+0000 7fcb6a692640 1 --2- 172.21.15.76:0/3366844064 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fcb50074490 0x7fcb500769a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:00.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:00.221+0000 7fcb6a692640 1 --2- 172.21.15.76:0/3366844064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb640d0810 0x7fcb640b6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:00.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:00.221+0000 7fcb6a692640 1 --2- 172.21.15.76:0/3366844064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcb640cea70 0x7fcb64155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:00.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:00.221+0000 7fcb6a692640 1 --2- 172.21.15.76:0/3366844064 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcb6406e160 0x7fcb641f6e10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:00.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:00.221+0000 7fcb6a692640 1 -- 172.21.15.76:0/3366844064 >> 172.21.15.76:0/3366844064 conn(0x7fcb640b3c00 msgr2=0x7fcb640b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:00.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:00.221+0000 7fcb6a692640 1 -- 172.21.15.76:0/3366844064 shutdown_connections 2024-04-25T19:17:00.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:00.222+0000 7fcb6a692640 1 -- 172.21.15.76:0/3366844064 wait complete. 2024-04-25T19:17:03.238 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:17:03.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.351+0000 7fbae9492640 1 Processor -- start 2024-04-25T19:17:03.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.351+0000 7fbae9492640 1 -- start start 2024-04-25T19:17:03.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.352+0000 7fbae9492640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbae40d0810 0x7fbae406f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.352+0000 7fbae9492640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbae406e870 0x7fbae40cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.352+0000 7fbae9492640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbae415ce80 con 0x7fbae40cea70 2024-04-25T19:17:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.352+0000 7fbae9492640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fbae415d160 con 0x7fbae40d0810 2024-04-25T19:17:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.352+0000 7fbae9492640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fbae415d440 con 0x7fbae406e870 2024-04-25T19:17:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.352+0000 7fbae27fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fbae40cea70 0x7fbae406e160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33886/0 (socket says 172.21.15.76:33886) 2024-04-25T19:17:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.352+0000 7fbae27fc640 1 -- 172.21.15.76:0/1563495899 learned_addr learned my addr 172.21.15.76:0/1563495899 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:17:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.353+0000 7fbae37fe640 1 --2- 172.21.15.76:0/1563495899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbae406e870 0x7fbae40cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.352+0000 7fbae2ffd640 1 -- 172.21.15.76:0/1563495899 >> v2:172.21.15.105:3300/0 conn(0x7fbae40d0810 msgr2=0x7fbae406f000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-04-25T19:17:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.353+0000 7fbae2ffd640 1 --2- 172.21.15.76:0/1563495899 >> v2:172.21.15.105:3300/0 conn(0x7fbae40d0810 0x7fbae406f000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:17:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.353+0000 7fbae37fe640 1 -- 172.21.15.76:0/1563495899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbae406e870 msgr2=0x7fbae40cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T19:17:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.353+0000 7fbae37fe640 1 -- 172.21.15.76:0/1563495899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbae406e870 msgr2=0x7fbae40cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:17:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.353+0000 7fbae37fe640 1 --2- 172.21.15.76:0/1563495899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbae406e870 0x7fbae40cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:03.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.353+0000 7fbae37fe640 1 --2- 172.21.15.76:0/1563495899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbae406e870 0x7fbae40cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:17:03.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.554+0000 7fbae2ffd640 1 -- 172.21.15.76:0/1563495899 >> v2:172.21.15.105:3300/0 conn(0x7fbae40d0810 msgr2=0x7fbae406f000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-04-25T19:17:03.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.554+0000 7fbae2ffd640 1 --2- 172.21.15.76:0/1563495899 >> v2:172.21.15.105:3300/0 conn(0x7fbae40d0810 0x7fbae406f000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T19:17:03.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.554+0000 7fbae37fe640 1 --2- 172.21.15.76:0/1563495899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbae406e870 0x7fbae40cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:03.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.554+0000 7fbae37fe640 1 -- 172.21.15.76:0/1563495899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbae406e870 msgr2=0x7fbae40cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T19:17:03.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.554+0000 7fbae37fe640 1 -- 172.21.15.76:0/1563495899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbae406e870 msgr2=0x7fbae40cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:17:03.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.554+0000 7fbae37fe640 1 --2- 172.21.15.76:0/1563495899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbae406e870 0x7fbae40cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:03.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.554+0000 7fbae37fe640 1 --2- 172.21.15.76:0/1563495899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbae406e870 0x7fbae40cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T19:17:03.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.955+0000 7fbae2ffd640 1 -- 172.21.15.76:0/1563495899 >> v2:172.21.15.105:3300/0 conn(0x7fbae40d0810 msgr2=0x7fbae406f000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-04-25T19:17:03.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.955+0000 7fbae2ffd640 1 --2- 172.21.15.76:0/1563495899 >> v2:172.21.15.105:3300/0 conn(0x7fbae40d0810 0x7fbae406f000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T19:17:03.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.955+0000 7fbae37fe640 1 --2- 172.21.15.76:0/1563495899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbae406e870 0x7fbae40cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:03.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.955+0000 7fbae37fe640 1 -- 172.21.15.76:0/1563495899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbae406e870 msgr2=0x7fbae40cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T19:17:03.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.955+0000 7fbae37fe640 1 -- 172.21.15.76:0/1563495899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbae406e870 msgr2=0x7fbae40cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:17:03.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.955+0000 7fbae37fe640 1 --2- 172.21.15.76:0/1563495899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbae406e870 0x7fbae40cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:03.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:03.955+0000 7fbae37fe640 1 --2- 172.21.15.76:0/1563495899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbae406e870 0x7fbae40cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T19:17:04.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:04.756+0000 7fbae2ffd640 1 -- 172.21.15.76:0/1563495899 >> v2:172.21.15.105:3300/0 conn(0x7fbae40d0810 msgr2=0x7fbae406f000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-04-25T19:17:04.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:04.756+0000 7fbae2ffd640 1 --2- 172.21.15.76:0/1563495899 >> v2:172.21.15.105:3300/0 conn(0x7fbae40d0810 0x7fbae406f000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T19:17:04.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:04.756+0000 7fbae37fe640 1 --2- 172.21.15.76:0/1563495899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbae406e870 0x7fbae40cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:04.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:04.756+0000 7fbae37fe640 1 -- 172.21.15.76:0/1563495899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbae406e870 msgr2=0x7fbae40cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T19:17:04.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:04.756+0000 7fbae37fe640 1 -- 172.21.15.76:0/1563495899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbae406e870 msgr2=0x7fbae40cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:17:04.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:04.756+0000 7fbae37fe640 1 --2- 172.21.15.76:0/1563495899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbae406e870 0x7fbae40cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:04.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:04.756+0000 7fbae37fe640 1 --2- 172.21.15.76:0/1563495899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbae406e870 0x7fbae40cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T19:17:06.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:06.352+0000 7fbae0ff9640 1 -- 172.21.15.76:0/1563495899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbae406e870 msgr2=0x7fbae40cee80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:17:06.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:06.352+0000 7fbae0ff9640 1 --2- 172.21.15.76:0/1563495899 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbae406e870 0x7fbae40cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:06.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:06.352+0000 7fbae0ff9640 1 -- 172.21.15.76:0/1563495899 >> v2:172.21.15.105:3300/0 conn(0x7fbae40d0810 msgr2=0x7fbae406f000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:17:06.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:06.352+0000 7fbae0ff9640 1 --2- 172.21.15.76:0/1563495899 >> v2:172.21.15.105:3300/0 conn(0x7fbae40d0810 0x7fbae406f000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:06.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:06.352+0000 7fbae0ff9640 1 -- 172.21.15.76:0/1563495899 >> v1:172.21.15.76:6789/0 conn(0x7fbae40cea70 legacy=0x7fbae406e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:06.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:06.352+0000 7fbae0ff9640 1 --2- 172.21.15.76:0/1563495899 >> v2:172.21.15.76:3300/0 conn(0x7fbad00073a0 0x7fbad00098b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:06.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:06.352+0000 7fbae0ff9640 1 -- 172.21.15.76:0/1563495899 --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbae415d720 con 0x7fbad00036a0 2024-04-25T19:17:06.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:06.352+0000 7fbae0ff9640 1 -- 172.21.15.76:0/1563495899 --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fbae415d160 con 0x7fbad00073a0 2024-04-25T19:17:06.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:06.353+0000 7fbae2ffd640 1 -- 172.21.15.76:0/1563495899 >> v1:172.21.15.105:6789/0 conn(0x7fbad00036a0 legacy=0x7fbad0005b70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-04-25T19:17:06.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:06.353+0000 7fbae27fc640 1 --2- 172.21.15.76:0/1563495899 >> v2:172.21.15.76:3300/0 conn(0x7fbad00073a0 0x7fbad00098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:06.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:06.353+0000 7fbae27fc640 1 -- 172.21.15.76:0/1563495899 >> v2:172.21.15.76:3300/0 conn(0x7fbad00073a0 msgr2=0x7fbad00098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T19:17:06.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:06.353+0000 7fbae27fc640 1 -- 172.21.15.76:0/1563495899 >> v2:172.21.15.76:3300/0 conn(0x7fbad00073a0 msgr2=0x7fbad00098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:17:06.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:06.353+0000 7fbae27fc640 1 --2- 172.21.15.76:0/1563495899 >> v2:172.21.15.76:3300/0 conn(0x7fbad00073a0 0x7fbad00098b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:06.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:06.353+0000 7fbae27fc640 1 --2- 172.21.15.76:0/1563495899 >> v2:172.21.15.76:3300/0 conn(0x7fbad00073a0 0x7fbad00098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:17:06.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:06.554+0000 7fbae2ffd640 1 -- 172.21.15.76:0/1563495899 >> v1:172.21.15.105:6789/0 conn(0x7fbad00036a0 legacy=0x7fbad0005b70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-04-25T19:17:06.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:06.554+0000 7fbae27fc640 1 --2- 172.21.15.76:0/1563495899 >> v2:172.21.15.76:3300/0 conn(0x7fbad00073a0 0x7fbad00098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:06.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:06.554+0000 7fbae27fc640 1 -- 172.21.15.76:0/1563495899 >> v2:172.21.15.76:3300/0 conn(0x7fbad00073a0 msgr2=0x7fbad00098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T19:17:06.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:06.554+0000 7fbae27fc640 1 -- 172.21.15.76:0/1563495899 >> v2:172.21.15.76:3300/0 conn(0x7fbad00073a0 msgr2=0x7fbad00098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:17:06.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:06.554+0000 7fbae27fc640 1 --2- 172.21.15.76:0/1563495899 >> v2:172.21.15.76:3300/0 conn(0x7fbad00073a0 0x7fbad00098b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:06.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:06.554+0000 7fbae27fc640 1 --2- 172.21.15.76:0/1563495899 >> v2:172.21.15.76:3300/0 conn(0x7fbad00073a0 0x7fbad00098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T19:17:06.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:06.955+0000 7fbae2ffd640 1 -- 172.21.15.76:0/1563495899 >> v1:172.21.15.105:6789/0 conn(0x7fbad00036a0 legacy=0x7fbad0005b70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-04-25T19:17:06.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:06.955+0000 7fbae27fc640 1 --2- 172.21.15.76:0/1563495899 >> v2:172.21.15.76:3300/0 conn(0x7fbad00073a0 0x7fbad00098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:06.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:06.955+0000 7fbae27fc640 1 -- 172.21.15.76:0/1563495899 >> v2:172.21.15.76:3300/0 conn(0x7fbad00073a0 msgr2=0x7fbad00098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T19:17:06.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:06.955+0000 7fbae27fc640 1 -- 172.21.15.76:0/1563495899 >> v2:172.21.15.76:3300/0 conn(0x7fbad00073a0 msgr2=0x7fbad00098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:17:06.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:06.955+0000 7fbae27fc640 1 --2- 172.21.15.76:0/1563495899 >> v2:172.21.15.76:3300/0 conn(0x7fbad00073a0 0x7fbad00098b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:06.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:06.955+0000 7fbae27fc640 1 --2- 172.21.15.76:0/1563495899 >> v2:172.21.15.76:3300/0 conn(0x7fbad00073a0 0x7fbad00098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T19:17:07.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.756+0000 7fbae27fc640 1 --2- 172.21.15.76:0/1563495899 >> v2:172.21.15.76:3300/0 conn(0x7fbad00073a0 0x7fbad00098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:07.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.756+0000 7fbae2ffd640 1 -- 172.21.15.76:0/1563495899 >> v1:172.21.15.105:6789/0 conn(0x7fbad00036a0 legacy=0x7fbad0005b70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-04-25T19:17:07.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.756+0000 7fbae27fc640 1 -- 172.21.15.76:0/1563495899 >> v1:172.21.15.105:6789/0 conn(0x7fbad00036a0 legacy=0x7fbad0005b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:17:07.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.756+0000 7fbae27fc640 1 -- 172.21.15.76:0/1563495899 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad000b070 con 0x7fbad00073a0 2024-04-25T19:17:07.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.757+0000 7fbae27fc640 1 --2- 172.21.15.76:0/1563495899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbad00073a0 0x7fbad00098b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbae4090270 tx=0x7fbad802a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4114d5a8f95c458f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:07.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.757+0000 7fbae1ffb640 1 -- 172.21.15.76:0/1563495899 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8038070 con 0x7fbad00073a0 2024-04-25T19:17:07.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.757+0000 7fbae1ffb640 1 -- 172.21.15.76:0/1563495899 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbad8033410 con 0x7fbad00073a0 2024-04-25T19:17:07.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.757+0000 7fbae1ffb640 1 -- 172.21.15.76:0/1563495899 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8033800 con 0x7fbad00073a0 2024-04-25T19:17:07.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.759+0000 7fbae9492640 1 -- 172.21.15.76:0/1563495899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbad00073a0 msgr2=0x7fbad00098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:07.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.759+0000 7fbae9492640 1 --2- 172.21.15.76:0/1563495899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbad00073a0 0x7fbad00098b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbae4090270 tx=0x7fbad802a800 comp rx=0 tx=0).stop 2024-04-25T19:17:07.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.759+0000 7fbae37fe640 1 -- 172.21.15.76:0/1563495899 reap_dead start 2024-04-25T19:17:07.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.759+0000 7fbae9492640 1 -- 172.21.15.76:0/1563495899 shutdown_connections 2024-04-25T19:17:07.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.759+0000 7fbae9492640 1 --2- 172.21.15.76:0/1563495899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbad00073a0 0x7fbad00098b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:07.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.759+0000 7fbae9492640 1 -- 172.21.15.76:0/1563495899 >> 172.21.15.76:0/1563495899 conn(0x7fbae40b3c00 msgr2=0x7fbae40b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:07.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.759+0000 7fbae9492640 1 -- 172.21.15.76:0/1563495899 shutdown_connections 2024-04-25T19:17:07.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.759+0000 7fbae9492640 1 -- 172.21.15.76:0/1563495899 wait complete. 2024-04-25T19:17:07.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.762+0000 7fbae9492640 1 Processor -- start 2024-04-25T19:17:07.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.762+0000 7fbae9492640 1 -- start start 2024-04-25T19:17:07.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.764+0000 7fbae9492640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbad00073a0 0x7fbae4153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:07.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.764+0000 7fbae9492640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbae4160b60 0x7fbae4150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:07.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.764+0000 7fbae9492640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbae415dd30 0x7fbae41507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:07.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.764+0000 7fbae9492640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbae415cb70 con 0x7fbae415dd30 2024-04-25T19:17:07.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.764+0000 7fbae9492640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbae406dc40 con 0x7fbae4160b60 2024-04-25T19:17:07.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.764+0000 7fbae9492640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fbae406f540 con 0x7fbad00073a0 2024-04-25T19:17:07.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.764+0000 7fbae27fc640 1 -- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbae4160b60 msgr2=0x7fbae4150270 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3300/0 2024-04-25T19:17:07.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.764+0000 7fbae27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbae4160b60 0x7fbae4150270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:17:07.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.764+0000 7fbae2ffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbad00073a0 0x7fbae4153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:07.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.764+0000 7fbae37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbae415dd30 0x7fbae41507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:07.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.764+0000 7fbae2ffd640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbad00073a0 0x7fbae4153610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:37666/0 (socket says 172.21.15.76:37666) 2024-04-25T19:17:07.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.764+0000 7fbae2ffd640 1 -- 172.21.15.76:0/2679398530 learned_addr learned my addr 172.21.15.76:0/2679398530 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:17:07.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.765+0000 7fbae37fe640 1 -- 172.21.15.76:0/2679398530 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbad00073a0 msgr2=0x7fbae4153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:07.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.765+0000 7fbae37fe640 1 --2- 172.21.15.76:0/2679398530 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbad00073a0 0x7fbae4153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:07.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.765+0000 7fbae37fe640 1 -- 172.21.15.76:0/2679398530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbae4160b60 msgr2=0x7fbae4150270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:17:07.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.765+0000 7fbae37fe640 1 --2- 172.21.15.76:0/2679398530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbae4160b60 0x7fbae4150270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:07.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.765+0000 7fbae37fe640 1 -- 172.21.15.76:0/2679398530 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae4150cf0 con 0x7fbae415dd30 2024-04-25T19:17:07.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.765+0000 7fbae37fe640 1 --2- 172.21.15.76:0/2679398530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbae415dd30 0x7fbae41507b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbae408fd70 tx=0x7fbadc012610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:07.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.765+0000 7fbacb7fe640 1 -- 172.21.15.76:0/2679398530 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc01a070 con 0x7fbae415dd30 2024-04-25T19:17:07.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.765+0000 7fbae9492640 1 -- 172.21.15.76:0/2679398530 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae415fc20 con 0x7fbae415dd30 2024-04-25T19:17:07.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.765+0000 7fbacb7fe640 1 -- 172.21.15.76:0/2679398530 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbadc012e90 con 0x7fbae415dd30 2024-04-25T19:17:07.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.765+0000 7fbacb7fe640 1 -- 172.21.15.76:0/2679398530 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc01f650 con 0x7fbae415dd30 2024-04-25T19:17:07.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.765+0000 7fbae9492640 1 -- 172.21.15.76:0/2679398530 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae41581c0 con 0x7fbae415dd30 2024-04-25T19:17:07.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.767+0000 7fbae9492640 1 -- 172.21.15.76:0/2679398530 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbac4005290 con 0x7fbae415dd30 2024-04-25T19:17:07.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.768+0000 7fbacb7fe640 1 -- 172.21.15.76:0/2679398530 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbadc01d070 con 0x7fbae415dd30 2024-04-25T19:17:07.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.768+0000 7fbacb7fe640 1 --2- 172.21.15.76:0/2679398530 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbad0075210 0x7fbad0077720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:07.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.768+0000 7fbacb7fe640 1 -- 172.21.15.76:0/2679398530 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fbadc0959a0 con 0x7fbae415dd30 2024-04-25T19:17:07.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.769+0000 7fbae2ffd640 1 --2- 172.21.15.76:0/2679398530 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbad0075210 0x7fbad0077720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:07.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.769+0000 7fbae2ffd640 1 --2- 172.21.15.76:0/2679398530 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbad0075210 0x7fbad0077720 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbad4009770 tx=0x7fbad4009700 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:07.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:07.777+0000 7fbacb7fe640 1 -- 172.21.15.76:0/2679398530 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbadc024030 con 0x7fbae415dd30 2024-04-25T19:17:08.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.087+0000 7fbae9492640 1 -- 172.21.15.76:0/2679398530 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fbac4005580 con 0x7fbae415dd30 2024-04-25T19:17:08.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.088+0000 7fbacb7fe640 1 -- 172.21.15.76:0/2679398530 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7fbadc010050 con 0x7fbae415dd30 2024-04-25T19:17:08.089 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":14,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:17:08.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.090+0000 7fbae9492640 1 -- 172.21.15.76:0/2679398530 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbad0075210 msgr2=0x7fbad0077720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:08.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.090+0000 7fbae9492640 1 --2- 172.21.15.76:0/2679398530 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbad0075210 0x7fbad0077720 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbad4009770 tx=0x7fbad4009700 comp rx=0 tx=0).stop 2024-04-25T19:17:08.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.090+0000 7fbae9492640 1 -- 172.21.15.76:0/2679398530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbae415dd30 msgr2=0x7fbae41507b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:08.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.090+0000 7fbae9492640 1 --2- 172.21.15.76:0/2679398530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbae415dd30 0x7fbae41507b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbae408fd70 tx=0x7fbadc012610 comp rx=0 tx=0).stop 2024-04-25T19:17:08.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.090+0000 7fbae9492640 1 -- 172.21.15.76:0/2679398530 shutdown_connections 2024-04-25T19:17:08.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.090+0000 7fbae9492640 1 --2- 172.21.15.76:0/2679398530 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbad0075210 0x7fbad0077720 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:08.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.090+0000 7fbae9492640 1 --2- 172.21.15.76:0/2679398530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbae415dd30 0x7fbae41507b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:08.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.090+0000 7fbae9492640 1 --2- 172.21.15.76:0/2679398530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbae4160b60 0x7fbae4150270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:08.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.090+0000 7fbae9492640 1 --2- 172.21.15.76:0/2679398530 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbad00073a0 0x7fbae4153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:08.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.090+0000 7fbae9492640 1 -- 172.21.15.76:0/2679398530 >> 172.21.15.76:0/2679398530 conn(0x7fbae40b3c00 msgr2=0x7fbae40ca550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:08.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.090+0000 7fbae9492640 1 -- 172.21.15.76:0/2679398530 shutdown_connections 2024-04-25T19:17:08.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.090+0000 7fbae9492640 1 -- 172.21.15.76:0/2679398530 wait complete. 2024-04-25T19:17:08.102 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-04-25T19:17:08.103 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-04-25T19:17:08.103 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T19:17:08.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.211+0000 7ff1a0e92640 1 Processor -- start 2024-04-25T19:17:08.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.211+0000 7ff1a0e92640 1 -- start start 2024-04-25T19:17:08.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.213+0000 7ff1a0e92640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff19c15aa10 0x7ff19c15ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:08.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.213+0000 7ff1a0e92640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff19c0cb820 con 0x7ff19c0c6040 2024-04-25T19:17:08.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.214+0000 7ff1a0e92640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff19c0cbac0 con 0x7ff19c154c20 2024-04-25T19:17:08.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.214+0000 7ff1a0e92640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff19c0cc730 con 0x7ff19c15aa10 2024-04-25T19:17:08.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.214+0000 7ff199d9b640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7ff19c154c20 legacy=0x7ff19c0cd020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-04-25T19:17:08.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.214+0000 7ff19ad9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff19c15aa10 0x7ff19c15ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:08.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.214+0000 7ff19ad9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff19c15aa10 0x7ff19c15ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33934/0 (socket says 172.21.15.76:33934) 2024-04-25T19:17:08.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.214+0000 7ff19ad9d640 1 -- 172.21.15.76:0/1166708411 learned_addr learned my addr 172.21.15.76:0/1166708411 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:17:08.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.214+0000 7ff19ad9d640 1 -- 172.21.15.76:0/1166708411 >> v1:172.21.15.105:6789/0 conn(0x7ff19c154c20 legacy=0x7ff19c0cd020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:17:08.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.214+0000 7ff19ad9d640 1 -- 172.21.15.76:0/1166708411 >> v1:172.21.15.76:6789/0 conn(0x7ff19c0c6040 legacy=0x7ff19c0cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:08.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.214+0000 7ff19ad9d640 1 -- 172.21.15.76:0/1166708411 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff19c0cbda0 con 0x7ff19c15aa10 2024-04-25T19:17:08.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.215+0000 7ff19ad9d640 1 --2- 172.21.15.76:0/1166708411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c15aa10 0x7ff19c15ae20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff194009e50 tx=0x7ff19402a7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2e043807768fc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:08.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.215+0000 7ff19959a640 1 -- 172.21.15.76:0/1166708411 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194036070 con 0x7ff19c15aa10 2024-04-25T19:17:08.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.215+0000 7ff19959a640 1 -- 172.21.15.76:0/1166708411 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff194031600 con 0x7ff19c15aa10 2024-04-25T19:17:08.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.215+0000 7ff19959a640 1 -- 172.21.15.76:0/1166708411 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1940319d0 con 0x7ff19c15aa10 2024-04-25T19:17:08.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.216+0000 7ff1a0e92640 1 -- 172.21.15.76:0/1166708411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c15aa10 msgr2=0x7ff19c15ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:08.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.216+0000 7ff1a0e92640 1 --2- 172.21.15.76:0/1166708411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c15aa10 0x7ff19c15ae20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff194009e50 tx=0x7ff19402a7f0 comp rx=0 tx=0).stop 2024-04-25T19:17:08.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.216+0000 7ff1a0e92640 1 -- 172.21.15.76:0/1166708411 shutdown_connections 2024-04-25T19:17:08.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.216+0000 7ff1a0e92640 1 --2- 172.21.15.76:0/1166708411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c15aa10 0x7ff19c15ae20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:08.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.216+0000 7ff1a0e92640 1 -- 172.21.15.76:0/1166708411 >> 172.21.15.76:0/1166708411 conn(0x7ff19c072b60 msgr2=0x7ff19c072f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:08.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.216+0000 7ff1a0e92640 1 -- 172.21.15.76:0/1166708411 shutdown_connections 2024-04-25T19:17:08.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.216+0000 7ff1a0e92640 1 -- 172.21.15.76:0/1166708411 wait complete. 2024-04-25T19:17:08.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.219+0000 7ff1a0e92640 1 Processor -- start 2024-04-25T19:17:08.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.219+0000 7ff1a0e92640 1 -- start start 2024-04-25T19:17:08.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.221+0000 7ff1a0e92640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff19c0c6040 0x7ff19c071010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:08.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.221+0000 7ff1a0e92640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff19c154c20 0x7ff19c0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:08.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.221+0000 7ff1a0e92640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c15aa10 0x7ff19c1e70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:08.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.221+0000 7ff1a0e92640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c155030 con 0x7ff19c15aa10 2024-04-25T19:17:08.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.221+0000 7ff1a0e92640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c0cfe00 con 0x7ff19c0c6040 2024-04-25T19:17:08.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.221+0000 7ff1a0e92640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff19c15b360 con 0x7ff19c154c20 2024-04-25T19:17:08.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.221+0000 7ff19a59c640 1 -- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff19c0c6040 msgr2=0x7ff19c071010 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3300/0 2024-04-25T19:17:08.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.221+0000 7ff19a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff19c0c6040 0x7ff19c071010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:17:08.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.221+0000 7ff19ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c15aa10 0x7ff19c1e70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:08.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.221+0000 7ff19ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c15aa10 0x7ff19c1e70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33938/0 (socket says 172.21.15.76:33938) 2024-04-25T19:17:08.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.221+0000 7ff19ad9d640 1 -- 172.21.15.76:0/622984165 learned_addr learned my addr 172.21.15.76:0/622984165 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:17:08.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.221+0000 7ff199d9b640 1 --2- 172.21.15.76:0/622984165 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff19c154c20 0x7ff19c0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:08.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.222+0000 7ff19ad9d640 1 -- 172.21.15.76:0/622984165 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff19c154c20 msgr2=0x7ff19c0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:08.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.222+0000 7ff19ad9d640 1 --2- 172.21.15.76:0/622984165 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff19c154c20 0x7ff19c0b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:08.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.222+0000 7ff19ad9d640 1 -- 172.21.15.76:0/622984165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff19c0c6040 msgr2=0x7ff19c071010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:17:08.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.222+0000 7ff19ad9d640 1 --2- 172.21.15.76:0/622984165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff19c0c6040 0x7ff19c071010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:08.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.222+0000 7ff19ad9d640 1 -- 172.21.15.76:0/622984165 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff19c071550 con 0x7ff19c15aa10 2024-04-25T19:17:08.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.222+0000 7ff19ad9d640 1 --2- 172.21.15.76:0/622984165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c15aa10 0x7ff19c1e70a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff194009e50 tx=0x7ff194004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:08.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.222+0000 7ff18affd640 1 -- 172.21.15.76:0/622984165 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19403f070 con 0x7ff19c15aa10 2024-04-25T19:17:08.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.222+0000 7ff18affd640 1 -- 172.21.15.76:0/622984165 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff19403b690 con 0x7ff19c15aa10 2024-04-25T19:17:08.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.222+0000 7ff18affd640 1 -- 172.21.15.76:0/622984165 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194036040 con 0x7ff19c15aa10 2024-04-25T19:17:08.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.222+0000 7ff1a0e92640 1 -- 172.21.15.76:0/622984165 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff19c148040 con 0x7ff19c15aa10 2024-04-25T19:17:08.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.222+0000 7ff1a0e92640 1 -- 172.21.15.76:0/622984165 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff19c148600 con 0x7ff19c15aa10 2024-04-25T19:17:08.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.224+0000 7ff1a0e92640 1 --2- 172.21.15.76:0/622984165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c071720 0x7ff19c148840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:08.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.224+0000 7ff1a0e92640 1 -- 172.21.15.76:0/622984165 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c0b5370 con 0x7ff19c071720 2024-04-25T19:17:08.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.224+0000 7ff19a59c640 1 --2- 172.21.15.76:0/622984165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c071720 0x7ff19c148840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:08.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.224+0000 7ff19a59c640 1 -- 172.21.15.76:0/622984165 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff19c06a390 con 0x7ff19c071720 2024-04-25T19:17:08.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.224+0000 7ff19a59c640 1 --2- 172.21.15.76:0/622984165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c071720 0x7ff19c148840 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0cb9f0 tx=0x7ff18c00c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:08.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.224+0000 7ff18affd640 1 -- 172.21.15.76:0/622984165 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c017070 con 0x7ff19c071720 2024-04-25T19:17:08.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.225+0000 7ff18affd640 1 -- 172.21.15.76:0/622984165 <== mon.0 v2:172.21.15.76:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff18c0074e0 con 0x7ff19c071720 2024-04-25T19:17:08.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.225+0000 7ff18affd640 1 -- 172.21.15.76:0/622984165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c071720 msgr2=0x7ff19c148840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:08.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.225+0000 7ff18affd640 1 --2- 172.21.15.76:0/622984165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c071720 0x7ff19c148840 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0cb9f0 tx=0x7ff18c00c7a0 comp rx=0 tx=0).stop 2024-04-25T19:17:08.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.225+0000 7ff18affd640 1 -- 172.21.15.76:0/622984165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c071720 msgr2=0x7ff19c148840 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:17:08.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.225+0000 7ff18affd640 1 --2- 172.21.15.76:0/622984165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c071720 0x7ff19c148840 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:08.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.225+0000 7ff18affd640 1 -- 172.21.15.76:0/622984165 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff194042070 con 0x7ff19c15aa10 2024-04-25T19:17:08.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.225+0000 7ff18affd640 1 --2- 172.21.15.76:0/622984165 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff17c074070 0x7ff17c076580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:08.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.226+0000 7ff18affd640 1 -- 172.21.15.76:0/622984165 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7ff1940b5aa0 con 0x7ff19c15aa10 2024-04-25T19:17:08.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.226+0000 7ff19a59c640 1 --2- 172.21.15.76:0/622984165 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff17c074070 0x7ff17c076580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:08.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.226+0000 7ff19a59c640 1 --2- 172.21.15.76:0/622984165 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff17c074070 0x7ff17c076580 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff18c001eb0 tx=0x7ff18c00a000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:08.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.231+0000 7ff1a0e92640 1 --2- 172.21.15.76:0/622984165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c149890 0x7ff19c15d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:08.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.232+0000 7ff1a0e92640 1 -- 172.21.15.76:0/622984165 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c1516a0 con 0x7ff19c149890 2024-04-25T19:17:08.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.232+0000 7ff199d9b640 1 --2- 172.21.15.76:0/622984165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c149890 0x7ff19c15d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:08.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.232+0000 7ff199d9b640 1 -- 172.21.15.76:0/622984165 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff19c0c71e0 con 0x7ff19c149890 2024-04-25T19:17:08.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.232+0000 7ff199d9b640 1 --2- 172.21.15.76:0/622984165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c149890 0x7ff19c15d370 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0c6990 tx=0x7ff190009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:08.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.232+0000 7ff18affd640 1 -- 172.21.15.76:0/622984165 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff190016070 con 0x7ff19c149890 2024-04-25T19:17:08.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.233+0000 7ff18affd640 1 -- 172.21.15.76:0/622984165 <== mon.0 v2:172.21.15.76:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5056 (secure 0 0 0) 0x7ff19000e710 con 0x7ff19c149890 2024-04-25T19:17:08.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.233+0000 7ff18affd640 1 -- 172.21.15.76:0/622984165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c149890 msgr2=0x7ff19c15d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:08.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.233+0000 7ff18affd640 1 --2- 172.21.15.76:0/622984165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c149890 0x7ff19c15d370 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0c6990 tx=0x7ff190009e70 comp rx=0 tx=0).stop 2024-04-25T19:17:08.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.233+0000 7ff18affd640 1 -- 172.21.15.76:0/622984165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c149890 msgr2=0x7ff19c15d370 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:17:08.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.233+0000 7ff18affd640 1 --2- 172.21.15.76:0/622984165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c149890 0x7ff19c15d370 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:08.235 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:17:08.236 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:17:08.236 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:17:08.236 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "leader", 2024-04-25T19:17:08.236 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 14, 2024-04-25T19:17:08.236 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 126758, 2024-04-25T19:17:08.236 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:17:08.236 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:17:08.236 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:17:08.237 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:08.237 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 0, 2024-04-25T19:17:08.237 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:17:08.237 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:17:08.237 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:17:08.237 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:17:08.237 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:17:08.237 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:17:08.237 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:17:08.237 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:17:08.237 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:17:08.238 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:17:08.238 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:17:08.238 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:17:08.238 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:17:08.238 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:17:08.238 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:08.238 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:17:08.238 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:17:08.238 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:17:08.238 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:17:08.238 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:17:08.239 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:17:08.239 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:17:08.239 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:17:08.239 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:17:08.239 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:17:08.239 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:17:08.239 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:17:08.239 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:17:08.239 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:08.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:08.240 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:17:08.240 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:17:08.240 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:17:08.240 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:17:08.240 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:17:08.240 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:17:08.240 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:17:08.240 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:17:08.241 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:17:08.241 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:17:08.241 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:17:08.241 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:17:08.241 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:17:08.241 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:17:08.241 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:17:08.241 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:17:08.241 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:17:08.241 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:17:08.241 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:17:08.241 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:17:08.242 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:17:08.242 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:17:08.242 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:17:08.242 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:17:08.242 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:17:08.242 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:17:08.242 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:17:08.242 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:17:08.242 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:08.242 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:17:08.243 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:08.243 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:17:08.243 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:08.243 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:17:08.243 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:17:08.243 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:17:08.243 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:17:08.243 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:08.243 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:17:08.243 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:17:08.243 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:08.244 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:08.244 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:08.244 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:17:08.244 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:17:08.244 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:08.244 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:08.244 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:08.244 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:08.244 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:17:08.244 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:17:08.244 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:17:08.244 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:17:08.244 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:17:08.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:08.245 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:08.245 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:17:08.245 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:17:08.245 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:17:08.245 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:17:08.245 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:08.245 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:17:08.245 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:17:08.245 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:08.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:08.245 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:08.246 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:17:08.246 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:17:08.246 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:08.246 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:08.246 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:08.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:08.246 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:17:08.246 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:17:08.246 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:17:08.246 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:17:08.246 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:17:08.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:08.247 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:08.247 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:17:08.247 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:17:08.247 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:17:08.247 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:17:08.247 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:08.247 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:17:08.247 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:17:08.247 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:08.247 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:08.247 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:08.247 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:17:08.248 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:17:08.248 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:08.248 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:08.248 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:08.248 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:08.248 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:17:08.248 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:17:08.248 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:17:08.248 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:17:08.248 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:17:08.248 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:08.248 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:08.248 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:08.249 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:17:08.249 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:17:08.249 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:08.249 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:08.249 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:08.249 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:17:08.249 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:08.249 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:08.249 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": [ 2024-04-25T19:17:08.249 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:08.249 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:08.250 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:08.250 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 6 2024-04-25T19:17:08.250 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:08.250 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:08.250 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": [ 2024-04-25T19:17:08.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:08.250 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:08.250 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:08.250 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 7 2024-04-25T19:17:08.250 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:08.250 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:08.250 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:17:08.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:08.251 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:08.251 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:08.251 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 6 2024-04-25T19:17:08.251 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:08.251 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:08.251 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": [ 2024-04-25T19:17:08.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:08.251 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:08.251 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:08.251 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 2 2024-04-25T19:17:08.251 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:08.253 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:08.253 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:08.253 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:17:08.253 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:17:08.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.233+0000 7ff1a0e92640 1 -- 172.21.15.76:0/622984165 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff17c074070 msgr2=0x7ff17c076580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:08.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.233+0000 7ff1a0e92640 1 --2- 172.21.15.76:0/622984165 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff17c074070 0x7ff17c076580 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff18c001eb0 tx=0x7ff18c00a000 comp rx=0 tx=0).stop 2024-04-25T19:17:08.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.233+0000 7ff1a0e92640 1 -- 172.21.15.76:0/622984165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c15aa10 msgr2=0x7ff19c1e70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:08.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.233+0000 7ff1a0e92640 1 --2- 172.21.15.76:0/622984165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c15aa10 0x7ff19c1e70a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff194009e50 tx=0x7ff194004a40 comp rx=0 tx=0).stop 2024-04-25T19:17:08.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.233+0000 7ff19ad9d640 1 -- 172.21.15.76:0/622984165 reap_dead start 2024-04-25T19:17:08.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.233+0000 7ff1a0e92640 1 -- 172.21.15.76:0/622984165 shutdown_connections 2024-04-25T19:17:08.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.233+0000 7ff1a0e92640 1 --2- 172.21.15.76:0/622984165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c15aa10 0x7ff19c1e70a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:08.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.233+0000 7ff1a0e92640 1 -- 172.21.15.76:0/622984165 >> 172.21.15.76:0/622984165 conn(0x7ff19c072b60 msgr2=0x7ff19c0ce650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:08.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.233+0000 7ff1a0e92640 1 -- 172.21.15.76:0/622984165 shutdown_connections 2024-04-25T19:17:08.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.233+0000 7ff1a0e92640 1 -- 172.21.15.76:0/622984165 wait complete. 2024-04-25T19:17:08.256 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T19:17:08.256 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-04-25T19:17:08.257 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T19:17:08.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.370+0000 7fb78338f640 1 Processor -- start 2024-04-25T19:17:08.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.370+0000 7fb78338f640 1 -- start start 2024-04-25T19:17:08.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.371+0000 7fb78338f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb77c154c20 0x7fb77c0cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:08.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.371+0000 7fb78338f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb77c0cb820 con 0x7fb77c0c6040 2024-04-25T19:17:08.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.371+0000 7fb78338f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb77c0cbac0 con 0x7fb77c0cd560 2024-04-25T19:17:08.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.371+0000 7fb78338f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb77c0cc730 con 0x7fb77c154c20 2024-04-25T19:17:08.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.371+0000 7fb78192c640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7fb77c0cd560 legacy=0x7fb77c15aa10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-04-25T19:17:08.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.371+0000 7fb78092a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb77c154c20 0x7fb77c0cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:08.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.371+0000 7fb78092a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb77c154c20 0x7fb77c0cd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33958/0 (socket says 172.21.15.76:33958) 2024-04-25T19:17:08.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.371+0000 7fb78112b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb77c0c6040 0x7fb77c0cab50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33906/0 (socket says 172.21.15.76:33906) 2024-04-25T19:17:08.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.371+0000 7fb78092a640 1 -- 172.21.15.76:0/1957859122 learned_addr learned my addr 172.21.15.76:0/1957859122 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:17:08.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.372+0000 7fb78092a640 1 -- 172.21.15.76:0/1957859122 >> v1:172.21.15.105:6789/0 conn(0x7fb77c0cd560 legacy=0x7fb77c15aa10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:17:08.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.372+0000 7fb78092a640 1 -- 172.21.15.76:0/1957859122 >> v1:172.21.15.76:6789/0 conn(0x7fb77c0c6040 legacy=0x7fb77c0cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:08.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.372+0000 7fb773fff640 1 -- 172.21.15.76:0/1957859122 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4178264197 0 0) 0x7fb77c0cb820 con 0x7fb77c0c6040 2024-04-25T19:17:08.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.372+0000 7fb78092a640 1 -- 172.21.15.76:0/1957859122 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77c0cbda0 con 0x7fb77c154c20 2024-04-25T19:17:08.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.373+0000 7fb78092a640 1 --2- 172.21.15.76:0/1957859122 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb77c154c20 0x7fb77c0cd020 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009870 tx=0x7fb76c02a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582a670daeff5515 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:08.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.373+0000 7fb773fff640 1 -- 172.21.15.76:0/1957859122 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c037070 con 0x7fb77c154c20 2024-04-25T19:17:08.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.373+0000 7fb773fff640 1 -- 172.21.15.76:0/1957859122 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb76c032640 con 0x7fb77c154c20 2024-04-25T19:17:08.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.373+0000 7fb773fff640 1 -- 172.21.15.76:0/1957859122 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c032a10 con 0x7fb77c154c20 2024-04-25T19:17:08.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.374+0000 7fb78338f640 1 -- 172.21.15.76:0/1957859122 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb77c154c20 msgr2=0x7fb77c0cd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:08.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.374+0000 7fb78338f640 1 --2- 172.21.15.76:0/1957859122 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb77c154c20 0x7fb77c0cd020 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009870 tx=0x7fb76c02a330 comp rx=0 tx=0).stop 2024-04-25T19:17:08.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.374+0000 7fb78338f640 1 -- 172.21.15.76:0/1957859122 shutdown_connections 2024-04-25T19:17:08.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.374+0000 7fb78338f640 1 --2- 172.21.15.76:0/1957859122 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb77c154c20 0x7fb77c0cd020 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:08.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.374+0000 7fb78338f640 1 -- 172.21.15.76:0/1957859122 >> 172.21.15.76:0/1957859122 conn(0x7fb77c072b60 msgr2=0x7fb77c072f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:08.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.374+0000 7fb78338f640 1 -- 172.21.15.76:0/1957859122 shutdown_connections 2024-04-25T19:17:08.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.374+0000 7fb78338f640 1 -- 172.21.15.76:0/1957859122 wait complete. 2024-04-25T19:17:08.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.376+0000 7fb78338f640 1 Processor -- start 2024-04-25T19:17:08.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.376+0000 7fb78338f640 1 -- start start 2024-04-25T19:17:08.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.377+0000 7fb78338f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb77c0c6040 0x7fb77c1f4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:08.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.377+0000 7fb78338f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb77c0cd560 0x7fb77c070790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:08.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.377+0000 7fb78338f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb77c154c20 0x7fb77c14acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:08.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.377+0000 7fb78338f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c155030 con 0x7fb77c154c20 2024-04-25T19:17:08.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.377+0000 7fb78338f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c0cfe00 con 0x7fb77c0cd560 2024-04-25T19:17:08.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.377+0000 7fb78338f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb77c15b120 con 0x7fb77c0c6040 2024-04-25T19:17:08.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.377+0000 7fb78092a640 1 -- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb77c0cd560 msgr2=0x7fb77c070790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3300/0 2024-04-25T19:17:08.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.377+0000 7fb78092a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb77c0cd560 0x7fb77c070790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:17:08.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.377+0000 7fb78112b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb77c0c6040 0x7fb77c1f4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:08.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.377+0000 7fb78112b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb77c0c6040 0x7fb77c1f4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:37674/0 (socket says 172.21.15.76:37674) 2024-04-25T19:17:08.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.377+0000 7fb78112b640 1 -- 172.21.15.76:0/242279872 learned_addr learned my addr 172.21.15.76:0/242279872 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:17:08.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.377+0000 7fb78192c640 1 --2- 172.21.15.76:0/242279872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb77c154c20 0x7fb77c14acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:08.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.378+0000 7fb78112b640 1 -- 172.21.15.76:0/242279872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb77c0cd560 msgr2=0x7fb77c070790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:17:08.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.378+0000 7fb78112b640 1 --2- 172.21.15.76:0/242279872 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb77c0cd560 0x7fb77c070790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:08.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.378+0000 7fb78112b640 1 -- 172.21.15.76:0/242279872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb77c154c20 msgr2=0x7fb77c14acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:08.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.378+0000 7fb78112b640 1 --2- 172.21.15.76:0/242279872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb77c154c20 0x7fb77c14acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:08.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.378+0000 7fb78112b640 1 -- 172.21.15.76:0/242279872 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77c1f5070 con 0x7fb77c0c6040 2024-04-25T19:17:08.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.378+0000 7fb78192c640 1 --2- 172.21.15.76:0/242279872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb77c154c20 0x7fb77c14acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:17:08.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.378+0000 7fb78112b640 1 --2- 172.21.15.76:0/242279872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb77c0c6040 0x7fb77c1f4b30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb77800c440 tx=0x7fb77800c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:08.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.378+0000 7fb771ffb640 1 -- 172.21.15.76:0/242279872 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778018070 con 0x7fb77c0c6040 2024-04-25T19:17:08.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.378+0000 7fb771ffb640 1 -- 172.21.15.76:0/242279872 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb778007480 con 0x7fb77c0c6040 2024-04-25T19:17:08.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.378+0000 7fb771ffb640 1 -- 172.21.15.76:0/242279872 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7780047e0 con 0x7fb77c0c6040 2024-04-25T19:17:08.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.378+0000 7fb78338f640 1 -- 172.21.15.76:0/242279872 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77c1f4370 con 0x7fb77c0c6040 2024-04-25T19:17:08.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.379+0000 7fb78338f640 1 -- 172.21.15.76:0/242279872 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb77c071600 con 0x7fb77c0c6040 2024-04-25T19:17:08.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.380+0000 7fb75f7fe640 1 --2- 172.21.15.76:0/242279872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb758002a70 0x7fb758004f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:08.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.380+0000 7fb75f7fe640 1 -- 172.21.15.76:0/242279872 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb758007f10 con 0x7fb758002a70 2024-04-25T19:17:08.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.380+0000 7fb78092a640 1 --2- 172.21.15.76:0/242279872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb758002a70 0x7fb758004f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:08.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.381+0000 7fb78092a640 1 -- 172.21.15.76:0/242279872 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb758008110 con 0x7fb758002a70 2024-04-25T19:17:08.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.381+0000 7fb771ffb640 1 -- 172.21.15.76:0/242279872 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb778004980 con 0x7fb77c0c6040 2024-04-25T19:17:08.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.381+0000 7fb78092a640 1 --2- 172.21.15.76:0/242279872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb758002a70 0x7fb758004f80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb77c0b2de0 tx=0x7fb76c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:08.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.381+0000 7fb771ffb640 1 --2- 172.21.15.76:0/242279872 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb760074490 0x7fb7600769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:08.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.381+0000 7fb771ffb640 1 -- 172.21.15.76:0/242279872 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fb7780943a0 con 0x7fb77c0c6040 2024-04-25T19:17:08.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.382+0000 7fb78192c640 1 --2- 172.21.15.76:0/242279872 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb760074490 0x7fb7600769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:08.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.382+0000 7fb771ffb640 1 -- 172.21.15.76:0/242279872 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c042030 con 0x7fb758002a70 2024-04-25T19:17:08.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.382+0000 7fb771ffb640 1 -- 172.21.15.76:0/242279872 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb76c02ae40 con 0x7fb758002a70 2024-04-25T19:17:08.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.382+0000 7fb771ffb640 1 -- 172.21.15.76:0/242279872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb758002a70 msgr2=0x7fb758004f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:08.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.382+0000 7fb771ffb640 1 --2- 172.21.15.76:0/242279872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb758002a70 0x7fb758004f80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb77c0b2de0 tx=0x7fb76c0046c0 comp rx=0 tx=0).stop 2024-04-25T19:17:08.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.382+0000 7fb78192c640 1 --2- 172.21.15.76:0/242279872 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb760074490 0x7fb7600769a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb77c15d5d0 tx=0x7fb77400d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:08.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.385+0000 7fb771ffb640 1 -- 172.21.15.76:0/242279872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb758002a70 msgr2=0x7fb758004f80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:17:08.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.385+0000 7fb771ffb640 1 --2- 172.21.15.76:0/242279872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb758002a70 0x7fb758004f80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:08.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.388+0000 7fb75f7fe640 1 --2- 172.21.15.76:0/242279872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb758006f90 0x7fb758007470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:08.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.388+0000 7fb75f7fe640 1 -- 172.21.15.76:0/242279872 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb75800a4f0 con 0x7fb758006f90 2024-04-25T19:17:08.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.389+0000 7fb78092a640 1 --2- 172.21.15.76:0/242279872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb758006f90 0x7fb758007470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:08.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.389+0000 7fb78092a640 1 -- 172.21.15.76:0/242279872 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb7580079b0 con 0x7fb758006f90 2024-04-25T19:17:08.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.389+0000 7fb78092a640 1 --2- 172.21.15.76:0/242279872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb758006f90 0x7fb758007470 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb76c000c00 tx=0x7fb76c009240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:08.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.390+0000 7fb771ffb640 1 -- 172.21.15.76:0/242279872 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c042070 con 0x7fb758006f90 2024-04-25T19:17:08.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.390+0000 7fb771ffb640 1 -- 172.21.15.76:0/242279872 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4713 (secure 0 0 0) 0x7fb76c003a20 con 0x7fb758006f90 2024-04-25T19:17:08.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.390+0000 7fb771ffb640 1 -- 172.21.15.76:0/242279872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb758006f90 msgr2=0x7fb758007470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:08.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.390+0000 7fb771ffb640 1 --2- 172.21.15.76:0/242279872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb758006f90 0x7fb758007470 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb76c000c00 tx=0x7fb76c009240 comp rx=0 tx=0).stop 2024-04-25T19:17:08.392 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:17:08.392 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:17:08.392 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:17:08.392 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "peon", 2024-04-25T19:17:08.392 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 14, 2024-04-25T19:17:08.392 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 32167, 2024-04-25T19:17:08.392 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:17:08.392 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:17:08.392 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:17:08.393 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:08.393 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 0, 2024-04-25T19:17:08.393 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:17:08.393 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:17:08.393 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:17:08.393 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:17:08.393 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:17:08.393 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:17:08.393 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:17:08.393 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:17:08.393 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:17:08.393 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:17:08.393 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:17:08.393 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:17:08.393 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:17:08.393 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:17:08.394 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:08.394 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:17:08.394 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:17:08.394 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:17:08.394 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:17:08.394 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:17:08.394 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:17:08.394 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:17:08.394 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:17:08.394 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:17:08.394 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:17:08.394 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:17:08.395 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:17:08.395 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:17:08.395 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:08.395 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:08.395 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:17:08.395 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:17:08.395 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:17:08.395 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:17:08.395 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:17:08.395 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:17:08.395 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:17:08.395 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:17:08.395 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:17:08.396 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:17:08.396 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:17:08.396 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:17:08.396 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:17:08.396 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:17:08.396 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:17:08.396 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:17:08.396 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:17:08.396 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:17:08.396 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:17:08.396 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:17:08.396 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:17:08.396 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:17:08.396 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:17:08.396 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:17:08.396 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:17:08.397 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:17:08.397 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:17:08.397 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:17:08.397 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:08.397 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:17:08.397 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:08.397 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:17:08.397 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:08.397 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:17:08.397 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:17:08.397 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:17:08.397 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:17:08.397 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:08.397 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:17:08.397 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:17:08.398 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:08.398 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:08.398 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:08.398 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:17:08.398 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:17:08.398 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:08.398 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:08.398 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:08.398 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:08.398 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:17:08.398 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:17:08.398 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:17:08.398 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:17:08.398 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:17:08.399 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:08.399 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:08.399 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:17:08.399 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:17:08.399 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:17:08.399 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:17:08.399 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:08.399 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:17:08.399 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:17:08.399 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:08.399 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:08.399 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:08.399 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:17:08.399 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:17:08.399 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:08.399 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:08.400 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:08.400 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:08.400 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:17:08.400 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:17:08.400 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:17:08.400 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:17:08.400 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:17:08.400 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:08.400 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:08.400 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:17:08.400 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:17:08.400 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:17:08.400 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:17:08.400 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:08.400 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:17:08.401 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:17:08.401 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:08.401 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:08.401 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:08.401 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:17:08.401 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:17:08.401 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:08.401 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:08.401 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:08.401 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:08.401 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:17:08.401 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:17:08.401 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:17:08.401 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:17:08.401 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:17:08.401 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:08.402 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:08.402 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:08.402 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:17:08.402 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:17:08.402 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:08.402 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:08.402 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:08.402 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:17:08.402 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:08.402 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:08.402 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": [ 2024-04-25T19:17:08.402 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:08.403 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:08.403 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:08.403 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:17:08.403 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:08.403 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:08.403 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:17:08.403 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:08.403 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:08.403 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:08.403 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 3 2024-04-25T19:17:08.403 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:08.403 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:08.403 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:08.404 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:17:08.404 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:17:08.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.390+0000 7fb771ffb640 1 -- 172.21.15.76:0/242279872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb758006f90 msgr2=0x7fb758007470 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:17:08.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.390+0000 7fb771ffb640 1 --2- 172.21.15.76:0/242279872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb758006f90 0x7fb758007470 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb76c000c00 tx=0x7fb76c009240 comp rx=0 tx=0).stop 2024-04-25T19:17:08.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.390+0000 7fb75f7fe640 1 -- 172.21.15.76:0/242279872 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb760074490 msgr2=0x7fb7600769a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:08.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.390+0000 7fb75f7fe640 1 --2- 172.21.15.76:0/242279872 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb760074490 0x7fb7600769a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb77c15d5d0 tx=0x7fb77400d040 comp rx=0 tx=0).stop 2024-04-25T19:17:08.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.390+0000 7fb75f7fe640 1 -- 172.21.15.76:0/242279872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb77c0c6040 msgr2=0x7fb77c1f4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:08.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.390+0000 7fb75f7fe640 1 --2- 172.21.15.76:0/242279872 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb77c0c6040 0x7fb77c1f4b30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb77800c440 tx=0x7fb77800c900 comp rx=0 tx=0).stop 2024-04-25T19:17:08.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.390+0000 7fb78192c640 1 -- 172.21.15.76:0/242279872 reap_dead start 2024-04-25T19:17:08.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.390+0000 7fb78192c640 1 -- 172.21.15.76:0/242279872 reap_dead start 2024-04-25T19:17:08.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.390+0000 7fb75f7fe640 1 -- 172.21.15.76:0/242279872 shutdown_connections 2024-04-25T19:17:08.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.390+0000 7fb75f7fe640 1 -- 172.21.15.76:0/242279872 >> 172.21.15.76:0/242279872 conn(0x7fb77c072b60 msgr2=0x7fb77c075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:08.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.391+0000 7fb75f7fe640 1 -- 172.21.15.76:0/242279872 shutdown_connections 2024-04-25T19:17:08.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:08.391+0000 7fb75f7fe640 1 -- 172.21.15.76:0/242279872 wait complete. 2024-04-25T19:17:08.406 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T19:17:08.406 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-04-25T19:17:25.551 INFO:tasks.workunit.client.0.smithi076.stderr:Note: switching to '994e2f20102a20fa9a27966c14b625cc01884df8'. 2024-04-25T19:17:25.551 INFO:tasks.workunit.client.0.smithi076.stderr: 2024-04-25T19:17:25.551 INFO:tasks.workunit.client.0.smithi076.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T19:17:25.551 INFO:tasks.workunit.client.0.smithi076.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T19:17:25.551 INFO:tasks.workunit.client.0.smithi076.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T19:17:25.552 INFO:tasks.workunit.client.0.smithi076.stderr: 2024-04-25T19:17:25.552 INFO:tasks.workunit.client.0.smithi076.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T19:17:25.552 INFO:tasks.workunit.client.0.smithi076.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T19:17:25.552 INFO:tasks.workunit.client.0.smithi076.stderr: 2024-04-25T19:17:25.552 INFO:tasks.workunit.client.0.smithi076.stderr: git switch -c 2024-04-25T19:17:25.552 INFO:tasks.workunit.client.0.smithi076.stderr: 2024-04-25T19:17:25.552 INFO:tasks.workunit.client.0.smithi076.stderr:Or undo this operation with: 2024-04-25T19:17:25.552 INFO:tasks.workunit.client.0.smithi076.stderr: 2024-04-25T19:17:25.552 INFO:tasks.workunit.client.0.smithi076.stderr: git switch - 2024-04-25T19:17:25.552 INFO:tasks.workunit.client.0.smithi076.stderr: 2024-04-25T19:17:25.552 INFO:tasks.workunit.client.0.smithi076.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T19:17:25.552 INFO:tasks.workunit.client.0.smithi076.stderr: 2024-04-25T19:17:25.553 INFO:tasks.workunit.client.0.smithi076.stderr:HEAD is now at 994e2f20102 Merge PR #56810 into wip-pdonnell-testing-20240425.015853-debug 2024-04-25T19:17:25.557 DEBUG:teuthology.orchestra.run.smithi076:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T19:17:25.590 INFO:tasks.workunit.client.0.smithi076.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T19:17:25.592 INFO:tasks.workunit.client.0.smithi076.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T19:17:25.592 INFO:tasks.workunit.client.0.smithi076.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T19:17:25.955 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:17:25.958 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:17:26.243 INFO:tasks.workunit.client.0.smithi076.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T19:17:26.300 INFO:tasks.workunit.client.0.smithi076.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T19:17:26.340 INFO:tasks.workunit.client.0.smithi076.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T19:17:26.341 INFO:tasks.workunit.client.0.smithi076.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T19:17:26.341 INFO:tasks.workunit.client.0.smithi076.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T19:17:26.381 INFO:tasks.workunit.client.0.smithi076.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T19:17:26.384 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:17:26.384 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T19:17:26.399 INFO:tasks.workunit:Running workunits matching suites/iozone.sh on client.0... 2024-04-25T19:17:26.400 INFO:tasks.workunit:Running workunit suites/iozone.sh... 2024-04-25T19:17:26.400 DEBUG:teuthology.orchestra.run.smithi076:workunit test suites/iozone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=994e2f20102a20fa9a27966c14b625cc01884df8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh 2024-04-25T19:17:26.465 INFO:tasks.workunit.client.0.smithi076.stderr:+ iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-04-25T19:17:26.551 INFO:tasks.workunit.client.0.smithi076.stdout: Iozone: Performance Test of File I/O 2024-04-25T19:17:26.552 INFO:tasks.workunit.client.0.smithi076.stdout: Version $Revision: 3.494 $ 2024-04-25T19:17:26.552 INFO:tasks.workunit.client.0.smithi076.stdout: Compiled for 64 bit mode. 2024-04-25T19:17:26.552 INFO:tasks.workunit.client.0.smithi076.stdout: Build: linux-AMD64 2024-04-25T19:17:26.552 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:17:26.552 INFO:tasks.workunit.client.0.smithi076.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-04-25T19:17:26.552 INFO:tasks.workunit.client.0.smithi076.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-04-25T19:17:26.552 INFO:tasks.workunit.client.0.smithi076.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-04-25T19:17:26.552 INFO:tasks.workunit.client.0.smithi076.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-04-25T19:17:26.552 INFO:tasks.workunit.client.0.smithi076.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-04-25T19:17:26.552 INFO:tasks.workunit.client.0.smithi076.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-04-25T19:17:26.552 INFO:tasks.workunit.client.0.smithi076.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-04-25T19:17:26.552 INFO:tasks.workunit.client.0.smithi076.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-04-25T19:17:26.552 INFO:tasks.workunit.client.0.smithi076.stdout: Alexey Skidanov, Sudhir Kumar. 2024-04-25T19:17:26.552 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:17:26.552 INFO:tasks.workunit.client.0.smithi076.stdout: Run began: Thu Apr 25 19:17:26 2024 2024-04-25T19:17:26.552 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:17:26.553 INFO:tasks.workunit.client.0.smithi076.stdout: Include close in write timing 2024-04-25T19:17:26.553 INFO:tasks.workunit.client.0.smithi076.stdout: Include fsync in write timing 2024-04-25T19:17:26.553 INFO:tasks.workunit.client.0.smithi076.stdout: File size set to 1048576 kB 2024-04-25T19:17:26.553 INFO:tasks.workunit.client.0.smithi076.stdout: Record Size 16 kB 2024-04-25T19:17:26.553 INFO:tasks.workunit.client.0.smithi076.stdout: Command line used: iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-04-25T19:17:26.553 INFO:tasks.workunit.client.0.smithi076.stdout: Output is in kBytes/sec 2024-04-25T19:17:26.553 INFO:tasks.workunit.client.0.smithi076.stdout: Time Resolution = 0.000001 seconds. 2024-04-25T19:17:26.553 INFO:tasks.workunit.client.0.smithi076.stdout: Processor cache size set to 1024 kBytes. 2024-04-25T19:17:26.553 INFO:tasks.workunit.client.0.smithi076.stdout: Processor cache line size set to 32 bytes. 2024-04-25T19:17:26.553 INFO:tasks.workunit.client.0.smithi076.stdout: File stride size set to 17 * record size. 2024-04-25T19:17:26.553 INFO:tasks.workunit.client.0.smithi076.stdout: Throughput test with 1 process 2024-04-25T19:17:26.553 INFO:tasks.workunit.client.0.smithi076.stdout: Each process writes a 1048576 kByte file in 16 kByte records 2024-04-25T19:17:28.407 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-04-25T19:17:28.407 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-04-25T19:17:28.407 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T19:17:28.407 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T19:17:28.410 INFO:tasks.ceph.mon.b:Started 2024-04-25T19:17:28.410 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T19:17:28.410 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:17:28.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.545+0000 7f5752020640 1 Processor -- start 2024-04-25T19:17:28.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.546+0000 7f5752020640 1 -- start start 2024-04-25T19:17:28.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.547+0000 7f5752020640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f574c0d0810 0x7f574c06f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:28.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.547+0000 7f5752020640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f574c0cea70 0x7f574c0cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:28.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.547+0000 7f5752020640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f574c06e160 0x7f574c06e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:28.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.547+0000 7f5752020640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f574c15ce80 con 0x7f574c0d0810 2024-04-25T19:17:28.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.547+0000 7f5752020640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f574c15d160 con 0x7f574c0cea70 2024-04-25T19:17:28.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.547+0000 7f5752020640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f574c15d440 con 0x7f574c06e160 2024-04-25T19:17:28.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.547+0000 7f574b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f574c0d0810 0x7f574c06f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:28.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.548+0000 7f574b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f574c0d0810 0x7f574c06f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52888/0 (socket says 172.21.15.76:52888) 2024-04-25T19:17:28.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.548+0000 7f574b7fe640 1 -- 172.21.15.76:0/3410090639 learned_addr learned my addr 172.21.15.76:0/3410090639 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:17:28.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.548+0000 7f574bfff640 1 --2- 172.21.15.76:0/3410090639 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f574c06e160 0x7f574c06e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:28.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.548+0000 7f574affd640 1 --2- 172.21.15.76:0/3410090639 >> v2:172.21.15.105:3300/0 conn(0x7f574c0cea70 0x7f574c0cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:28.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.548+0000 7f574affd640 1 -- 172.21.15.76:0/3410090639 >> v2:172.21.15.105:3300/0 conn(0x7f574c0cea70 msgr2=0x7f574c0cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T19:17:28.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.548+0000 7f574affd640 1 -- 172.21.15.76:0/3410090639 >> v2:172.21.15.105:3300/0 conn(0x7f574c0cea70 msgr2=0x7f574c0cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:17:28.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.548+0000 7f574affd640 1 --2- 172.21.15.76:0/3410090639 >> v2:172.21.15.105:3300/0 conn(0x7f574c0cea70 0x7f574c0cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:28.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.548+0000 7f574affd640 1 --2- 172.21.15.76:0/3410090639 >> v2:172.21.15.105:3300/0 conn(0x7f574c0cea70 0x7f574c0cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:17:28.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.548+0000 7f574b7fe640 1 -- 172.21.15.76:0/3410090639 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f574c06e160 msgr2=0x7f574c06e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:28.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.548+0000 7f574b7fe640 1 --2- 172.21.15.76:0/3410090639 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f574c06e160 0x7f574c06e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:28.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.548+0000 7f574b7fe640 1 -- 172.21.15.76:0/3410090639 >> v2:172.21.15.105:3300/0 conn(0x7f574c0cea70 msgr2=0x7f574c0cee80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:17:28.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.548+0000 7f574b7fe640 1 --2- 172.21.15.76:0/3410090639 >> v2:172.21.15.105:3300/0 conn(0x7f574c0cea70 0x7f574c0cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:28.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.548+0000 7f574b7fe640 1 -- 172.21.15.76:0/3410090639 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574c15d720 con 0x7f574c0d0810 2024-04-25T19:17:28.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.549+0000 7f574b7fe640 1 --2- 172.21.15.76:0/3410090639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f574c0d0810 0x7f574c06f000 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f574c08f710 tx=0x7f573c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f298d2e65435536 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:28.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.549+0000 7f574a7fc640 1 -- 172.21.15.76:0/3410090639 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c037070 con 0x7f574c0d0810 2024-04-25T19:17:28.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.549+0000 7f574a7fc640 1 -- 172.21.15.76:0/3410090639 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f573c032440 con 0x7f574c0d0810 2024-04-25T19:17:28.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.549+0000 7f574a7fc640 1 -- 172.21.15.76:0/3410090639 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c032810 con 0x7f574c0d0810 2024-04-25T19:17:28.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.550+0000 7f5752020640 1 -- 172.21.15.76:0/3410090639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f574c0d0810 msgr2=0x7f574c06f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:28.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.550+0000 7f5752020640 1 --2- 172.21.15.76:0/3410090639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f574c0d0810 0x7f574c06f000 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f574c08f710 tx=0x7f573c02a1e0 comp rx=0 tx=0).stop 2024-04-25T19:17:28.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.550+0000 7f5752020640 1 -- 172.21.15.76:0/3410090639 shutdown_connections 2024-04-25T19:17:28.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.550+0000 7f5752020640 1 --2- 172.21.15.76:0/3410090639 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f574c06e160 0x7f574c06e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:28.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.550+0000 7f5752020640 1 --2- 172.21.15.76:0/3410090639 >> v2:172.21.15.105:3300/0 conn(0x7f574c0cea70 0x7f574c0cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:28.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.550+0000 7f5752020640 1 --2- 172.21.15.76:0/3410090639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f574c0d0810 0x7f574c06f000 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:28.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.550+0000 7f5752020640 1 -- 172.21.15.76:0/3410090639 >> 172.21.15.76:0/3410090639 conn(0x7f574c0b3c00 msgr2=0x7f574c0b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:28.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.550+0000 7f5752020640 1 -- 172.21.15.76:0/3410090639 shutdown_connections 2024-04-25T19:17:28.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.550+0000 7f5752020640 1 -- 172.21.15.76:0/3410090639 wait complete. 2024-04-25T19:17:28.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.552+0000 7f5752020640 1 Processor -- start 2024-04-25T19:17:28.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.552+0000 7f5752020640 1 -- start start 2024-04-25T19:17:28.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.554+0000 7f5752020640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f574c06e160 0x7f574c1e70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:28.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.554+0000 7f5752020640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f574c0cea70 0x7f574c15df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:28.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.554+0000 7f5752020640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f574c0d0810 0x7f574c150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:28.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.554+0000 7f5752020640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f574c06eab0 con 0x7f574c0d0810 2024-04-25T19:17:28.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.554+0000 7f574bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f574c0d0810 0x7f574c150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:28.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.554+0000 7f574b7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f574c06e160 0x7f574c1e70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:28.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.554+0000 7f5752020640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f574c06dc40 con 0x7f574c0cea70 2024-04-25T19:17:28.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.554+0000 7f574b7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f574c06e160 0x7f574c1e70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:43776/0 (socket says 172.21.15.76:43776) 2024-04-25T19:17:28.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.554+0000 7f574bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f574c0d0810 0x7f574c150350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52898/0 (socket says 172.21.15.76:52898) 2024-04-25T19:17:28.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.554+0000 7f574b7fe640 1 -- 172.21.15.76:0/2146612575 learned_addr learned my addr 172.21.15.76:0/2146612575 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:17:28.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.554+0000 7f5752020640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f574c15cb70 con 0x7f574c06e160 2024-04-25T19:17:28.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.554+0000 7f574affd640 1 --2- 172.21.15.76:0/2146612575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f574c0cea70 0x7f574c15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:28.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.554+0000 7f574affd640 1 -- 172.21.15.76:0/2146612575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f574c0cea70 msgr2=0x7f574c15df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T19:17:28.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.554+0000 7f574affd640 1 -- 172.21.15.76:0/2146612575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f574c0cea70 msgr2=0x7f574c15df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T19:17:28.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.554+0000 7f574affd640 1 --2- 172.21.15.76:0/2146612575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f574c0cea70 0x7f574c15df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:28.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.554+0000 7f574affd640 1 --2- 172.21.15.76:0/2146612575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f574c0cea70 0x7f574c15df10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:17:28.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.554+0000 7f574b7fe640 1 -- 172.21.15.76:0/2146612575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f574c0cea70 msgr2=0x7f574c15df10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:17:28.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.554+0000 7f574b7fe640 1 --2- 172.21.15.76:0/2146612575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f574c0cea70 0x7f574c15df10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:28.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.554+0000 7f574b7fe640 1 -- 172.21.15.76:0/2146612575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f574c0d0810 msgr2=0x7f574c150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:28.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.554+0000 7f574b7fe640 1 --2- 172.21.15.76:0/2146612575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f574c0d0810 0x7f574c150350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:28.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.554+0000 7f574b7fe640 1 -- 172.21.15.76:0/2146612575 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574c1e75f0 con 0x7f574c06e160 2024-04-25T19:17:28.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.555+0000 7f574b7fe640 1 --2- 172.21.15.76:0/2146612575 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f574c06e160 0x7f574c1e70b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f573c02a220 tx=0x7f573c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:28.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.555+0000 7f5733fff640 1 -- 172.21.15.76:0/2146612575 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c037040 con 0x7f574c06e160 2024-04-25T19:17:28.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.555+0000 7f5733fff640 1 -- 172.21.15.76:0/2146612575 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f573c042070 con 0x7f574c06e160 2024-04-25T19:17:28.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.555+0000 7f5733fff640 1 -- 172.21.15.76:0/2146612575 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c030880 con 0x7f574c06e160 2024-04-25T19:17:28.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.555+0000 7f5752020640 1 -- 172.21.15.76:0/2146612575 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f574c1f2930 con 0x7f574c06e160 2024-04-25T19:17:28.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.555+0000 7f5752020640 1 -- 172.21.15.76:0/2146612575 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f574c1f2eb0 con 0x7f574c06e160 2024-04-25T19:17:28.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.556+0000 7f5752020640 1 -- 172.21.15.76:0/2146612575 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f574c06a390 con 0x7f574c06e160 2024-04-25T19:17:28.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.557+0000 7f5733fff640 1 -- 172.21.15.76:0/2146612575 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f573c030a20 con 0x7f574c06e160 2024-04-25T19:17:28.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.557+0000 7f5733fff640 1 --2- 172.21.15.76:0/2146612575 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f57380744b0 0x7f57380769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:28.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.557+0000 7f5733fff640 1 -- 172.21.15.76:0/2146612575 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f573c0b5b50 con 0x7f574c06e160 2024-04-25T19:17:28.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.557+0000 7f574affd640 1 --2- 172.21.15.76:0/2146612575 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f57380744b0 0x7f57380769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:28.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.558+0000 7f574affd640 1 --2- 172.21.15.76:0/2146612575 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f57380744b0 0x7f57380769c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f574c15c8b0 tx=0x7f5740008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:28.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.562+0000 7f5733fff640 1 -- 172.21.15.76:0/2146612575 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f573c080260 con 0x7f574c06e160 2024-04-25T19:17:28.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.864+0000 7f5752020640 1 -- 172.21.15.76:0/2146612575 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f574c0d01a0 con 0x7f574c06e160 2024-04-25T19:17:28.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.864+0000 7f5733fff640 1 -- 172.21.15.76:0/2146612575 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7f573c0882e0 con 0x7f574c06e160 2024-04-25T19:17:28.867 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:17:28.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.867+0000 7f5752020640 1 -- 172.21.15.76:0/2146612575 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f57380744b0 msgr2=0x7f57380769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:28.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.867+0000 7f5752020640 1 --2- 172.21.15.76:0/2146612575 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f57380744b0 0x7f57380769c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f574c15c8b0 tx=0x7f5740008040 comp rx=0 tx=0).stop 2024-04-25T19:17:28.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.867+0000 7f5752020640 1 -- 172.21.15.76:0/2146612575 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f574c06e160 msgr2=0x7f574c1e70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:28.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.867+0000 7f5752020640 1 --2- 172.21.15.76:0/2146612575 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f574c06e160 0x7f574c1e70b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f573c02a220 tx=0x7f573c0046c0 comp rx=0 tx=0).stop 2024-04-25T19:17:28.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.867+0000 7f5752020640 1 -- 172.21.15.76:0/2146612575 shutdown_connections 2024-04-25T19:17:28.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.867+0000 7f5752020640 1 --2- 172.21.15.76:0/2146612575 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f57380744b0 0x7f57380769c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:28.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.867+0000 7f5752020640 1 --2- 172.21.15.76:0/2146612575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f574c0d0810 0x7f574c150350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:28.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.867+0000 7f5752020640 1 --2- 172.21.15.76:0/2146612575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f574c0cea70 0x7f574c15df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:28.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.867+0000 7f5752020640 1 --2- 172.21.15.76:0/2146612575 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f574c06e160 0x7f574c1e70b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:28.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.867+0000 7f5752020640 1 -- 172.21.15.76:0/2146612575 >> 172.21.15.76:0/2146612575 conn(0x7f574c0b3c00 msgr2=0x7f574c15b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:28.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.867+0000 7f5752020640 1 -- 172.21.15.76:0/2146612575 shutdown_connections 2024-04-25T19:17:28.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:28.867+0000 7f5752020640 1 -- 172.21.15.76:0/2146612575 wait complete. 2024-04-25T19:17:28.883 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T19:17:28.883 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T19:17:28.883 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T19:17:29.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.000+0000 7fb76936c640 1 Processor -- start 2024-04-25T19:17:29.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.000+0000 7fb76936c640 1 -- start start 2024-04-25T19:17:29.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.001+0000 7fb76936c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb7640c60c0 0x7fb76414ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.001+0000 7fb76936c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb764159f40 0x7fb7640c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.001+0000 7fb76936c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb764159760 con 0x7fb76415a350 2024-04-25T19:17:29.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.001+0000 7fb76936c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb7640c83d0 con 0x7fb764159f40 2024-04-25T19:17:29.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.001+0000 7fb76936c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fb7640c6ae0 con 0x7fb7640c60c0 2024-04-25T19:17:29.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.002+0000 7fb7627fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb764159f40 0x7fb7640c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.002+0000 7fb7627fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb764159f40 0x7fb7640c87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52912/0 (socket says 172.21.15.76:52912) 2024-04-25T19:17:29.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.002+0000 7fb7627fc640 1 -- 172.21.15.76:0/1714105094 learned_addr learned my addr 172.21.15.76:0/1714105094 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:17:29.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.002+0000 7fb762ffd640 1 --2- 172.21.15.76:0/1714105094 >> v2:172.21.15.105:3300/0 conn(0x7fb7640c60c0 0x7fb76414ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.002+0000 7fb7627fc640 1 -- 172.21.15.76:0/1714105094 >> v2:172.21.15.105:3300/0 conn(0x7fb7640c60c0 msgr2=0x7fb76414ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:29.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.002+0000 7fb7627fc640 1 --2- 172.21.15.76:0/1714105094 >> v2:172.21.15.105:3300/0 conn(0x7fb7640c60c0 0x7fb76414ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.002+0000 7fb7627fc640 1 -- 172.21.15.76:0/1714105094 >> v1:172.21.15.105:6789/0 conn(0x7fb76415a350 legacy=0x7fb76415cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:29.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.002+0000 7fb7627fc640 1 -- 172.21.15.76:0/1714105094 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7640c6dc0 con 0x7fb764159f40 2024-04-25T19:17:29.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.002+0000 7fb762ffd640 1 --2- 172.21.15.76:0/1714105094 >> v2:172.21.15.105:3300/0 conn(0x7fb7640c60c0 0x7fb76414ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:17:29.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.003+0000 7fb7627fc640 1 --2- 172.21.15.76:0/1714105094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb764159f40 0x7fb7640c87b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb7640c9980 tx=0x7fb75802c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb71ef075c453c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:29.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.003+0000 7fb761ffb640 1 -- 172.21.15.76:0/1714105094 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758032710 con 0x7fb764159f40 2024-04-25T19:17:29.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.003+0000 7fb761ffb640 1 -- 172.21.15.76:0/1714105094 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7580328b0 con 0x7fb764159f40 2024-04-25T19:17:29.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.003+0000 7fb761ffb640 1 -- 172.21.15.76:0/1714105094 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758032ca0 con 0x7fb764159f40 2024-04-25T19:17:29.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.004+0000 7fb76936c640 1 -- 172.21.15.76:0/1714105094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb764159f40 msgr2=0x7fb7640c87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.004+0000 7fb76936c640 1 --2- 172.21.15.76:0/1714105094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb764159f40 0x7fb7640c87b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb7640c9980 tx=0x7fb75802c850 comp rx=0 tx=0).stop 2024-04-25T19:17:29.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.004+0000 7fb76936c640 1 -- 172.21.15.76:0/1714105094 shutdown_connections 2024-04-25T19:17:29.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.004+0000 7fb76936c640 1 --2- 172.21.15.76:0/1714105094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb764159f40 0x7fb7640c87b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.004+0000 7fb76936c640 1 --2- 172.21.15.76:0/1714105094 >> v2:172.21.15.105:3300/0 conn(0x7fb7640c60c0 0x7fb76414ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.004+0000 7fb76936c640 1 -- 172.21.15.76:0/1714105094 >> 172.21.15.76:0/1714105094 conn(0x7fb764071a30 msgr2=0x7fb764071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:29.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.004+0000 7fb76936c640 1 -- 172.21.15.76:0/1714105094 shutdown_connections 2024-04-25T19:17:29.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.004+0000 7fb76936c640 1 -- 172.21.15.76:0/1714105094 wait complete. 2024-04-25T19:17:29.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.006+0000 7fb76936c640 1 Processor -- start 2024-04-25T19:17:29.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.006+0000 7fb76936c640 1 -- start start 2024-04-25T19:17:29.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.007+0000 7fb76936c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7640c60c0 0x7fb76414d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.007+0000 7fb76936c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb764159f40 0x7fb764148880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.007+0000 7fb76936c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb76415a350 0x7fb7641f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.007+0000 7fb76936c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb76415ac50 con 0x7fb764159f40 2024-04-25T19:17:29.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.007+0000 7fb76936c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb764159a20 con 0x7fb7640c60c0 2024-04-25T19:17:29.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.007+0000 7fb76936c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb7640c6600 con 0x7fb76415a350 2024-04-25T19:17:29.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.007+0000 7fb7627fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb764159f40 0x7fb764148880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.007+0000 7fb7627fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb764159f40 0x7fb764148880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52914/0 (socket says 172.21.15.76:52914) 2024-04-25T19:17:29.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.007+0000 7fb7627fc640 1 -- 172.21.15.76:0/2937549787 learned_addr learned my addr 172.21.15.76:0/2937549787 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:17:29.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.007+0000 7fb762ffd640 1 --2- 172.21.15.76:0/2937549787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7640c60c0 0x7fb76414d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.007+0000 7fb7637fe640 1 --2- 172.21.15.76:0/2937549787 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb76415a350 0x7fb7641f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.007+0000 7fb762ffd640 1 -- 172.21.15.76:0/2937549787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7640c60c0 msgr2=0x7fb76414d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-04-25T19:17:29.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.007+0000 7fb762ffd640 1 -- 172.21.15.76:0/2937549787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7640c60c0 msgr2=0x7fb76414d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T19:17:29.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.007+0000 7fb762ffd640 1 --2- 172.21.15.76:0/2937549787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7640c60c0 0x7fb76414d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:29.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.007+0000 7fb762ffd640 1 --2- 172.21.15.76:0/2937549787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7640c60c0 0x7fb76414d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:17:29.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.007+0000 7fb7627fc640 1 -- 172.21.15.76:0/2937549787 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb76415a350 msgr2=0x7fb7641f5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.007+0000 7fb7627fc640 1 --2- 172.21.15.76:0/2937549787 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb76415a350 0x7fb7641f5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.007+0000 7fb7627fc640 1 -- 172.21.15.76:0/2937549787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7640c60c0 msgr2=0x7fb76414d9a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:17:29.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.007+0000 7fb7627fc640 1 --2- 172.21.15.76:0/2937549787 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7640c60c0 0x7fb76414d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.007+0000 7fb7627fc640 1 -- 172.21.15.76:0/2937549787 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76414dee0 con 0x7fb764159f40 2024-04-25T19:17:29.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.008+0000 7fb7627fc640 1 --2- 172.21.15.76:0/2937549787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb764159f40 0x7fb764148880 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb75802c890 tx=0x7fb7580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:29.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.008+0000 7fb74b7fe640 1 -- 172.21.15.76:0/2937549787 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758003810 con 0x7fb764159f40 2024-04-25T19:17:29.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.008+0000 7fb74b7fe640 1 -- 172.21.15.76:0/2937549787 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb758043070 con 0x7fb764159f40 2024-04-25T19:17:29.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.008+0000 7fb74b7fe640 1 -- 172.21.15.76:0/2937549787 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758003c30 con 0x7fb764159f40 2024-04-25T19:17:29.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.008+0000 7fb76936c640 1 -- 172.21.15.76:0/2937549787 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7640ca7c0 con 0x7fb764159f40 2024-04-25T19:17:29.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.008+0000 7fb76936c640 1 -- 172.21.15.76:0/2937549787 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb76414d1a0 con 0x7fb764159f40 2024-04-25T19:17:29.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.009+0000 7fb76936c640 1 --2- 172.21.15.76:0/2937549787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb76414d3a0 0x7fb7641f61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.009+0000 7fb76936c640 1 -- 172.21.15.76:0/2937549787 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb76414e3b0 con 0x7fb76414d3a0 2024-04-25T19:17:29.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.009+0000 7fb762ffd640 1 --2- 172.21.15.76:0/2937549787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb76414d3a0 0x7fb7641f61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.009+0000 7fb762ffd640 1 -- 172.21.15.76:0/2937549787 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb76406a390 con 0x7fb76414d3a0 2024-04-25T19:17:29.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.010+0000 7fb762ffd640 1 --2- 172.21.15.76:0/2937549787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb76414d3a0 0x7fb7641f61b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb75400e770 tx=0x7fb75400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:29.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.010+0000 7fb74b7fe640 1 -- 172.21.15.76:0/2937549787 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754019020 con 0x7fb76414d3a0 2024-04-25T19:17:29.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.010+0000 7fb74b7fe640 1 -- 172.21.15.76:0/2937549787 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb75803d070 con 0x7fb764159f40 2024-04-25T19:17:29.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.010+0000 7fb74b7fe640 1 --2- 172.21.15.76:0/2937549787 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb7500743b0 0x7fb7500768c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.010+0000 7fb74b7fe640 1 -- 172.21.15.76:0/2937549787 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fb7580b5f60 con 0x7fb764159f40 2024-04-25T19:17:29.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.011+0000 7fb74b7fe640 1 -- 172.21.15.76:0/2937549787 <== mon.0 v2:172.21.15.76:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb75400a780 con 0x7fb76414d3a0 2024-04-25T19:17:29.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.011+0000 7fb74b7fe640 1 -- 172.21.15.76:0/2937549787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb76414d3a0 msgr2=0x7fb7641f61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.011+0000 7fb74b7fe640 1 --2- 172.21.15.76:0/2937549787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb76414d3a0 0x7fb7641f61b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb75400e770 tx=0x7fb75400ec30 comp rx=0 tx=0).stop 2024-04-25T19:17:29.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.011+0000 7fb74b7fe640 1 -- 172.21.15.76:0/2937549787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb76414d3a0 msgr2=0x7fb7641f61b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:17:29.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.011+0000 7fb74b7fe640 1 --2- 172.21.15.76:0/2937549787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb76414d3a0 0x7fb7641f61b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.011+0000 7fb7637fe640 1 --2- 172.21.15.76:0/2937549787 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb7500743b0 0x7fb7500768c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.011+0000 7fb7637fe640 1 --2- 172.21.15.76:0/2937549787 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb7500743b0 0x7fb7500768c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb7641f0330 tx=0x7fb75c00d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:29.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.020+0000 7fb76936c640 1 --2- 172.21.15.76:0/2937549787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7640c9ff0 0x7fb7641f2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.020+0000 7fb76936c640 1 -- 172.21.15.76:0/2937549787 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb764156e60 con 0x7fb7640c9ff0 2024-04-25T19:17:29.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.020+0000 7fb762ffd640 1 --2- 172.21.15.76:0/2937549787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7640c9ff0 0x7fb7641f2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.021+0000 7fb762ffd640 1 -- 172.21.15.76:0/2937549787 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb7640ca4e0 con 0x7fb7640c9ff0 2024-04-25T19:17:29.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.021+0000 7fb762ffd640 1 --2- 172.21.15.76:0/2937549787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7640c9ff0 0x7fb7641f2700 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb75401a000 tx=0x7fb754004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:29.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.021+0000 7fb74b7fe640 1 -- 172.21.15.76:0/2937549787 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754026070 con 0x7fb7640c9ff0 2024-04-25T19:17:29.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.021+0000 7fb74b7fe640 1 -- 172.21.15.76:0/2937549787 <== mon.0 v2:172.21.15.76:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5067 (secure 0 0 0) 0x7fb754015850 con 0x7fb7640c9ff0 2024-04-25T19:17:29.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.021+0000 7fb74b7fe640 1 -- 172.21.15.76:0/2937549787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7640c9ff0 msgr2=0x7fb7641f2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.021+0000 7fb74b7fe640 1 --2- 172.21.15.76:0/2937549787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7640c9ff0 0x7fb7641f2700 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb75401a000 tx=0x7fb754004730 comp rx=0 tx=0).stop 2024-04-25T19:17:29.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.021+0000 7fb74b7fe640 1 -- 172.21.15.76:0/2937549787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7640c9ff0 msgr2=0x7fb7641f2700 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:17:29.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.021+0000 7fb74b7fe640 1 --2- 172.21.15.76:0/2937549787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7640c9ff0 0x7fb7641f2700 secure :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb75401a000 tx=0x7fb754004730 comp rx=0 tx=0).stop 2024-04-25T19:17:29.024 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:17:29.024 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:17:29.025 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:17:29.025 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "leader", 2024-04-25T19:17:29.025 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 16, 2024-04-25T19:17:29.025 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 147547, 2024-04-25T19:17:29.025 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:17:29.025 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:17:29.025 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-04-25T19:17:29.025 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:17:29.025 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:29.025 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 0, 2024-04-25T19:17:29.025 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:17:29.025 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:17:29.025 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:17:29.025 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:17:29.025 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:17:29.026 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:17:29.026 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:17:29.026 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:17:29.026 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:17:29.026 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:17:29.026 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:17:29.026 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:17:29.026 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:17:29.026 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:17:29.026 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:29.026 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:17:29.026 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:17:29.026 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:17:29.026 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:17:29.026 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:17:29.027 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:17:29.027 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:17:29.027 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:17:29.027 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:17:29.027 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:17:29.027 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:17:29.027 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:17:29.027 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:17:29.027 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:29.027 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.027 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:17:29.027 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:17:29.027 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:17:29.027 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:17:29.027 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:17:29.027 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:17:29.028 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:17:29.028 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:17:29.028 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:17:29.028 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:17:29.028 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:17:29.028 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:17:29.028 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:17:29.028 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:17:29.028 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:17:29.028 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:17:29.028 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:17:29.028 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:17:29.028 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:17:29.028 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:17:29.028 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:17:29.028 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:17:29.028 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:17:29.029 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:17:29.029 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:17:29.029 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:17:29.029 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:17:29.029 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:17:29.029 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:29.029 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:17:29.029 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.029 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:17:29.029 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.029 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:17:29.029 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:17:29.029 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:17:29.029 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:17:29.029 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.030 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:17:29.030 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:17:29.030 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:29.030 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.030 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.030 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:17:29.030 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:17:29.030 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:29.030 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:29.030 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:29.030 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.030 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:17:29.030 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:17:29.030 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:17:29.030 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:17:29.030 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:17:29.031 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.031 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.031 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:17:29.031 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:17:29.031 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:17:29.031 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:17:29.031 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.031 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:17:29.031 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:17:29.031 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:29.031 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.031 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.031 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:17:29.031 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:17:29.031 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:29.031 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:29.031 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:29.032 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.032 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:17:29.032 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:17:29.032 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:17:29.032 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:17:29.032 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:17:29.032 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.032 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.032 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:17:29.032 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:17:29.032 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:17:29.032 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:17:29.032 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.032 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:17:29.032 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:17:29.032 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:29.032 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.033 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.033 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:17:29.033 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:17:29.033 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:29.033 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:29.033 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:29.033 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.033 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:17:29.033 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:17:29.033 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:17:29.033 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:17:29.033 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:17:29.033 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:29.033 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:29.033 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.033 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:17:29.034 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:17:29.034 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.034 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:29.034 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:29.034 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:17:29.034 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:29.034 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:29.034 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": [ 2024-04-25T19:17:29.034 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.034 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:29.034 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:29.034 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 6 2024-04-25T19:17:29.034 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:29.034 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:29.034 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": [ 2024-04-25T19:17:29.034 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.035 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:29.035 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:29.035 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 7 2024-04-25T19:17:29.035 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:29.035 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:29.035 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:17:29.035 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.035 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:29.035 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:29.035 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 6 2024-04-25T19:17:29.035 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:29.035 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:29.035 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": [ 2024-04-25T19:17:29.035 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.035 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:29.036 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:29.036 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 2 2024-04-25T19:17:29.036 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:29.036 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:29.036 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.036 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:17:29.036 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:17:29.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.022+0000 7fb76936c640 1 -- 172.21.15.76:0/2937549787 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb7500743b0 msgr2=0x7fb7500768c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.022+0000 7fb76936c640 1 --2- 172.21.15.76:0/2937549787 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb7500743b0 0x7fb7500768c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb7641f0330 tx=0x7fb75c00d040 comp rx=0 tx=0).stop 2024-04-25T19:17:29.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.022+0000 7fb76936c640 1 -- 172.21.15.76:0/2937549787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb764159f40 msgr2=0x7fb764148880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.022+0000 7fb76936c640 1 --2- 172.21.15.76:0/2937549787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb764159f40 0x7fb764148880 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb75802c890 tx=0x7fb7580046c0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.022+0000 7fb7637fe640 1 -- 172.21.15.76:0/2937549787 reap_dead start 2024-04-25T19:17:29.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.022+0000 7fb76936c640 1 -- 172.21.15.76:0/2937549787 shutdown_connections 2024-04-25T19:17:29.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.022+0000 7fb76936c640 1 --2- 172.21.15.76:0/2937549787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb764159f40 0x7fb764148880 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.022+0000 7fb76936c640 1 -- 172.21.15.76:0/2937549787 >> 172.21.15.76:0/2937549787 conn(0x7fb764071a30 msgr2=0x7fb7640c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:29.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.022+0000 7fb76936c640 1 -- 172.21.15.76:0/2937549787 shutdown_connections 2024-04-25T19:17:29.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.022+0000 7fb76936c640 1 -- 172.21.15.76:0/2937549787 wait complete. 2024-04-25T19:17:29.038 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T19:17:29.038 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T19:17:29.038 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T19:17:29.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.152+0000 7f98af372640 1 Processor -- start 2024-04-25T19:17:29.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.152+0000 7f98af372640 1 -- start start 2024-04-25T19:17:29.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.154+0000 7f98af372640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f98a8154c20 0x7f98a80cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.154+0000 7f98af372640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f98a80cd560 0x7f98a815aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.154+0000 7f98af372640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98a80cb820 con 0x7f98a80c6040 2024-04-25T19:17:29.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.154+0000 7f98af372640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f98a80cc730 con 0x7f98a8154c20 2024-04-25T19:17:29.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.154+0000 7f98af372640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f98a80cbac0 con 0x7f98a80cd560 2024-04-25T19:17:29.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.154+0000 7f98ac90d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f98a8154c20 0x7f98a80cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.154+0000 7f98ac90d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f98a8154c20 0x7f98a80cd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52942/0 (socket says 172.21.15.76:52942) 2024-04-25T19:17:29.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.154+0000 7f98ac90d640 1 -- 172.21.15.76:0/3716934780 learned_addr learned my addr 172.21.15.76:0/3716934780 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:17:29.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.154+0000 7f98ad90f640 1 --2- 172.21.15.76:0/3716934780 >> v2:172.21.15.105:3300/0 conn(0x7f98a80cd560 0x7f98a815aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.154+0000 7f98ad90f640 1 -- 172.21.15.76:0/3716934780 >> v2:172.21.15.105:3300/0 conn(0x7f98a80cd560 msgr2=0x7f98a815aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T19:17:29.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.154+0000 7f98ad90f640 1 -- 172.21.15.76:0/3716934780 >> v2:172.21.15.105:3300/0 conn(0x7f98a80cd560 msgr2=0x7f98a815aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:17:29.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.154+0000 7f98ad90f640 1 --2- 172.21.15.76:0/3716934780 >> v2:172.21.15.105:3300/0 conn(0x7f98a80cd560 0x7f98a815aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:29.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.154+0000 7f98ac90d640 1 -- 172.21.15.76:0/3716934780 >> v2:172.21.15.105:3300/0 conn(0x7f98a80cd560 msgr2=0x7f98a815aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:29.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.154+0000 7f98ad90f640 1 --2- 172.21.15.76:0/3716934780 >> v2:172.21.15.105:3300/0 conn(0x7f98a80cd560 0x7f98a815aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:17:29.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.154+0000 7f98ac90d640 1 --2- 172.21.15.76:0/3716934780 >> v2:172.21.15.105:3300/0 conn(0x7f98a80cd560 0x7f98a815aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.154+0000 7f98ac90d640 1 -- 172.21.15.76:0/3716934780 >> v1:172.21.15.105:6789/0 conn(0x7f98a80c6040 legacy=0x7f98a80cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.154+0000 7f98ac90d640 1 -- 172.21.15.76:0/3716934780 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a80cbda0 con 0x7f98a8154c20 2024-04-25T19:17:29.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.155+0000 7f98ac90d640 1 --2- 172.21.15.76:0/3716934780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98a8154c20 0x7f98a80cd020 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f98a80b3690 tx=0x7f989802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a42e3fb124d06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:29.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.156+0000 7f989ffff640 1 -- 172.21.15.76:0/3716934780 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9898037070 con 0x7f98a8154c20 2024-04-25T19:17:29.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.156+0000 7f989ffff640 1 -- 172.21.15.76:0/3716934780 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f989802ae50 con 0x7f98a8154c20 2024-04-25T19:17:29.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.156+0000 7f989ffff640 1 -- 172.21.15.76:0/3716934780 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9898032860 con 0x7f98a8154c20 2024-04-25T19:17:29.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.157+0000 7f98af372640 1 -- 172.21.15.76:0/3716934780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98a8154c20 msgr2=0x7f98a80cd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.157+0000 7f98af372640 1 --2- 172.21.15.76:0/3716934780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98a8154c20 0x7f98a80cd020 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f98a80b3690 tx=0x7f989802a300 comp rx=0 tx=0).stop 2024-04-25T19:17:29.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.157+0000 7f98af372640 1 -- 172.21.15.76:0/3716934780 shutdown_connections 2024-04-25T19:17:29.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.157+0000 7f98af372640 1 --2- 172.21.15.76:0/3716934780 >> v2:172.21.15.105:3300/0 conn(0x7f98a80cd560 0x7f98a815aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.157+0000 7f98af372640 1 --2- 172.21.15.76:0/3716934780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98a8154c20 0x7f98a80cd020 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.157+0000 7f98af372640 1 -- 172.21.15.76:0/3716934780 >> 172.21.15.76:0/3716934780 conn(0x7f98a8072b60 msgr2=0x7f98a8072f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:29.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.157+0000 7f98af372640 1 -- 172.21.15.76:0/3716934780 shutdown_connections 2024-04-25T19:17:29.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.157+0000 7f98af372640 1 -- 172.21.15.76:0/3716934780 wait complete. 2024-04-25T19:17:29.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.159+0000 7f98af372640 1 Processor -- start 2024-04-25T19:17:29.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.159+0000 7f98af372640 1 -- start start 2024-04-25T19:17:29.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.161+0000 7f98af372640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98a80c6040 0x7f98a81f13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.161+0000 7f98af372640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98a80cd560 0x7f98a8070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.161+0000 7f98af372640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98a8154c20 0x7f98a80d0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.161+0000 7f98af372640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f98a8155030 con 0x7f98a80c6040 2024-04-25T19:17:29.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.161+0000 7f98af372640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f98a80cfe00 con 0x7f98a8154c20 2024-04-25T19:17:29.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.161+0000 7f98af372640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f98a815af50 con 0x7f98a80cd560 2024-04-25T19:17:29.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.161+0000 7f98ad10e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98a80c6040 0x7f98a81f13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.161+0000 7f98ad10e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98a80c6040 0x7f98a81f13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52956/0 (socket says 172.21.15.76:52956) 2024-04-25T19:17:29.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.161+0000 7f98ad10e640 1 -- 172.21.15.76:0/3548584262 learned_addr learned my addr 172.21.15.76:0/3548584262 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:17:29.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.161+0000 7f98ac90d640 1 --2- 172.21.15.76:0/3548584262 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98a80cd560 0x7f98a8070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.162+0000 7f98ad90f640 1 --2- 172.21.15.76:0/3548584262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98a8154c20 0x7f98a80d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.162+0000 7f98ad10e640 1 -- 172.21.15.76:0/3548584262 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98a80cd560 msgr2=0x7f98a8070460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.162+0000 7f98ad10e640 1 --2- 172.21.15.76:0/3548584262 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f98a80cd560 0x7f98a8070460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.162+0000 7f98ad10e640 1 -- 172.21.15.76:0/3548584262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98a8154c20 msgr2=0x7f98a80d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.162+0000 7f98ad10e640 1 --2- 172.21.15.76:0/3548584262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98a8154c20 0x7f98a80d0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.162+0000 7f98ad10e640 1 -- 172.21.15.76:0/3548584262 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a81f18f0 con 0x7f98a80c6040 2024-04-25T19:17:29.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.162+0000 7f98ad10e640 1 --2- 172.21.15.76:0/3548584262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98a80c6040 0x7f98a81f13b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f98a400c370 tx=0x7f98a400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:29.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.162+0000 7f989dffb640 1 -- 172.21.15.76:0/3548584262 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a4017070 con 0x7f98a80c6040 2024-04-25T19:17:29.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.162+0000 7f989dffb640 1 -- 172.21.15.76:0/3548584262 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98a40052d0 con 0x7f98a80c6040 2024-04-25T19:17:29.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.162+0000 7f989dffb640 1 -- 172.21.15.76:0/3548584262 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a4003850 con 0x7f98a80c6040 2024-04-25T19:17:29.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.162+0000 7f98af372640 1 -- 172.21.15.76:0/3548584262 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a80718e0 con 0x7f98a80c6040 2024-04-25T19:17:29.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.162+0000 7f98af372640 1 -- 172.21.15.76:0/3548584262 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a8071320 con 0x7f98a80c6040 2024-04-25T19:17:29.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.163+0000 7f98af372640 1 --2- 172.21.15.76:0/3548584262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98a815d800 0x7f98a81f4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.163+0000 7f98af372640 1 -- 172.21.15.76:0/3548584262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f98a8070d80 con 0x7f98a815d800 2024-04-25T19:17:29.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.164+0000 7f98ac90d640 1 --2- 172.21.15.76:0/3548584262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98a815d800 0x7f98a81f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.164+0000 7f98ac90d640 1 -- 172.21.15.76:0/3548584262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f98a806a390 con 0x7f98a815d800 2024-04-25T19:17:29.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.164+0000 7f98ac90d640 1 --2- 172.21.15.76:0/3548584262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98a815d800 0x7f98a81f4260 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f98a80b2de0 tx=0x7f98980046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:29.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.164+0000 7f989dffb640 1 -- 172.21.15.76:0/3548584262 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f98a401a070 con 0x7f98a80c6040 2024-04-25T19:17:29.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.165+0000 7f989dffb640 1 --2- 172.21.15.76:0/3548584262 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f988c0744b0 0x7f988c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.165+0000 7f989dffb640 1 -- 172.21.15.76:0/3548584262 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f98a4094520 con 0x7f98a80c6040 2024-04-25T19:17:29.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.165+0000 7f98ad90f640 1 --2- 172.21.15.76:0/3548584262 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f988c0744b0 0x7f988c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.165+0000 7f989dffb640 1 -- 172.21.15.76:0/3548584262 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9898042070 con 0x7f98a815d800 2024-04-25T19:17:29.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.165+0000 7f989dffb640 1 -- 172.21.15.76:0/3548584262 <== mon.1 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f989802ae50 con 0x7f98a815d800 2024-04-25T19:17:29.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.165+0000 7f989dffb640 1 -- 172.21.15.76:0/3548584262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98a815d800 msgr2=0x7f98a81f4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.165+0000 7f989dffb640 1 --2- 172.21.15.76:0/3548584262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98a815d800 0x7f98a81f4260 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f98a80b2de0 tx=0x7f98980046c0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.165+0000 7f989dffb640 1 -- 172.21.15.76:0/3548584262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98a815d800 msgr2=0x7f98a81f4260 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:17:29.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.165+0000 7f989dffb640 1 --2- 172.21.15.76:0/3548584262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98a815d800 0x7f98a81f4260 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.165+0000 7f98ad90f640 1 --2- 172.21.15.76:0/3548584262 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f988c0744b0 0x7f988c0769c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f98a80d0120 tx=0x7f98a00097d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:29.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.172+0000 7f98af372640 1 --2- 172.21.15.76:0/3548584262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98a81f47a0 0x7f98a81f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.172+0000 7f98af372640 1 -- 172.21.15.76:0/3548584262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f98a81f50f0 con 0x7f98a81f47a0 2024-04-25T19:17:29.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.172+0000 7f98ac90d640 1 --2- 172.21.15.76:0/3548584262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98a81f47a0 0x7f98a81f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.173+0000 7f98ac90d640 1 -- 172.21.15.76:0/3548584262 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f98a81f0cf0 con 0x7f98a81f47a0 2024-04-25T19:17:29.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.173+0000 7f98ac90d640 1 --2- 172.21.15.76:0/3548584262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98a81f47a0 0x7f98a81f4bb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9898000c00 tx=0x7f9898008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:29.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.174+0000 7f989dffb640 1 -- 172.21.15.76:0/3548584262 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9898037040 con 0x7f98a81f47a0 2024-04-25T19:17:29.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.174+0000 7f989dffb640 1 -- 172.21.15.76:0/3548584262 <== mon.1 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4552 (secure 0 0 0) 0x7f9898003610 con 0x7f98a81f47a0 2024-04-25T19:17:29.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.174+0000 7f989dffb640 1 -- 172.21.15.76:0/3548584262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98a81f47a0 msgr2=0x7f98a81f4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.174+0000 7f989dffb640 1 --2- 172.21.15.76:0/3548584262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98a81f47a0 0x7f98a81f4bb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9898000c00 tx=0x7f9898008030 comp rx=0 tx=0).stop 2024-04-25T19:17:29.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.174+0000 7f989dffb640 1 -- 172.21.15.76:0/3548584262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98a81f47a0 msgr2=0x7f98a81f4bb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:17:29.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.174+0000 7f989dffb640 1 --2- 172.21.15.76:0/3548584262 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98a81f47a0 0x7f98a81f4bb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.177 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:17:29.177 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:17:29.177 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:17:29.177 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "peon", 2024-04-25T19:17:29.177 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 16, 2024-04-25T19:17:29.177 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 666, 2024-04-25T19:17:29.177 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:17:29.178 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:17:29.178 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-04-25T19:17:29.178 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:17:29.178 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:29.178 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 0, 2024-04-25T19:17:29.178 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:17:29.178 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:17:29.178 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:17:29.178 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:17:29.178 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:17:29.178 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:17:29.178 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:17:29.178 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:17:29.178 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:17:29.178 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:17:29.179 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:17:29.179 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:17:29.179 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:17:29.179 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:17:29.179 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:29.179 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:17:29.179 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:17:29.179 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:17:29.179 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:17:29.179 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:17:29.179 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:17:29.179 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:17:29.179 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:17:29.179 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:17:29.179 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:17:29.180 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:17:29.180 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:17:29.180 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:17:29.180 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:29.180 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.180 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:17:29.180 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:17:29.180 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:17:29.180 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:17:29.180 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:17:29.180 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:17:29.180 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:17:29.180 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:17:29.180 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:17:29.180 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:17:29.180 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:17:29.180 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:17:29.181 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:17:29.181 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:17:29.181 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:17:29.181 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:17:29.181 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:17:29.181 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:17:29.181 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:17:29.181 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:17:29.181 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:17:29.181 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:17:29.181 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:17:29.181 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:17:29.181 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:17:29.181 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:17:29.181 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:17:29.182 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:17:29.182 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:29.182 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:17:29.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.182 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:17:29.182 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.182 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:17:29.182 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:17:29.182 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:17:29.182 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:17:29.182 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.182 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:17:29.182 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:17:29.182 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:29.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.183 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:17:29.183 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:17:29.183 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:29.183 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:29.183 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:29.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.183 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:17:29.183 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:17:29.183 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:17:29.183 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:17:29.183 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:17:29.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.183 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:17:29.184 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:17:29.184 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:17:29.184 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:17:29.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.184 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:17:29.184 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:17:29.184 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:29.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.184 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:17:29.184 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:17:29.184 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:29.184 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:29.184 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:29.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.184 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:17:29.185 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:17:29.185 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:17:29.185 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:17:29.185 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:17:29.185 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.185 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:17:29.185 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:17:29.185 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:17:29.185 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:17:29.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.185 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:17:29.185 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:17:29.185 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:29.185 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.186 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:17:29.186 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:17:29.186 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:29.186 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:29.186 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:29.186 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.186 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:17:29.186 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:17:29.186 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:17:29.186 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:17:29.186 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:17:29.186 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:29.186 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:29.186 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.186 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:17:29.187 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:17:29.187 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.187 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:29.187 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:29.187 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:17:29.187 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:29.187 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:29.187 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:17:29.187 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.187 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:29.187 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:29.187 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:17:29.187 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:29.187 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:29.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.187 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:17:29.187 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:17:29.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.174+0000 7f98af372640 1 -- 172.21.15.76:0/3548584262 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f988c0744b0 msgr2=0x7f988c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.174+0000 7f98af372640 1 --2- 172.21.15.76:0/3548584262 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f988c0744b0 0x7f988c0769c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f98a80d0120 tx=0x7f98a00097d0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.174+0000 7f98af372640 1 -- 172.21.15.76:0/3548584262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98a80c6040 msgr2=0x7f98a81f13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.174+0000 7f98ad90f640 1 -- 172.21.15.76:0/3548584262 reap_dead start 2024-04-25T19:17:29.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.174+0000 7f98af372640 1 --2- 172.21.15.76:0/3548584262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98a80c6040 0x7f98a81f13b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f98a400c370 tx=0x7f98a400c830 comp rx=0 tx=0).stop 2024-04-25T19:17:29.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.175+0000 7f98af372640 1 -- 172.21.15.76:0/3548584262 shutdown_connections 2024-04-25T19:17:29.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.175+0000 7f98af372640 1 --2- 172.21.15.76:0/3548584262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98a80c6040 0x7f98a81f13b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.175+0000 7f98af372640 1 -- 172.21.15.76:0/3548584262 >> 172.21.15.76:0/3548584262 conn(0x7f98a8072b60 msgr2=0x7f98a80ce8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:29.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.175+0000 7f98af372640 1 -- 172.21.15.76:0/3548584262 shutdown_connections 2024-04-25T19:17:29.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.175+0000 7f98af372640 1 -- 172.21.15.76:0/3548584262 wait complete. 2024-04-25T19:17:29.190 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T19:17:29.190 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T19:17:29.190 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T19:17:29.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.308+0000 7f207eb06640 1 Processor -- start 2024-04-25T19:17:29.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.308+0000 7f207eb06640 1 -- start start 2024-04-25T19:17:29.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.309+0000 7f207eb06640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f20780d0810 0x7f207806f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.310+0000 7f207eb06640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f207815ce80 con 0x7f20780cea70 2024-04-25T19:17:29.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.310+0000 7f207eb06640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f207815d160 con 0x7f207806e870 2024-04-25T19:17:29.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.310+0000 7f207eb06640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f207815d440 con 0x7f20780d0810 2024-04-25T19:17:29.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.310+0000 7f2077fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f20780cea70 0x7f207806e160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48706/0 (socket says 172.21.15.76:48706) 2024-04-25T19:17:29.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.310+0000 7f2077fff640 1 -- 172.21.15.76:0/947796502 learned_addr learned my addr 172.21.15.76:0/947796502 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:17:29.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.310+0000 7f207c8a2640 1 --2- 172.21.15.76:0/947796502 >> v2:172.21.15.105:3300/0 conn(0x7f20780d0810 0x7f207806f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.310+0000 7f20777fe640 1 -- 172.21.15.76:0/947796502 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 475865278 0 0) 0x7f207815ce80 con 0x7f20780cea70 2024-04-25T19:17:29.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.310+0000 7f20777fe640 1 -- 172.21.15.76:0/947796502 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2068003520 con 0x7f20780cea70 2024-04-25T19:17:29.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.310+0000 7f20777fe640 1 -- 172.21.15.76:0/947796502 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 93780299 0 0) 0x7f207815d160 con 0x7f207806e870 2024-04-25T19:17:29.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.310+0000 7f20777fe640 1 -- 172.21.15.76:0/947796502 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f207815ce80 con 0x7f207806e870 2024-04-25T19:17:29.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.310+0000 7f207c8a2640 1 -- 172.21.15.76:0/947796502 >> v1:172.21.15.105:6789/0 conn(0x7f207806e870 legacy=0x7f20781674f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.310+0000 7f207c8a2640 1 -- 172.21.15.76:0/947796502 >> v1:172.21.15.76:6789/0 conn(0x7f20780cea70 legacy=0x7f207806e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.310+0000 7f207c8a2640 1 -- 172.21.15.76:0/947796502 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207815d720 con 0x7f20780d0810 2024-04-25T19:17:29.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.311+0000 7f207c8a2640 1 --2- 172.21.15.76:0/947796502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20780d0810 0x7f207806f000 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f206c0097a0 tx=0x7f206c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b6dbd67d5a6beea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:29.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.311+0000 7f20777fe640 1 -- 172.21.15.76:0/947796502 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c037070 con 0x7f20780d0810 2024-04-25T19:17:29.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.311+0000 7f20777fe640 1 -- 172.21.15.76:0/947796502 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f206c032440 con 0x7f20780d0810 2024-04-25T19:17:29.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.311+0000 7f20777fe640 1 -- 172.21.15.76:0/947796502 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c032810 con 0x7f20780d0810 2024-04-25T19:17:29.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.312+0000 7f207eb06640 1 -- 172.21.15.76:0/947796502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20780d0810 msgr2=0x7f207806f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.312+0000 7f207eb06640 1 --2- 172.21.15.76:0/947796502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20780d0810 0x7f207806f000 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f206c0097a0 tx=0x7f206c02a230 comp rx=0 tx=0).stop 2024-04-25T19:17:29.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.312+0000 7f207eb06640 1 -- 172.21.15.76:0/947796502 shutdown_connections 2024-04-25T19:17:29.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.312+0000 7f207eb06640 1 --2- 172.21.15.76:0/947796502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20780d0810 0x7f207806f000 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.312+0000 7f207eb06640 1 -- 172.21.15.76:0/947796502 >> 172.21.15.76:0/947796502 conn(0x7f20780b3c00 msgr2=0x7f20780b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:29.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.312+0000 7f207eb06640 1 -- 172.21.15.76:0/947796502 shutdown_connections 2024-04-25T19:17:29.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.312+0000 7f207eb06640 1 -- 172.21.15.76:0/947796502 wait complete. 2024-04-25T19:17:29.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.314+0000 7f207eb06640 1 Processor -- start 2024-04-25T19:17:29.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.314+0000 7f207eb06640 1 -- start start 2024-04-25T19:17:29.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.315+0000 7f207eb06640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f207806e870 0x7f20781f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.315+0000 7f207eb06640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20780cea70 0x7f20781f3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.315+0000 7f207eb06640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f20780d0810 0x7f20780d0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.315+0000 7f207eb06640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f207815cb70 con 0x7f207806e870 2024-04-25T19:17:29.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.315+0000 7f207eb06640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f207806dc40 con 0x7f20780cea70 2024-04-25T19:17:29.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.315+0000 7f207eb06640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f207806f540 con 0x7f20780d0810 2024-04-25T19:17:29.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.315+0000 7f207c8a2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f207806e870 0x7f20781f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.315+0000 7f207c8a2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f207806e870 0x7f20781f6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52960/0 (socket says 172.21.15.76:52960) 2024-04-25T19:17:29.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.315+0000 7f207c8a2640 1 -- 172.21.15.76:0/3201917124 learned_addr learned my addr 172.21.15.76:0/3201917124 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:17:29.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.315+0000 7f2077fff640 1 --2- 172.21.15.76:0/3201917124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20780cea70 0x7f20781f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.315+0000 7f207d0a3640 1 --2- 172.21.15.76:0/3201917124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f20780d0810 0x7f20780d0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.316+0000 7f207c8a2640 1 -- 172.21.15.76:0/3201917124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f20780d0810 msgr2=0x7f20780d0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.316+0000 7f207c8a2640 1 --2- 172.21.15.76:0/3201917124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f20780d0810 0x7f20780d0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.316+0000 7f207c8a2640 1 -- 172.21.15.76:0/3201917124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20780cea70 msgr2=0x7f20781f3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.316+0000 7f207c8a2640 1 --2- 172.21.15.76:0/3201917124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20780cea70 0x7f20781f3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.316+0000 7f207c8a2640 1 -- 172.21.15.76:0/3201917124 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20781f73c0 con 0x7f207806e870 2024-04-25T19:17:29.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.316+0000 7f207c8a2640 1 --2- 172.21.15.76:0/3201917124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f207806e870 0x7f20781f6e80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f206c0098c0 tx=0x7f206c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:29.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.316+0000 7f20757fa640 1 -- 172.21.15.76:0/3201917124 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c041030 con 0x7f207806e870 2024-04-25T19:17:29.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.316+0000 7f20757fa640 1 -- 172.21.15.76:0/3201917124 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f206c03d4e0 con 0x7f207806e870 2024-04-25T19:17:29.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.316+0000 7f207eb06640 1 -- 172.21.15.76:0/3201917124 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2078152600 con 0x7f207806e870 2024-04-25T19:17:29.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.316+0000 7f20757fa640 1 -- 172.21.15.76:0/3201917124 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c037040 con 0x7f207806e870 2024-04-25T19:17:29.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.316+0000 7f207eb06640 1 -- 172.21.15.76:0/3201917124 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2078152320 con 0x7f207806e870 2024-04-25T19:17:29.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.317+0000 7f207eb06640 1 --2- 172.21.15.76:0/3201917124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f20781517c0 0x7f20780d11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.317+0000 7f207eb06640 1 -- 172.21.15.76:0/3201917124 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f2078151d40 con 0x7f20781517c0 2024-04-25T19:17:29.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.317+0000 7f2077fff640 1 --2- 172.21.15.76:0/3201917124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f20781517c0 0x7f20780d11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.318+0000 7f2077fff640 1 -- 172.21.15.76:0/3201917124 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2078153320 con 0x7f20781517c0 2024-04-25T19:17:29.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.318+0000 7f20757fa640 1 -- 172.21.15.76:0/3201917124 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f206c044070 con 0x7f207806e870 2024-04-25T19:17:29.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.318+0000 7f2077fff640 1 --2- 172.21.15.76:0/3201917124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f20781517c0 0x7f20780d11f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2064000f60 tx=0x7f2064001910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:29.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.318+0000 7f20757fa640 1 --2- 172.21.15.76:0/3201917124 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f205c0744b0 0x7f205c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.319+0000 7f20757fa640 1 -- 172.21.15.76:0/3201917124 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f206c0b5810 con 0x7f207806e870 2024-04-25T19:17:29.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.319+0000 7f207d0a3640 1 --2- 172.21.15.76:0/3201917124 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f205c0744b0 0x7f205c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.319+0000 7f20757fa640 1 -- 172.21.15.76:0/3201917124 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064019070 con 0x7f20781517c0 2024-04-25T19:17:29.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.319+0000 7f20757fa640 1 -- 172.21.15.76:0/3201917124 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2064005a60 con 0x7f20781517c0 2024-04-25T19:17:29.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.319+0000 7f20757fa640 1 -- 172.21.15.76:0/3201917124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f20781517c0 msgr2=0x7f20780d11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.319+0000 7f20757fa640 1 --2- 172.21.15.76:0/3201917124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f20781517c0 0x7f20780d11f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2064000f60 tx=0x7f2064001910 comp rx=0 tx=0).stop 2024-04-25T19:17:29.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.319+0000 7f20757fa640 1 -- 172.21.15.76:0/3201917124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f20781517c0 msgr2=0x7f20780d11f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:17:29.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.319+0000 7f20757fa640 1 --2- 172.21.15.76:0/3201917124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f20781517c0 0x7f20780d11f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.319+0000 7f207d0a3640 1 --2- 172.21.15.76:0/3201917124 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f205c0744b0 0x7f205c0769c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2078156e20 tx=0x7f2070009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:29.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.328+0000 7f207eb06640 1 --2- 172.21.15.76:0/3201917124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f207815dd30 0x7f20781f65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.328+0000 7f207eb06640 1 -- 172.21.15.76:0/3201917124 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f20781f6b20 con 0x7f207815dd30 2024-04-25T19:17:29.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.328+0000 7f2077fff640 1 --2- 172.21.15.76:0/3201917124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f207815dd30 0x7f20781f65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.328+0000 7f2077fff640 1 -- 172.21.15.76:0/3201917124 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f207806a390 con 0x7f207815dd30 2024-04-25T19:17:29.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.329+0000 7f2077fff640 1 --2- 172.21.15.76:0/3201917124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f207815dd30 0x7f20781f65e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f207815a780 tx=0x7f2064001150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:29.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.329+0000 7f20757fa640 1 -- 172.21.15.76:0/3201917124 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064019070 con 0x7f207815dd30 2024-04-25T19:17:29.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.329+0000 7f20757fa640 1 -- 172.21.15.76:0/3201917124 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4724 (secure 0 0 0) 0x7f2064005240 con 0x7f207815dd30 2024-04-25T19:17:29.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.329+0000 7f20757fa640 1 -- 172.21.15.76:0/3201917124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f207815dd30 msgr2=0x7f20781f65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.329+0000 7f20757fa640 1 --2- 172.21.15.76:0/3201917124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f207815dd30 0x7f20781f65e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f207815a780 tx=0x7f2064001150 comp rx=0 tx=0).stop 2024-04-25T19:17:29.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.329+0000 7f20757fa640 1 -- 172.21.15.76:0/3201917124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f207815dd30 msgr2=0x7f20781f65e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:17:29.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.329+0000 7f20757fa640 1 --2- 172.21.15.76:0/3201917124 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f207815dd30 0x7f20781f65e0 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f207815a780 tx=0x7f2064001150 comp rx=0 tx=0).stop 2024-04-25T19:17:29.334 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:17:29.334 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:17:29.334 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:17:29.334 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "peon", 2024-04-25T19:17:29.334 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 16, 2024-04-25T19:17:29.335 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 53108, 2024-04-25T19:17:29.335 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:17:29.335 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:17:29.335 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-04-25T19:17:29.335 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:17:29.335 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:29.335 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 0, 2024-04-25T19:17:29.335 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:17:29.335 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:17:29.335 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:17:29.335 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:17:29.335 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:17:29.335 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:17:29.335 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:17:29.336 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:17:29.336 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:17:29.336 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:17:29.336 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:17:29.336 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:17:29.336 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:17:29.336 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:17:29.336 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:29.336 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:17:29.336 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:17:29.336 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:17:29.336 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:17:29.336 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:17:29.336 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:17:29.336 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:17:29.337 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:17:29.337 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:17:29.337 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:17:29.337 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:17:29.337 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:17:29.337 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:17:29.337 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:29.337 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.337 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:17:29.337 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:17:29.337 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:17:29.337 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:17:29.337 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:17:29.337 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:17:29.337 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:17:29.338 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:17:29.338 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:17:29.338 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:17:29.338 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:17:29.338 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:17:29.338 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:17:29.338 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:17:29.338 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:17:29.338 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:17:29.338 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:17:29.338 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:17:29.338 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:17:29.338 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:17:29.338 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:17:29.338 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:17:29.339 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:17:29.339 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:17:29.339 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:17:29.339 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:17:29.339 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:17:29.339 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:17:29.339 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:29.339 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:17:29.339 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.339 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:17:29.339 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.339 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:17:29.339 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:17:29.339 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:17:29.340 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:17:29.340 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.340 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:17:29.340 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:17:29.340 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:29.340 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.340 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.340 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:17:29.340 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:17:29.340 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:29.340 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:29.340 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:29.340 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.340 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:17:29.340 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:17:29.341 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:17:29.341 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:17:29.341 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:17:29.341 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.341 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.341 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:17:29.341 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:17:29.341 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:17:29.341 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:17:29.341 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.341 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:17:29.341 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:17:29.341 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:29.341 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.342 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.342 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:17:29.342 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:17:29.342 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:29.342 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:29.342 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:29.342 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.342 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:17:29.342 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:17:29.342 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:17:29.342 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:17:29.342 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:17:29.342 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.342 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.343 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:17:29.343 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:17:29.343 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:17:29.343 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:17:29.343 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.343 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:17:29.343 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:17:29.343 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:29.343 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.343 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.343 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:17:29.343 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:17:29.343 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:29.343 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:29.344 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:29.344 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.344 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:17:29.344 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:17:29.344 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:17:29.344 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:17:29.344 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:17:29.344 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:29.344 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:29.344 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.344 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:17:29.344 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:17:29.344 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.344 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:29.344 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:29.345 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:17:29.345 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:29.345 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:29.345 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": [ 2024-04-25T19:17:29.345 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.345 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:29.345 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:29.345 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:17:29.345 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:29.345 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:29.345 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:17:29.345 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:29.345 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:29.345 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:29.345 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 2 2024-04-25T19:17:29.346 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:29.346 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:29.346 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:29.346 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:17:29.346 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:17:29.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.330+0000 7f207eb06640 1 -- 172.21.15.76:0/3201917124 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f205c0744b0 msgr2=0x7f205c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.330+0000 7f207eb06640 1 --2- 172.21.15.76:0/3201917124 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f205c0744b0 0x7f205c0769c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2078156e20 tx=0x7f2070009040 comp rx=0 tx=0).stop 2024-04-25T19:17:29.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.330+0000 7f207eb06640 1 -- 172.21.15.76:0/3201917124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f207806e870 msgr2=0x7f20781f6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.330+0000 7f207eb06640 1 --2- 172.21.15.76:0/3201917124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f207806e870 0x7f20781f6e80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f206c0098c0 tx=0x7f206c0046c0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.330+0000 7f207d0a3640 1 -- 172.21.15.76:0/3201917124 reap_dead start 2024-04-25T19:17:29.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.330+0000 7f207d0a3640 1 -- 172.21.15.76:0/3201917124 reap_dead start 2024-04-25T19:17:29.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.330+0000 7f207eb06640 1 -- 172.21.15.76:0/3201917124 shutdown_connections 2024-04-25T19:17:29.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.330+0000 7f207eb06640 1 -- 172.21.15.76:0/3201917124 >> 172.21.15.76:0/3201917124 conn(0x7f20780b3c00 msgr2=0x7f20780cc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:29.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.330+0000 7f207eb06640 1 -- 172.21.15.76:0/3201917124 shutdown_connections 2024-04-25T19:17:29.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.330+0000 7f207eb06640 1 -- 172.21.15.76:0/3201917124 wait complete. 2024-04-25T19:17:29.348 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T19:17:29.348 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-25T19:17:29.348 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-25T19:17:29.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.474+0000 7fcf8b172640 1 Processor -- start 2024-04-25T19:17:29.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.474+0000 7fcf8b172640 1 -- start start 2024-04-25T19:17:29.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.476+0000 7fcf8b172640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcf840ca750 0x7fcf840cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.476+0000 7fcf8b172640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf840c56a0 con 0x7fcf840cab60 2024-04-25T19:17:29.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.476+0000 7fcf8b172640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf840c5960 con 0x7fcf84158060 2024-04-25T19:17:29.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.476+0000 7fcf8b172640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fcf840c5c40 con 0x7fcf840ca750 2024-04-25T19:17:29.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.476+0000 7fcf8970f640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fcf840cab60 0x7fcf8415af80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48712/0 (socket says 172.21.15.76:48712) 2024-04-25T19:17:29.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.476+0000 7fcf8970f640 1 -- 172.21.15.76:0/3011507403 learned_addr learned my addr 172.21.15.76:0/3011507403 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:17:29.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.477+0000 7fcf83fff640 1 --2- 172.21.15.76:0/3011507403 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcf840ca750 0x7fcf840cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.477+0000 7fcf837fe640 1 -- 172.21.15.76:0/3011507403 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3414772011 0 0) 0x7fcf840c56a0 con 0x7fcf840cab60 2024-04-25T19:17:29.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.477+0000 7fcf837fe640 1 -- 172.21.15.76:0/3011507403 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf74003520 con 0x7fcf840cab60 2024-04-25T19:17:29.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.477+0000 7fcf837fe640 1 -- 172.21.15.76:0/3011507403 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3945997262 0 0) 0x7fcf840c5960 con 0x7fcf84158060 2024-04-25T19:17:29.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.477+0000 7fcf837fe640 1 -- 172.21.15.76:0/3011507403 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf840c56a0 con 0x7fcf84158060 2024-04-25T19:17:29.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.477+0000 7fcf83fff640 1 -- 172.21.15.76:0/3011507403 >> v1:172.21.15.105:6789/0 conn(0x7fcf84158060 legacy=0x7fcf840c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.477+0000 7fcf83fff640 1 -- 172.21.15.76:0/3011507403 >> v1:172.21.15.76:6789/0 conn(0x7fcf840cab60 legacy=0x7fcf8415af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.477+0000 7fcf83fff640 1 -- 172.21.15.76:0/3011507403 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf840c5f20 con 0x7fcf840ca750 2024-04-25T19:17:29.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.478+0000 7fcf83fff640 1 --2- 172.21.15.76:0/3011507403 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcf840ca750 0x7fcf840cc7e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcf700097d0 tx=0x7fcf7002a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13fd2ca3d5072990 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:29.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.478+0000 7fcf837fe640 1 -- 172.21.15.76:0/3011507403 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf70037070 con 0x7fcf840ca750 2024-04-25T19:17:29.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.478+0000 7fcf837fe640 1 -- 172.21.15.76:0/3011507403 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf70032440 con 0x7fcf840ca750 2024-04-25T19:17:29.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.478+0000 7fcf837fe640 1 -- 172.21.15.76:0/3011507403 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf700327f0 con 0x7fcf840ca750 2024-04-25T19:17:29.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.479+0000 7fcf8b172640 1 -- 172.21.15.76:0/3011507403 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcf840ca750 msgr2=0x7fcf840cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.479+0000 7fcf8b172640 1 --2- 172.21.15.76:0/3011507403 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcf840ca750 0x7fcf840cc7e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcf700097d0 tx=0x7fcf7002a290 comp rx=0 tx=0).stop 2024-04-25T19:17:29.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.479+0000 7fcf8b172640 1 -- 172.21.15.76:0/3011507403 shutdown_connections 2024-04-25T19:17:29.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.479+0000 7fcf8b172640 1 --2- 172.21.15.76:0/3011507403 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcf840ca750 0x7fcf840cc7e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.479+0000 7fcf8b172640 1 -- 172.21.15.76:0/3011507403 >> 172.21.15.76:0/3011507403 conn(0x7fcf84068b10 msgr2=0x7fcf84072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:29.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.479+0000 7fcf8b172640 1 -- 172.21.15.76:0/3011507403 shutdown_connections 2024-04-25T19:17:29.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.479+0000 7fcf8b172640 1 -- 172.21.15.76:0/3011507403 wait complete. 2024-04-25T19:17:29.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.481+0000 7fcf8b172640 1 Processor -- start 2024-04-25T19:17:29.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.481+0000 7fcf8b172640 1 -- start start 2024-04-25T19:17:29.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.482+0000 7fcf8b172640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcf840ca750 0x7fcf841f0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.482+0000 7fcf8b172640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf840cab60 0x7fcf840d0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.482+0000 7fcf8b172640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf84158060 0x7fcf84159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.482+0000 7fcf8b172640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8415b690 con 0x7fcf84158060 2024-04-25T19:17:29.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.482+0000 7fcf8b172640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcf840c6200 con 0x7fcf840cab60 2024-04-25T19:17:29.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.482+0000 7fcf8b172640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fcf8415b810 con 0x7fcf840ca750 2024-04-25T19:17:29.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.482+0000 7fcf8970f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf84158060 0x7fcf84159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.482+0000 7fcf88f0e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcf840ca750 0x7fcf841f0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.482+0000 7fcf83fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf840cab60 0x7fcf840d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.482+0000 7fcf8970f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf84158060 0x7fcf84159910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52964/0 (socket says 172.21.15.76:52964) 2024-04-25T19:17:29.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.482+0000 7fcf88f0e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcf840ca750 0x7fcf841f0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:43844/0 (socket says 172.21.15.76:43844) 2024-04-25T19:17:29.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.482+0000 7fcf8970f640 1 -- 172.21.15.76:0/2678568205 learned_addr learned my addr 172.21.15.76:0/2678568205 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:17:29.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.483+0000 7fcf88f0e640 1 -- 172.21.15.76:0/2678568205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf840cab60 msgr2=0x7fcf840d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.483+0000 7fcf88f0e640 1 --2- 172.21.15.76:0/2678568205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf840cab60 0x7fcf840d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.483+0000 7fcf88f0e640 1 -- 172.21.15.76:0/2678568205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf84158060 msgr2=0x7fcf84159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.483+0000 7fcf88f0e640 1 --2- 172.21.15.76:0/2678568205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf84158060 0x7fcf84159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.483+0000 7fcf88f0e640 1 -- 172.21.15.76:0/2678568205 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf840d08a0 con 0x7fcf840ca750 2024-04-25T19:17:29.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.483+0000 7fcf88f0e640 1 --2- 172.21.15.76:0/2678568205 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcf840ca750 0x7fcf841f0b80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800c3f0 tx=0x7fcf7800c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:29.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.483+0000 7fcf817fa640 1 -- 172.21.15.76:0/2678568205 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78017070 con 0x7fcf840ca750 2024-04-25T19:17:29.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.483+0000 7fcf817fa640 1 -- 172.21.15.76:0/2678568205 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf780052f0 con 0x7fcf840ca750 2024-04-25T19:17:29.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.483+0000 7fcf8b172640 1 -- 172.21.15.76:0/2678568205 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf841f4090 con 0x7fcf840ca750 2024-04-25T19:17:29.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.483+0000 7fcf817fa640 1 -- 172.21.15.76:0/2678568205 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78003830 con 0x7fcf840ca750 2024-04-25T19:17:29.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.483+0000 7fcf8b172640 1 -- 172.21.15.76:0/2678568205 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf841f4650 con 0x7fcf840ca750 2024-04-25T19:17:29.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.484+0000 7fcf8b172640 1 -- 172.21.15.76:0/2678568205 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf8406a390 con 0x7fcf840ca750 2024-04-25T19:17:29.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.485+0000 7fcf817fa640 1 -- 172.21.15.76:0/2678568205 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcf7801a070 con 0x7fcf840ca750 2024-04-25T19:17:29.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.486+0000 7fcf817fa640 1 --2- 172.21.15.76:0/2678568205 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fcf680744b0 0x7fcf680769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:29.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.486+0000 7fcf817fa640 1 -- 172.21.15.76:0/2678568205 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fcf780945d0 con 0x7fcf840ca750 2024-04-25T19:17:29.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.486+0000 7fcf83fff640 1 --2- 172.21.15.76:0/2678568205 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fcf680744b0 0x7fcf680769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:29.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.486+0000 7fcf83fff640 1 --2- 172.21.15.76:0/2678568205 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fcf680744b0 0x7fcf680769c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcf70034000 tx=0x7fcf7002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:29.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.490+0000 7fcf817fa640 1 -- 172.21.15.76:0/2678568205 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcf7805e5f0 con 0x7fcf840ca750 2024-04-25T19:17:29.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.837+0000 7fcf8b172640 1 -- 172.21.15.76:0/2678568205 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fcf841571b0 con 0x7fcf840ca750 2024-04-25T19:17:29.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.839+0000 7fcf817fa640 1 -- 172.21.15.76:0/2678568205 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fcf780628f0 con 0x7fcf840ca750 2024-04-25T19:17:29.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.841+0000 7fcf8b172640 1 -- 172.21.15.76:0/2678568205 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fcf680744b0 msgr2=0x7fcf680769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.841+0000 7fcf8b172640 1 --2- 172.21.15.76:0/2678568205 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fcf680744b0 0x7fcf680769c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcf70034000 tx=0x7fcf7002e000 comp rx=0 tx=0).stop 2024-04-25T19:17:29.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.841+0000 7fcf8b172640 1 -- 172.21.15.76:0/2678568205 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcf840ca750 msgr2=0x7fcf841f0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:29.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.841+0000 7fcf8b172640 1 --2- 172.21.15.76:0/2678568205 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcf840ca750 0x7fcf841f0b80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800c3f0 tx=0x7fcf7800c8b0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.841+0000 7fcf8b172640 1 -- 172.21.15.76:0/2678568205 shutdown_connections 2024-04-25T19:17:29.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.841+0000 7fcf8b172640 1 --2- 172.21.15.76:0/2678568205 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fcf680744b0 0x7fcf680769c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.841+0000 7fcf8b172640 1 --2- 172.21.15.76:0/2678568205 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf84158060 0x7fcf84159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.841+0000 7fcf8b172640 1 --2- 172.21.15.76:0/2678568205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf840cab60 0x7fcf840d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.841+0000 7fcf8b172640 1 --2- 172.21.15.76:0/2678568205 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcf840ca750 0x7fcf841f0b80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:29.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.841+0000 7fcf8b172640 1 -- 172.21.15.76:0/2678568205 >> 172.21.15.76:0/2678568205 conn(0x7fcf84068b10 msgr2=0x7fcf841557d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:29.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.841+0000 7fcf8b172640 1 -- 172.21.15.76:0/2678568205 shutdown_connections 2024-04-25T19:17:29.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:29.841+0000 7fcf8b172640 1 -- 172.21.15.76:0/2678568205 wait complete. 2024-04-25T19:17:29.859 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-04-25T19:17:36.153 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:17:36.154 INFO:tasks.workunit.client.0.smithi076.stdout: Children see throughput for 1 initial writers = 109242.18 kB/sec 2024-04-25T19:17:36.154 INFO:tasks.workunit.client.0.smithi076.stdout: Parent sees throughput for 1 initial writers = 109239.97 kB/sec 2024-04-25T19:17:36.154 INFO:tasks.workunit.client.0.smithi076.stdout: Min throughput per process = 109242.18 kB/sec 2024-04-25T19:17:36.154 INFO:tasks.workunit.client.0.smithi076.stdout: Max throughput per process = 109242.18 kB/sec 2024-04-25T19:17:36.154 INFO:tasks.workunit.client.0.smithi076.stdout: Avg throughput per process = 109242.18 kB/sec 2024-04-25T19:17:36.154 INFO:tasks.workunit.client.0.smithi076.stdout: Min xfer = 1048576.00 kB 2024-04-25T19:17:39.860 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T19:17:39.860 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:17:39.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.981+0000 7fe6a6a6e640 1 Processor -- start 2024-04-25T19:17:39.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.981+0000 7fe6a6a6e640 1 -- start start 2024-04-25T19:17:39.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.982+0000 7fe6a6a6e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe6a00d0810 0x7fe6a006f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:39.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.982+0000 7fe6a6a6e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe6a00cea70 0x7fe6a00cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:39.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.982+0000 7fe6a6a6e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6a015ce80 con 0x7fe6a006e160 2024-04-25T19:17:39.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.982+0000 7fe6a6a6e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe6a015d160 con 0x7fe6a00cea70 2024-04-25T19:17:39.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.982+0000 7fe6a6a6e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fe6a015d440 con 0x7fe6a00d0810 2024-04-25T19:17:39.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.982+0000 7fe69ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe6a00cea70 0x7fe6a00cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:39.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.982+0000 7fe6a500b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe6a006e160 0x7fe6a006e570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59010/0 (socket says 172.21.15.76:59010) 2024-04-25T19:17:39.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.982+0000 7fe6a500b640 1 -- 172.21.15.76:0/1284015694 learned_addr learned my addr 172.21.15.76:0/1284015694 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:17:39.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.982+0000 7fe6a480a640 1 --2- 172.21.15.76:0/1284015694 >> v2:172.21.15.105:3300/0 conn(0x7fe6a00d0810 0x7fe6a006f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:39.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.983+0000 7fe69ffff640 1 -- 172.21.15.76:0/1284015694 >> v2:172.21.15.105:3300/0 conn(0x7fe6a00d0810 msgr2=0x7fe6a006f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:39.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.983+0000 7fe69ffff640 1 --2- 172.21.15.76:0/1284015694 >> v2:172.21.15.105:3300/0 conn(0x7fe6a00d0810 0x7fe6a006f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:39.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.983+0000 7fe69ffff640 1 -- 172.21.15.76:0/1284015694 >> v1:172.21.15.76:6789/0 conn(0x7fe6a006e160 legacy=0x7fe6a006e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:39.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.983+0000 7fe69f7fe640 1 -- 172.21.15.76:0/1284015694 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 485336119 0 0) 0x7fe6a015ce80 con 0x7fe6a006e160 2024-04-25T19:17:39.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.983+0000 7fe69ffff640 1 -- 172.21.15.76:0/1284015694 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a015d720 con 0x7fe6a00cea70 2024-04-25T19:17:39.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.983+0000 7fe6a480a640 1 --2- 172.21.15.76:0/1284015694 >> v2:172.21.15.105:3300/0 conn(0x7fe6a00d0810 0x7fe6a006f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:17:39.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.983+0000 7fe69ffff640 1 --2- 172.21.15.76:0/1284015694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6a00cea70 0x7fe6a00cee80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe6a008fb10 tx=0x7fe69402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91535da3ab3e03c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:39.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.983+0000 7fe69f7fe640 1 -- 172.21.15.76:0/1284015694 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe694037070 con 0x7fe6a00cea70 2024-04-25T19:17:39.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.983+0000 7fe69f7fe640 1 -- 172.21.15.76:0/1284015694 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe694032440 con 0x7fe6a00cea70 2024-04-25T19:17:39.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.984+0000 7fe69f7fe640 1 -- 172.21.15.76:0/1284015694 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe694032810 con 0x7fe6a00cea70 2024-04-25T19:17:39.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.984+0000 7fe6a6a6e640 1 -- 172.21.15.76:0/1284015694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6a00cea70 msgr2=0x7fe6a00cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:39.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.984+0000 7fe6a6a6e640 1 --2- 172.21.15.76:0/1284015694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6a00cea70 0x7fe6a00cee80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe6a008fb10 tx=0x7fe69402a1a0 comp rx=0 tx=0).stop 2024-04-25T19:17:39.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.984+0000 7fe6a6a6e640 1 -- 172.21.15.76:0/1284015694 shutdown_connections 2024-04-25T19:17:39.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.984+0000 7fe6a6a6e640 1 --2- 172.21.15.76:0/1284015694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6a00cea70 0x7fe6a00cee80 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:39.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.984+0000 7fe6a6a6e640 1 --2- 172.21.15.76:0/1284015694 >> v2:172.21.15.105:3300/0 conn(0x7fe6a00d0810 0x7fe6a006f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:39.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.984+0000 7fe6a6a6e640 1 -- 172.21.15.76:0/1284015694 >> 172.21.15.76:0/1284015694 conn(0x7fe6a00b3c00 msgr2=0x7fe6a00b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:39.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.984+0000 7fe6a6a6e640 1 -- 172.21.15.76:0/1284015694 shutdown_connections 2024-04-25T19:17:39.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.984+0000 7fe6a6a6e640 1 -- 172.21.15.76:0/1284015694 wait complete. 2024-04-25T19:17:39.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.986+0000 7fe6a6a6e640 1 Processor -- start 2024-04-25T19:17:39.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.986+0000 7fe6a6a6e640 1 -- start start 2024-04-25T19:17:39.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.987+0000 7fe6a6a6e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6a006e160 0x7fe6a01f2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:39.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.987+0000 7fe6a6a6e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe6a00cea70 0x7fe6a00b74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:39.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.987+0000 7fe6a6a6e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6a00d0810 0x7fe6a01602f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:39.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.987+0000 7fe6a6a6e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a015cb70 con 0x7fe6a006e160 2024-04-25T19:17:39.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.987+0000 7fe6a6a6e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a006dc40 con 0x7fe6a00d0810 2024-04-25T19:17:39.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.987+0000 7fe6a6a6e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fe6a006f540 con 0x7fe6a00cea70 2024-04-25T19:17:39.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.987+0000 7fe6a480a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6a006e160 0x7fe6a01f2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:39.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.988+0000 7fe6a480a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6a006e160 0x7fe6a01f2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59704/0 (socket says 172.21.15.76:59704) 2024-04-25T19:17:39.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.988+0000 7fe6a480a640 1 -- 172.21.15.76:0/3776515038 learned_addr learned my addr 172.21.15.76:0/3776515038 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:17:39.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.988+0000 7fe6a500b640 1 --2- 172.21.15.76:0/3776515038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6a00d0810 0x7fe6a01602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:39.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.988+0000 7fe69ffff640 1 --2- 172.21.15.76:0/3776515038 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe6a00cea70 0x7fe6a00b74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:39.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.988+0000 7fe6a480a640 1 -- 172.21.15.76:0/3776515038 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe6a00cea70 msgr2=0x7fe6a00b74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:39.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.988+0000 7fe6a480a640 1 --2- 172.21.15.76:0/3776515038 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe6a00cea70 0x7fe6a00b74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:39.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.988+0000 7fe6a480a640 1 -- 172.21.15.76:0/3776515038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6a00d0810 msgr2=0x7fe6a01602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:39.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.988+0000 7fe6a480a640 1 --2- 172.21.15.76:0/3776515038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6a00d0810 0x7fe6a01602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:39.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.988+0000 7fe6a480a640 1 -- 172.21.15.76:0/3776515038 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a01f3310 con 0x7fe6a006e160 2024-04-25T19:17:39.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.988+0000 7fe69ffff640 1 --2- 172.21.15.76:0/3776515038 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe6a00cea70 0x7fe6a00b74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:17:39.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.988+0000 7fe6a500b640 1 --2- 172.21.15.76:0/3776515038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6a00d0810 0x7fe6a01602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:17:39.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.988+0000 7fe6a480a640 1 --2- 172.21.15.76:0/3776515038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6a006e160 0x7fe6a01f2dd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe69000d800 tx=0x7fe69000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:39.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.988+0000 7fe69d7fa640 1 -- 172.21.15.76:0/3776515038 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690004030 con 0x7fe6a006e160 2024-04-25T19:17:39.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.988+0000 7fe69d7fa640 1 -- 172.21.15.76:0/3776515038 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6900041d0 con 0x7fe6a006e160 2024-04-25T19:17:39.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.988+0000 7fe6a6a6e640 1 -- 172.21.15.76:0/3776515038 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a00cb410 con 0x7fe6a006e160 2024-04-25T19:17:39.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.988+0000 7fe69d7fa640 1 -- 172.21.15.76:0/3776515038 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6900096a0 con 0x7fe6a006e160 2024-04-25T19:17:39.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.988+0000 7fe6a6a6e640 1 -- 172.21.15.76:0/3776515038 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a01f6600 con 0x7fe6a006e160 2024-04-25T19:17:39.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.989+0000 7fe6a6a6e640 1 -- 172.21.15.76:0/3776515038 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6a006a390 con 0x7fe6a006e160 2024-04-25T19:17:39.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.990+0000 7fe69d7fa640 1 -- 172.21.15.76:0/3776515038 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe690009840 con 0x7fe6a006e160 2024-04-25T19:17:39.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.990+0000 7fe69d7fa640 1 --2- 172.21.15.76:0/3776515038 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe68c0744b0 0x7fe68c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:39.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.991+0000 7fe69d7fa640 1 -- 172.21.15.76:0/3776515038 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fe690095430 con 0x7fe6a006e160 2024-04-25T19:17:39.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.991+0000 7fe69ffff640 1 --2- 172.21.15.76:0/3776515038 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe68c0744b0 0x7fe68c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:39.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.991+0000 7fe69ffff640 1 --2- 172.21.15.76:0/3776515038 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe68c0744b0 0x7fe68c0769c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe694034000 tx=0x7fe69402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:39.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:39.995+0000 7fe69d7fa640 1 -- 172.21.15.76:0/3776515038 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe69005fc60 con 0x7fe6a006e160 2024-04-25T19:17:40.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.240+0000 7fe6a6a6e640 1 -- 172.21.15.76:0/3776515038 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe6a01f2a30 con 0x7fe6a006e160 2024-04-25T19:17:40.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.240+0000 7fe69d7fa640 1 -- 172.21.15.76:0/3776515038 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7fe69005f560 con 0x7fe6a006e160 2024-04-25T19:17:40.242 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:17:40.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.242+0000 7fe6a6a6e640 1 -- 172.21.15.76:0/3776515038 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe68c0744b0 msgr2=0x7fe68c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.242+0000 7fe6a6a6e640 1 --2- 172.21.15.76:0/3776515038 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe68c0744b0 0x7fe68c0769c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe694034000 tx=0x7fe69402e000 comp rx=0 tx=0).stop 2024-04-25T19:17:40.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.242+0000 7fe6a6a6e640 1 -- 172.21.15.76:0/3776515038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6a006e160 msgr2=0x7fe6a01f2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.242+0000 7fe6a6a6e640 1 --2- 172.21.15.76:0/3776515038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6a006e160 0x7fe6a01f2dd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe69000d800 tx=0x7fe69000dcc0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.242+0000 7fe6a6a6e640 1 -- 172.21.15.76:0/3776515038 shutdown_connections 2024-04-25T19:17:40.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.242+0000 7fe6a6a6e640 1 --2- 172.21.15.76:0/3776515038 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe68c0744b0 0x7fe68c0769c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.242+0000 7fe6a6a6e640 1 --2- 172.21.15.76:0/3776515038 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe6a00d0810 0x7fe6a01602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.242+0000 7fe6a6a6e640 1 --2- 172.21.15.76:0/3776515038 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe6a00cea70 0x7fe6a00b74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.242+0000 7fe6a6a6e640 1 --2- 172.21.15.76:0/3776515038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6a006e160 0x7fe6a01f2dd0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.242+0000 7fe6a6a6e640 1 -- 172.21.15.76:0/3776515038 >> 172.21.15.76:0/3776515038 conn(0x7fe6a00b3c00 msgr2=0x7fe6a00d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:40.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.242+0000 7fe6a6a6e640 1 -- 172.21.15.76:0/3776515038 shutdown_connections 2024-04-25T19:17:40.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.242+0000 7fe6a6a6e640 1 -- 172.21.15.76:0/3776515038 wait complete. 2024-04-25T19:17:40.255 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T19:17:40.255 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-25T19:17:40.256 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T19:17:40.256 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T19:17:40.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.364+0000 7faac15ce640 1 Processor -- start 2024-04-25T19:17:40.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.364+0000 7faac15ce640 1 -- start start 2024-04-25T19:17:40.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.365+0000 7faac15ce640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faabc157150 0x7faabc0cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.365+0000 7faac15ce640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faabc0cdd10 0x7faabc06d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.365+0000 7faac15ce640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faabc06e680 con 0x7faabc0cd6f0 2024-04-25T19:17:40.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.365+0000 7faac15ce640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7faabc15d5a0 con 0x7faabc157150 2024-04-25T19:17:40.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.365+0000 7faac15ce640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7faabc06d1a0 con 0x7faabc0cdd10 2024-04-25T19:17:40.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.365+0000 7faabaffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faabc157150 0x7faabc0cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.365+0000 7faabb7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7faabc0cd6f0 0x7faabc06dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59026/0 (socket says 172.21.15.76:59026) 2024-04-25T19:17:40.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.365+0000 7faabaffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7faabc157150 0x7faabc0cfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59710/0 (socket says 172.21.15.76:59710) 2024-04-25T19:17:40.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.365+0000 7faabb7fe640 1 -- 172.21.15.76:0/2121726410 learned_addr learned my addr 172.21.15.76:0/2121726410 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:17:40.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.366+0000 7faab3fff640 1 --2- 172.21.15.76:0/2121726410 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faabc0cdd10 0x7faabc06d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.366+0000 7faabaffd640 1 -- 172.21.15.76:0/2121726410 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faabc0cdd10 msgr2=0x7faabc06d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:40.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.366+0000 7faabaffd640 1 --2- 172.21.15.76:0/2121726410 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faabc0cdd10 0x7faabc06d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.366+0000 7faabaffd640 1 -- 172.21.15.76:0/2121726410 >> v1:172.21.15.76:6789/0 conn(0x7faabc0cd6f0 legacy=0x7faabc06dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.366+0000 7faabaffd640 1 -- 172.21.15.76:0/2121726410 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faabc06d440 con 0x7faabc157150 2024-04-25T19:17:40.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.366+0000 7faaba7fc640 1 -- 172.21.15.76:0/2121726410 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2986170236 0 0) 0x7faabc06e680 con 0x7faabc0cd6f0 2024-04-25T19:17:40.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.366+0000 7faab3fff640 1 --2- 172.21.15.76:0/2121726410 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faabc0cdd10 0x7faabc06d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:17:40.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.366+0000 7faabaffd640 1 --2- 172.21.15.76:0/2121726410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faabc157150 0x7faabc0cfd30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7faaac0097a0 tx=0x7faaac02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8a8b9dc37efa26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:40.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.367+0000 7faaba7fc640 1 -- 172.21.15.76:0/2121726410 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac037070 con 0x7faabc157150 2024-04-25T19:17:40.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.367+0000 7faaba7fc640 1 -- 172.21.15.76:0/2121726410 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaac032440 con 0x7faabc157150 2024-04-25T19:17:40.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.367+0000 7faaba7fc640 1 -- 172.21.15.76:0/2121726410 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac032810 con 0x7faabc157150 2024-04-25T19:17:40.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.367+0000 7faac15ce640 1 -- 172.21.15.76:0/2121726410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faabc157150 msgr2=0x7faabc0cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.367+0000 7faac15ce640 1 --2- 172.21.15.76:0/2121726410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faabc157150 0x7faabc0cfd30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7faaac0097a0 tx=0x7faaac02a210 comp rx=0 tx=0).stop 2024-04-25T19:17:40.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.367+0000 7faac15ce640 1 -- 172.21.15.76:0/2121726410 shutdown_connections 2024-04-25T19:17:40.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.367+0000 7faac15ce640 1 --2- 172.21.15.76:0/2121726410 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faabc0cdd10 0x7faabc06d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.367+0000 7faac15ce640 1 --2- 172.21.15.76:0/2121726410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faabc157150 0x7faabc0cfd30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.367+0000 7faac15ce640 1 -- 172.21.15.76:0/2121726410 >> 172.21.15.76:0/2121726410 conn(0x7faabc0d1f30 msgr2=0x7faabc0d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:40.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.367+0000 7faac15ce640 1 -- 172.21.15.76:0/2121726410 shutdown_connections 2024-04-25T19:17:40.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.367+0000 7faac15ce640 1 -- 172.21.15.76:0/2121726410 wait complete. 2024-04-25T19:17:40.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.369+0000 7faac15ce640 1 Processor -- start 2024-04-25T19:17:40.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.369+0000 7faac15ce640 1 -- start start 2024-04-25T19:17:40.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.370+0000 7faac15ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faabc0cd6f0 0x7faabc157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.370+0000 7faac15ce640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faabc0cdd10 0x7faabc0b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.370+0000 7faac15ce640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faabc1f5b00 0x7faabc0b74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.370+0000 7faac15ce640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faabc06e3a0 con 0x7faabc0cdd10 2024-04-25T19:17:40.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.370+0000 7faac15ce640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faabc15d230 con 0x7faabc0cd6f0 2024-04-25T19:17:40.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.370+0000 7faac15ce640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7faabc15cc00 con 0x7faabc1f5b00 2024-04-25T19:17:40.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.370+0000 7faab3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faabc0cdd10 0x7faabc0b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.370+0000 7faab3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faabc0cdd10 0x7faabc0b5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59720/0 (socket says 172.21.15.76:59720) 2024-04-25T19:17:40.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.370+0000 7faab3fff640 1 -- 172.21.15.76:0/271765187 learned_addr learned my addr 172.21.15.76:0/271765187 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:17:40.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.371+0000 7faabb7fe640 1 --2- 172.21.15.76:0/271765187 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faabc1f5b00 0x7faabc0b74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.371+0000 7faabaffd640 1 --2- 172.21.15.76:0/271765187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faabc0cd6f0 0x7faabc157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.371+0000 7faab3fff640 1 -- 172.21.15.76:0/271765187 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faabc1f5b00 msgr2=0x7faabc0b74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.371+0000 7faab3fff640 1 --2- 172.21.15.76:0/271765187 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7faabc1f5b00 0x7faabc0b74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.371+0000 7faab3fff640 1 -- 172.21.15.76:0/271765187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faabc0cd6f0 msgr2=0x7faabc157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.371+0000 7faab3fff640 1 --2- 172.21.15.76:0/271765187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faabc0cd6f0 0x7faabc157600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.371+0000 7faab3fff640 1 -- 172.21.15.76:0/271765187 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faabc0b63c0 con 0x7faabc0cdd10 2024-04-25T19:17:40.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.371+0000 7faabaffd640 1 --2- 172.21.15.76:0/271765187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faabc0cd6f0 0x7faabc157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:17:40.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.371+0000 7faab3fff640 1 --2- 172.21.15.76:0/271765187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faabc0cdd10 0x7faabc0b5e80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7faaa400c7f0 tx=0x7faaa400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:40.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.371+0000 7faab37fe640 1 -- 172.21.15.76:0/271765187 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4015070 con 0x7faabc0cdd10 2024-04-25T19:17:40.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.371+0000 7faab37fe640 1 -- 172.21.15.76:0/271765187 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaa4007820 con 0x7faabc0cdd10 2024-04-25T19:17:40.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.371+0000 7faac15ce640 1 -- 172.21.15.76:0/271765187 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faabc0b65d0 con 0x7faabc0cdd10 2024-04-25T19:17:40.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.371+0000 7faab37fe640 1 -- 172.21.15.76:0/271765187 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4007bd0 con 0x7faabc0cdd10 2024-04-25T19:17:40.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.371+0000 7faac15ce640 1 -- 172.21.15.76:0/271765187 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faabc0b50c0 con 0x7faabc0cdd10 2024-04-25T19:17:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.372+0000 7faac15ce640 1 --2- 172.21.15.76:0/271765187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faabc0c84d0 0x7faabc0b5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.372+0000 7faac15ce640 1 -- 172.21.15.76:0/271765187 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faabc0b4bb0 con 0x7faabc0c84d0 2024-04-25T19:17:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.372+0000 7faabaffd640 1 --2- 172.21.15.76:0/271765187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faabc0c84d0 0x7faabc0b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.372+0000 7faabaffd640 1 -- 172.21.15.76:0/271765187 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faabc06a390 con 0x7faabc0c84d0 2024-04-25T19:17:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.373+0000 7faabaffd640 1 --2- 172.21.15.76:0/271765187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faabc0c84d0 0x7faabc0b5290 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7faabc08f420 tx=0x7faaac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.373+0000 7faab37fe640 1 -- 172.21.15.76:0/271765187 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac042020 con 0x7faabc0c84d0 2024-04-25T19:17:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.373+0000 7faab37fe640 1 -- 172.21.15.76:0/271765187 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faaa4018070 con 0x7faabc0cdd10 2024-04-25T19:17:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.373+0000 7faab37fe640 1 --2- 172.21.15.76:0/271765187 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7faa9c0743b0 0x7faa9c0768c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.373+0000 7faab37fe640 1 -- 172.21.15.76:0/271765187 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7faaa4095360 con 0x7faabc0cdd10 2024-04-25T19:17:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.373+0000 7faab37fe640 1 -- 172.21.15.76:0/271765187 <== mon.0 v2:172.21.15.76:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7faaac038040 con 0x7faabc0c84d0 2024-04-25T19:17:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.374+0000 7faab37fe640 1 -- 172.21.15.76:0/271765187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faabc0c84d0 msgr2=0x7faabc0b5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.374+0000 7faab37fe640 1 --2- 172.21.15.76:0/271765187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faabc0c84d0 0x7faabc0b5290 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7faabc08f420 tx=0x7faaac0046c0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.374+0000 7faab37fe640 1 -- 172.21.15.76:0/271765187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faabc0c84d0 msgr2=0x7faabc0b5290 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:17:40.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.374+0000 7faab37fe640 1 --2- 172.21.15.76:0/271765187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faabc0c84d0 0x7faabc0b5290 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.374+0000 7faabb7fe640 1 --2- 172.21.15.76:0/271765187 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7faa9c0743b0 0x7faa9c0768c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.374+0000 7faabb7fe640 1 --2- 172.21.15.76:0/271765187 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7faa9c0743b0 0x7faa9c0768c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faab4007fd0 tx=0x7faab4005b70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:40.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.380+0000 7faac15ce640 1 --2- 172.21.15.76:0/271765187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faabc0b57d0 0x7faabc155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.380+0000 7faac15ce640 1 -- 172.21.15.76:0/271765187 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faabc0cd3d0 con 0x7faabc0b57d0 2024-04-25T19:17:40.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.380+0000 7faabaffd640 1 --2- 172.21.15.76:0/271765187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faabc0b57d0 0x7faabc155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.381+0000 7faabaffd640 1 -- 172.21.15.76:0/271765187 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7faabc0ce370 con 0x7faabc0b57d0 2024-04-25T19:17:40.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.381+0000 7faabaffd640 1 --2- 172.21.15.76:0/271765187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faabc0b57d0 0x7faabc155d30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7faaac0097a0 tx=0x7faaac008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:40.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.381+0000 7faab37fe640 1 -- 172.21.15.76:0/271765187 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac037050 con 0x7faabc0b57d0 2024-04-25T19:17:40.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.381+0000 7faab37fe640 1 -- 172.21.15.76:0/271765187 <== mon.0 v2:172.21.15.76:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5068 (secure 0 0 0) 0x7faaac050420 con 0x7faabc0b57d0 2024-04-25T19:17:40.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.381+0000 7faab37fe640 1 -- 172.21.15.76:0/271765187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faabc0b57d0 msgr2=0x7faabc155d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.381+0000 7faab37fe640 1 --2- 172.21.15.76:0/271765187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faabc0b57d0 0x7faabc155d30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7faaac0097a0 tx=0x7faaac008030 comp rx=0 tx=0).stop 2024-04-25T19:17:40.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.381+0000 7faab37fe640 1 -- 172.21.15.76:0/271765187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faabc0b57d0 msgr2=0x7faabc155d30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:17:40.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.381+0000 7faab37fe640 1 --2- 172.21.15.76:0/271765187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faabc0b57d0 0x7faabc155d30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.383 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:17:40.385 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:17:40.385 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:17:40.385 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "leader", 2024-04-25T19:17:40.385 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 16, 2024-04-25T19:17:40.385 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 158907, 2024-04-25T19:17:40.385 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:17:40.385 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:17:40.386 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-04-25T19:17:40.386 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:17:40.386 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:40.386 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 11, 2024-04-25T19:17:40.386 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:17:40.386 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:17:40.386 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:17:40.386 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:17:40.386 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:17:40.386 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:17:40.386 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:17:40.386 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:17:40.386 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:17:40.386 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:17:40.387 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:17:40.387 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:17:40.387 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:17:40.387 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:17:40.387 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:40.387 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:17:40.387 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:17:40.387 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:17:40.387 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:17:40.387 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:17:40.387 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:17:40.387 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:17:40.387 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:17:40.387 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:17:40.387 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:17:40.387 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:17:40.388 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:17:40.388 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:17:40.388 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:40.388 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.388 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:17:40.388 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:17:40.388 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:17:40.388 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:17:40.388 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:17:40.388 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:17:40.388 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:17:40.388 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:17:40.388 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:17:40.388 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:17:40.388 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:17:40.389 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:17:40.389 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:17:40.389 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:17:40.389 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:17:40.389 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:17:40.389 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:17:40.389 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:17:40.389 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:17:40.389 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:17:40.389 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:17:40.389 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:17:40.389 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:17:40.389 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:17:40.389 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:17:40.389 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:17:40.390 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:17:40.390 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:17:40.390 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:40.390 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:17:40.390 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.390 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:17:40.390 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.390 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:17:40.390 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:17:40.390 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:17:40.390 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:17:40.390 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.390 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:17:40.390 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:17:40.391 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:40.391 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.391 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.391 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:17:40.391 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:17:40.391 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:40.391 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:40.391 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:40.391 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.391 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:17:40.391 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:17:40.391 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:17:40.391 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:17:40.391 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:17:40.391 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.391 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.392 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:17:40.392 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:17:40.392 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:17:40.392 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:17:40.392 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.392 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:17:40.392 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:17:40.392 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:40.392 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.392 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.392 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:17:40.392 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:17:40.392 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:40.392 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:40.392 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:40.392 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.393 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:17:40.393 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:17:40.393 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:17:40.393 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:17:40.393 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:17:40.393 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.393 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.393 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:17:40.393 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:17:40.393 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:17:40.393 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:17:40.393 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.393 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:17:40.393 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:17:40.394 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:40.394 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.394 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.394 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:17:40.394 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:17:40.394 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:40.394 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:40.394 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:40.394 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.394 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:17:40.394 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:17:40.394 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:17:40.395 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:17:40.395 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:17:40.395 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:40.395 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:40.395 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.395 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:17:40.395 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:17:40.395 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.395 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:40.395 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:40.395 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:17:40.395 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:40.395 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:40.395 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": [ 2024-04-25T19:17:40.395 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.395 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:40.396 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:40.396 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 6 2024-04-25T19:17:40.396 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:40.396 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:40.396 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": [ 2024-04-25T19:17:40.396 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.396 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:40.396 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:40.396 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 7 2024-04-25T19:17:40.396 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:40.396 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:40.396 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:17:40.396 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.396 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:40.396 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:40.397 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 6 2024-04-25T19:17:40.397 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:40.397 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:40.397 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": [ 2024-04-25T19:17:40.397 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.397 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:40.397 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:40.397 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 2 2024-04-25T19:17:40.397 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:40.397 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:40.397 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.397 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:17:40.397 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:17:40.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.381+0000 7faac15ce640 1 -- 172.21.15.76:0/271765187 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7faa9c0743b0 msgr2=0x7faa9c0768c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.381+0000 7faac15ce640 1 --2- 172.21.15.76:0/271765187 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7faa9c0743b0 0x7faa9c0768c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faab4007fd0 tx=0x7faab4005b70 comp rx=0 tx=0).stop 2024-04-25T19:17:40.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.381+0000 7faac15ce640 1 -- 172.21.15.76:0/271765187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faabc0cdd10 msgr2=0x7faabc0b5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.381+0000 7faac15ce640 1 --2- 172.21.15.76:0/271765187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faabc0cdd10 0x7faabc0b5e80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7faaa400c7f0 tx=0x7faaa400ccb0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.381+0000 7faabb7fe640 1 -- 172.21.15.76:0/271765187 reap_dead start 2024-04-25T19:17:40.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.382+0000 7faabb7fe640 1 -- 172.21.15.76:0/271765187 reap_dead start 2024-04-25T19:17:40.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.382+0000 7faac15ce640 1 -- 172.21.15.76:0/271765187 shutdown_connections 2024-04-25T19:17:40.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.382+0000 7faac15ce640 1 -- 172.21.15.76:0/271765187 >> 172.21.15.76:0/271765187 conn(0x7faabc0d1f30 msgr2=0x7faabc0cb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:40.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.382+0000 7faac15ce640 1 -- 172.21.15.76:0/271765187 shutdown_connections 2024-04-25T19:17:40.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.382+0000 7faac15ce640 1 -- 172.21.15.76:0/271765187 wait complete. 2024-04-25T19:17:40.400 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T19:17:40.400 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T19:17:40.400 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T19:17:40.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.507+0000 7fbfce53f640 1 Processor -- start 2024-04-25T19:17:40.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.507+0000 7fbfce53f640 1 -- start start 2024-04-25T19:17:40.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.509+0000 7fbfce53f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfc8073880 0x7fbfc80c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.509+0000 7fbfce53f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbfc80c8910 0x7fbfc815ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.509+0000 7fbfce53f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfc8071ad0 con 0x7fbfc8071520 2024-04-25T19:17:40.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.509+0000 7fbfce53f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fbfc8071d70 con 0x7fbfc80c8910 2024-04-25T19:17:40.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.509+0000 7fbfce53f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fbfc8074150 con 0x7fbfc8073880 2024-04-25T19:17:40.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.509+0000 7fbfccadc640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbfc80c8910 0x7fbfc815ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.509+0000 7fbfc77fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfc8073880 0x7fbfc80c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.509+0000 7fbfc7fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fbfc8071520 0x7fbfc8151350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.76:36630/0 (socket says 172.21.15.76:36630) 2024-04-25T19:17:40.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.509+0000 7fbfc7fff640 1 -- 172.21.15.76:0/3745932540 learned_addr learned my addr 172.21.15.76:0/3745932540 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:17:40.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.509+0000 7fbfc77fe640 1 -- 172.21.15.76:0/3745932540 >> v2:172.21.15.105:3300/0 conn(0x7fbfc80c8910 msgr2=0x7fbfc815ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:40.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.509+0000 7fbfc77fe640 1 --2- 172.21.15.76:0/3745932540 >> v2:172.21.15.105:3300/0 conn(0x7fbfc80c8910 0x7fbfc815ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.509+0000 7fbfc77fe640 1 -- 172.21.15.76:0/3745932540 >> v1:172.21.15.105:6789/0 conn(0x7fbfc8071520 legacy=0x7fbfc8151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.509+0000 7fbfc77fe640 1 -- 172.21.15.76:0/3745932540 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc8074430 con 0x7fbfc8073880 2024-04-25T19:17:40.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.509+0000 7fbfccadc640 1 --2- 172.21.15.76:0/3745932540 >> v2:172.21.15.105:3300/0 conn(0x7fbfc80c8910 0x7fbfc815ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:17:40.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.510+0000 7fbfc77fe640 1 --2- 172.21.15.76:0/3745932540 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfc8073880 0x7fbfc80c83d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0097a0 tx=0x7fbfbc02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d92db4292a8c57c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:40.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.510+0000 7fbfc6ffd640 1 -- 172.21.15.76:0/3745932540 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc037070 con 0x7fbfc8073880 2024-04-25T19:17:40.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.510+0000 7fbfc6ffd640 1 -- 172.21.15.76:0/3745932540 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfbc032440 con 0x7fbfc8073880 2024-04-25T19:17:40.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.510+0000 7fbfc6ffd640 1 -- 172.21.15.76:0/3745932540 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc032810 con 0x7fbfc8073880 2024-04-25T19:17:40.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.511+0000 7fbfce53f640 1 -- 172.21.15.76:0/3745932540 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfc8073880 msgr2=0x7fbfc80c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.511+0000 7fbfce53f640 1 --2- 172.21.15.76:0/3745932540 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfc8073880 0x7fbfc80c83d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0097a0 tx=0x7fbfbc02a210 comp rx=0 tx=0).stop 2024-04-25T19:17:40.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.511+0000 7fbfce53f640 1 -- 172.21.15.76:0/3745932540 shutdown_connections 2024-04-25T19:17:40.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.511+0000 7fbfce53f640 1 --2- 172.21.15.76:0/3745932540 >> v2:172.21.15.105:3300/0 conn(0x7fbfc80c8910 0x7fbfc815ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.511+0000 7fbfce53f640 1 --2- 172.21.15.76:0/3745932540 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfc8073880 0x7fbfc80c83d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.511+0000 7fbfce53f640 1 -- 172.21.15.76:0/3745932540 >> 172.21.15.76:0/3745932540 conn(0x7fbfc80cecc0 msgr2=0x7fbfc80cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:40.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.511+0000 7fbfce53f640 1 -- 172.21.15.76:0/3745932540 shutdown_connections 2024-04-25T19:17:40.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.511+0000 7fbfce53f640 1 -- 172.21.15.76:0/3745932540 wait complete. 2024-04-25T19:17:40.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.513+0000 7fbfce53f640 1 Processor -- start 2024-04-25T19:17:40.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.513+0000 7fbfce53f640 1 -- start start 2024-04-25T19:17:40.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.514+0000 7fbfce53f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfc8071520 0x7fbfc815a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.514+0000 7fbfce53f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc8073880 0x7fbfc81f1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.514+0000 7fbfce53f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfc80c8910 0x7fbfc814a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.514+0000 7fbfce53f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc8154fc0 con 0x7fbfc8071520 2024-04-25T19:17:40.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.514+0000 7fbfce53f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc80749f0 con 0x7fbfc8073880 2024-04-25T19:17:40.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.514+0000 7fbfce53f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fbfc80ccc80 con 0x7fbfc80c8910 2024-04-25T19:17:40.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.514+0000 7fbfc7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfc8071520 0x7fbfc815a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.514+0000 7fbfc7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfc8071520 0x7fbfc815a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59756/0 (socket says 172.21.15.76:59756) 2024-04-25T19:17:40.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.514+0000 7fbfc7fff640 1 -- 172.21.15.76:0/3833554463 learned_addr learned my addr 172.21.15.76:0/3833554463 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:17:40.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.514+0000 7fbfc77fe640 1 --2- 172.21.15.76:0/3833554463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc8073880 0x7fbfc81f1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.514+0000 7fbfccadc640 1 --2- 172.21.15.76:0/3833554463 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfc80c8910 0x7fbfc814a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.515+0000 7fbfc7fff640 1 -- 172.21.15.76:0/3833554463 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfc80c8910 msgr2=0x7fbfc814a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.515+0000 7fbfc7fff640 1 --2- 172.21.15.76:0/3833554463 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfc80c8910 0x7fbfc814a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.515+0000 7fbfc7fff640 1 -- 172.21.15.76:0/3833554463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc8073880 msgr2=0x7fbfc81f1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.515+0000 7fbfc7fff640 1 --2- 172.21.15.76:0/3833554463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc8073880 0x7fbfc81f1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.515+0000 7fbfc7fff640 1 -- 172.21.15.76:0/3833554463 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc8158330 con 0x7fbfc8071520 2024-04-25T19:17:40.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.515+0000 7fbfc7fff640 1 --2- 172.21.15.76:0/3833554463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfc8071520 0x7fbfc815a5f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800c3c0 tx=0x7fbfb800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:40.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.515+0000 7fbfc4ff9640 1 -- 172.21.15.76:0/3833554463 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8017070 con 0x7fbfc8071520 2024-04-25T19:17:40.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.515+0000 7fbfc4ff9640 1 -- 172.21.15.76:0/3833554463 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfb8005ce0 con 0x7fbfc8071520 2024-04-25T19:17:40.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.515+0000 7fbfce53f640 1 -- 172.21.15.76:0/3833554463 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc81f1d90 con 0x7fbfc8071520 2024-04-25T19:17:40.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.515+0000 7fbfc4ff9640 1 -- 172.21.15.76:0/3833554463 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb80072f0 con 0x7fbfc8071520 2024-04-25T19:17:40.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.515+0000 7fbfce53f640 1 -- 172.21.15.76:0/3833554463 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc81565e0 con 0x7fbfc8071520 2024-04-25T19:17:40.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.516+0000 7fbfce53f640 1 --2- 172.21.15.76:0/3833554463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc81568c0 0x7fbfc81f1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.516+0000 7fbfce53f640 1 -- 172.21.15.76:0/3833554463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc8156390 con 0x7fbfc81568c0 2024-04-25T19:17:40.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.516+0000 7fbfc77fe640 1 --2- 172.21.15.76:0/3833554463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc81568c0 0x7fbfc81f1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.517+0000 7fbfc77fe640 1 -- 172.21.15.76:0/3833554463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbfc8071330 con 0x7fbfc81568c0 2024-04-25T19:17:40.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.517+0000 7fbfc77fe640 1 --2- 172.21.15.76:0/3833554463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc81568c0 0x7fbfc81f1000 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80c92f0 tx=0x7fbfbc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:40.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.517+0000 7fbfc4ff9640 1 -- 172.21.15.76:0/3833554463 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbfb801a070 con 0x7fbfc8071520 2024-04-25T19:17:40.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.517+0000 7fbfc4ff9640 1 --2- 172.21.15.76:0/3833554463 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbfb40744b0 0x7fbfb40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.517+0000 7fbfc4ff9640 1 -- 172.21.15.76:0/3833554463 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fbfb80945c0 con 0x7fbfc8071520 2024-04-25T19:17:40.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.517+0000 7fbfccadc640 1 --2- 172.21.15.76:0/3833554463 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbfb40744b0 0x7fbfb40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.517+0000 7fbfc4ff9640 1 -- 172.21.15.76:0/3833554463 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc042070 con 0x7fbfc81568c0 2024-04-25T19:17:40.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.517+0000 7fbfc4ff9640 1 -- 172.21.15.76:0/3833554463 <== mon.1 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fbfbc03e660 con 0x7fbfc81568c0 2024-04-25T19:17:40.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.518+0000 7fbfc4ff9640 1 -- 172.21.15.76:0/3833554463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc81568c0 msgr2=0x7fbfc81f1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.518+0000 7fbfc4ff9640 1 --2- 172.21.15.76:0/3833554463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc81568c0 0x7fbfc81f1000 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80c92f0 tx=0x7fbfbc0046c0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.518+0000 7fbfc4ff9640 1 -- 172.21.15.76:0/3833554463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc81568c0 msgr2=0x7fbfc81f1000 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:17:40.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.518+0000 7fbfc4ff9640 1 --2- 172.21.15.76:0/3833554463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc81568c0 0x7fbfc81f1000 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.518+0000 7fbfccadc640 1 --2- 172.21.15.76:0/3833554463 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbfb40744b0 0x7fbfb40769c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009c10 tx=0x7fbfc000b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:40.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.524+0000 7fbfce53f640 1 --2- 172.21.15.76:0/3833554463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc8158fa0 0x7fbfc81593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.524+0000 7fbfce53f640 1 -- 172.21.15.76:0/3833554463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc8155140 con 0x7fbfc8158fa0 2024-04-25T19:17:40.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.524+0000 7fbfc77fe640 1 --2- 172.21.15.76:0/3833554463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc8158fa0 0x7fbfc81593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.525+0000 7fbfc77fe640 1 -- 172.21.15.76:0/3833554463 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbfc8150b30 con 0x7fbfc8158fa0 2024-04-25T19:17:40.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.525+0000 7fbfc77fe640 1 --2- 172.21.15.76:0/3833554463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc8158fa0 0x7fbfc81593b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc02a250 tx=0x7fbfbc008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:40.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.525+0000 7fbfc4ff9640 1 -- 172.21.15.76:0/3833554463 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc037040 con 0x7fbfc8158fa0 2024-04-25T19:17:40.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.525+0000 7fbfc4ff9640 1 -- 172.21.15.76:0/3833554463 <== mon.1 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4555 (secure 0 0 0) 0x7fbfbc03ee10 con 0x7fbfc8158fa0 2024-04-25T19:17:40.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.525+0000 7fbfc4ff9640 1 -- 172.21.15.76:0/3833554463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc8158fa0 msgr2=0x7fbfc81593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.525+0000 7fbfc4ff9640 1 --2- 172.21.15.76:0/3833554463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc8158fa0 0x7fbfc81593b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc02a250 tx=0x7fbfbc008030 comp rx=0 tx=0).stop 2024-04-25T19:17:40.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.525+0000 7fbfc4ff9640 1 -- 172.21.15.76:0/3833554463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc8158fa0 msgr2=0x7fbfc81593b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:17:40.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.525+0000 7fbfc4ff9640 1 --2- 172.21.15.76:0/3833554463 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfc8158fa0 0x7fbfc81593b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.528 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:17:40.528 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:17:40.528 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:17:40.528 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "peon", 2024-04-25T19:17:40.528 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 16, 2024-04-25T19:17:40.528 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 12017, 2024-04-25T19:17:40.528 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:17:40.528 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:17:40.528 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-04-25T19:17:40.528 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:17:40.529 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:40.529 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 11, 2024-04-25T19:17:40.529 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:17:40.529 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:17:40.529 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:17:40.529 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:17:40.529 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:17:40.529 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:17:40.529 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:17:40.529 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:17:40.529 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:17:40.529 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:17:40.529 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:17:40.529 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:17:40.530 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:17:40.530 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:17:40.530 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:40.530 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:17:40.530 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:17:40.530 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:17:40.530 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:17:40.530 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:17:40.530 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:17:40.530 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:17:40.530 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:17:40.530 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:17:40.530 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:17:40.531 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:17:40.531 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:17:40.531 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:17:40.531 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:40.531 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.531 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:17:40.531 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:17:40.531 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:17:40.531 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:17:40.531 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:17:40.531 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:17:40.531 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:17:40.531 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:17:40.531 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:17:40.531 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:17:40.532 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:17:40.532 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:17:40.532 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:17:40.532 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:17:40.532 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:17:40.532 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:17:40.532 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:17:40.532 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:17:40.532 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:17:40.532 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:17:40.532 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:17:40.532 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:17:40.532 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:17:40.532 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:17:40.532 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:17:40.533 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:17:40.533 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:17:40.533 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:17:40.533 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:40.533 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:17:40.533 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.533 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:17:40.533 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.533 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:17:40.533 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:17:40.533 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:17:40.533 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:17:40.533 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.533 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:17:40.533 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:17:40.534 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:40.534 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.534 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.534 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:17:40.534 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:17:40.534 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:40.534 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:40.534 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:40.534 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.534 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:17:40.534 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:17:40.534 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:17:40.534 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:17:40.534 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:17:40.534 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.535 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.535 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:17:40.535 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:17:40.535 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:17:40.535 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:17:40.535 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.535 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:17:40.535 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:17:40.535 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:40.535 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.535 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.535 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:17:40.535 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:17:40.535 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:40.535 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:40.535 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:40.536 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.536 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:17:40.536 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:17:40.536 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:17:40.536 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:17:40.536 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:17:40.536 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.536 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.536 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:17:40.536 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:17:40.536 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:17:40.536 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:17:40.536 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.536 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:17:40.537 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:17:40.537 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:40.537 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.537 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.537 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:17:40.537 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:17:40.537 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:40.537 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:40.537 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:40.537 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.537 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:17:40.537 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:17:40.537 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:17:40.538 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:17:40.538 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:17:40.538 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:40.538 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:40.538 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.538 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:17:40.538 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:17:40.538 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.538 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:40.538 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:40.538 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:17:40.538 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:40.538 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:40.538 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:17:40.539 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.539 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:40.539 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:40.539 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:17:40.539 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:40.539 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:40.539 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.539 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:17:40.539 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:17:40.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.526+0000 7fbfce53f640 1 -- 172.21.15.76:0/3833554463 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbfb40744b0 msgr2=0x7fbfb40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.526+0000 7fbfce53f640 1 --2- 172.21.15.76:0/3833554463 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbfb40744b0 0x7fbfb40769c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009c10 tx=0x7fbfc000b040 comp rx=0 tx=0).stop 2024-04-25T19:17:40.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.526+0000 7fbfce53f640 1 -- 172.21.15.76:0/3833554463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfc8071520 msgr2=0x7fbfc815a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.526+0000 7fbfce53f640 1 --2- 172.21.15.76:0/3833554463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfc8071520 0x7fbfc815a5f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800c3c0 tx=0x7fbfb800c7c0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.526+0000 7fbfccadc640 1 -- 172.21.15.76:0/3833554463 reap_dead start 2024-04-25T19:17:40.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.526+0000 7fbfccadc640 1 -- 172.21.15.76:0/3833554463 reap_dead start 2024-04-25T19:17:40.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.526+0000 7fbfce53f640 1 -- 172.21.15.76:0/3833554463 shutdown_connections 2024-04-25T19:17:40.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.526+0000 7fbfce53f640 1 -- 172.21.15.76:0/3833554463 >> 172.21.15.76:0/3833554463 conn(0x7fbfc80cecc0 msgr2=0x7fbfc814ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:40.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.526+0000 7fbfce53f640 1 -- 172.21.15.76:0/3833554463 shutdown_connections 2024-04-25T19:17:40.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.526+0000 7fbfce53f640 1 -- 172.21.15.76:0/3833554463 wait complete. 2024-04-25T19:17:40.542 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T19:17:40.542 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T19:17:40.542 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T19:17:40.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.649+0000 7f1116d35640 1 Processor -- start 2024-04-25T19:17:40.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.649+0000 7f1116d35640 1 -- start start 2024-04-25T19:17:40.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.650+0000 7f1116d35640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f11100c6040 0x7f1110154c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.650+0000 7f1116d35640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11100cab50 0x7f11100cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.650+0000 7f1116d35640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11100cb820 con 0x7f11100caf60 2024-04-25T19:17:40.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.650+0000 7f1116d35640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f11100cc730 con 0x7f11100c6040 2024-04-25T19:17:40.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.650+0000 7f1116d35640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f11100cbac0 con 0x7f11100cab50 2024-04-25T19:17:40.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.650+0000 7f11152d2640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f11100caf60 0x7f111015aa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59038/0 (socket says 172.21.15.76:59038) 2024-04-25T19:17:40.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.650+0000 7f11152d2640 1 -- 172.21.15.76:0/554489060 learned_addr learned my addr 172.21.15.76:0/554489060 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:17:40.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.650+0000 7f1114ad1640 1 --2- 172.21.15.76:0/554489060 >> v2:172.21.15.105:3300/0 conn(0x7f11100c6040 0x7f1110154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.650+0000 7f110ffff640 1 --2- 172.21.15.76:0/554489060 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11100cab50 0x7f11100cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.650+0000 7f110f7fe640 1 -- 172.21.15.76:0/554489060 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1769346947 0 0) 0x7f11100cb820 con 0x7f11100caf60 2024-04-25T19:17:40.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.650+0000 7f110f7fe640 1 -- 172.21.15.76:0/554489060 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10fc003520 con 0x7f11100caf60 2024-04-25T19:17:40.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.651+0000 7f1114ad1640 1 -- 172.21.15.76:0/554489060 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11100cab50 msgr2=0x7f11100cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:17:40.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.651+0000 7f1114ad1640 1 --2- 172.21.15.76:0/554489060 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11100cab50 0x7f11100cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.651+0000 7f1114ad1640 1 -- 172.21.15.76:0/554489060 >> v1:172.21.15.76:6789/0 conn(0x7f11100caf60 legacy=0x7f111015aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.651+0000 7f1114ad1640 1 -- 172.21.15.76:0/554489060 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11100cbda0 con 0x7f11100c6040 2024-04-25T19:17:40.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.651+0000 7f110ffff640 1 --2- 172.21.15.76:0/554489060 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11100cab50 0x7f11100cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:17:40.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.651+0000 7f1114ad1640 1 --2- 172.21.15.76:0/554489060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11100c6040 0x7f1110154c20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f10f80097a0 tx=0x7f10f802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db534a66052924f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:40.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.651+0000 7f110f7fe640 1 -- 172.21.15.76:0/554489060 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f8037070 con 0x7f11100c6040 2024-04-25T19:17:40.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.651+0000 7f110f7fe640 1 -- 172.21.15.76:0/554489060 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f8032440 con 0x7f11100c6040 2024-04-25T19:17:40.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.652+0000 7f110f7fe640 1 -- 172.21.15.76:0/554489060 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f80327f0 con 0x7f11100c6040 2024-04-25T19:17:40.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.652+0000 7f1116d35640 1 -- 172.21.15.76:0/554489060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11100c6040 msgr2=0x7f1110154c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.652+0000 7f1116d35640 1 --2- 172.21.15.76:0/554489060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11100c6040 0x7f1110154c20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f10f80097a0 tx=0x7f10f802a210 comp rx=0 tx=0).stop 2024-04-25T19:17:40.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.652+0000 7f1116d35640 1 -- 172.21.15.76:0/554489060 shutdown_connections 2024-04-25T19:17:40.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.652+0000 7f1116d35640 1 --2- 172.21.15.76:0/554489060 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11100cab50 0x7f11100cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.652+0000 7f1116d35640 1 --2- 172.21.15.76:0/554489060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11100c6040 0x7f1110154c20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.652+0000 7f1116d35640 1 -- 172.21.15.76:0/554489060 >> 172.21.15.76:0/554489060 conn(0x7f1110072b60 msgr2=0x7f1110072f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:40.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.652+0000 7f1116d35640 1 -- 172.21.15.76:0/554489060 shutdown_connections 2024-04-25T19:17:40.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.652+0000 7f1116d35640 1 -- 172.21.15.76:0/554489060 wait complete. 2024-04-25T19:17:40.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.654+0000 7f1116d35640 1 Processor -- start 2024-04-25T19:17:40.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.654+0000 7f1116d35640 1 -- start start 2024-04-25T19:17:40.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.655+0000 7f1116d35640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11100c6040 0x7f11101f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.655+0000 7f1116d35640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11100cab50 0x7f11101e4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.655+0000 7f1116d35640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11100caf60 0x7f111014aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.655+0000 7f1116d35640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f111015b120 con 0x7f11100cab50 2024-04-25T19:17:40.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.655+0000 7f1116d35640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f11100cfe00 con 0x7f11100caf60 2024-04-25T19:17:40.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.655+0000 7f1116d35640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f111015b2a0 con 0x7f11100c6040 2024-04-25T19:17:40.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.655+0000 7f110ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11100cab50 0x7f11101e4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.655+0000 7f110ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11100cab50 0x7f11101e4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59770/0 (socket says 172.21.15.76:59770) 2024-04-25T19:17:40.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.655+0000 7f110ffff640 1 -- 172.21.15.76:0/2746347885 learned_addr learned my addr 172.21.15.76:0/2746347885 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:17:40.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.656+0000 7f11152d2640 1 --2- 172.21.15.76:0/2746347885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11100caf60 0x7f111014aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.656+0000 7f1114ad1640 1 --2- 172.21.15.76:0/2746347885 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11100c6040 0x7f11101f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.656+0000 7f110ffff640 1 -- 172.21.15.76:0/2746347885 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11100c6040 msgr2=0x7f11101f42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.656+0000 7f110ffff640 1 --2- 172.21.15.76:0/2746347885 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11100c6040 0x7f11101f42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.656+0000 7f110ffff640 1 -- 172.21.15.76:0/2746347885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11100caf60 msgr2=0x7f111014aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.656+0000 7f110ffff640 1 --2- 172.21.15.76:0/2746347885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11100caf60 0x7f111014aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.656+0000 7f110ffff640 1 -- 172.21.15.76:0/2746347885 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11101f4830 con 0x7f11100cab50 2024-04-25T19:17:40.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.656+0000 7f11152d2640 1 --2- 172.21.15.76:0/2746347885 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11100caf60 0x7f111014aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:17:40.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.656+0000 7f110ffff640 1 --2- 172.21.15.76:0/2746347885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11100cab50 0x7f11101e4160 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f110000e990 tx=0x7f110000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:40.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.656+0000 7f110d7fa640 1 -- 172.21.15.76:0/2746347885 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1100017070 con 0x7f11100cab50 2024-04-25T19:17:40.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.656+0000 7f110d7fa640 1 -- 172.21.15.76:0/2746347885 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f110000c9e0 con 0x7f11100cab50 2024-04-25T19:17:40.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.656+0000 7f1116d35640 1 -- 172.21.15.76:0/2746347885 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1110152f80 con 0x7f11100cab50 2024-04-25T19:17:40.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.656+0000 7f110d7fa640 1 -- 172.21.15.76:0/2746347885 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110000cdb0 con 0x7f11100cab50 2024-04-25T19:17:40.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.656+0000 7f1116d35640 1 -- 172.21.15.76:0/2746347885 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f111015d780 con 0x7f11100cab50 2024-04-25T19:17:40.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.657+0000 7f1116d35640 1 --2- 172.21.15.76:0/2746347885 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11100b54f0 0x7f111014af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.657+0000 7f1116d35640 1 -- 172.21.15.76:0/2746347885 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f11101f4d00 con 0x7f11100b54f0 2024-04-25T19:17:40.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.657+0000 7f1114ad1640 1 --2- 172.21.15.76:0/2746347885 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11100b54f0 0x7f111014af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.657+0000 7f1114ad1640 1 -- 172.21.15.76:0/2746347885 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f11101f1b90 con 0x7f11100b54f0 2024-04-25T19:17:40.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.658+0000 7f1114ad1640 1 --2- 172.21.15.76:0/2746347885 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11100b54f0 0x7f111014af70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1110072730 tx=0x7f10f80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:40.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.658+0000 7f110d7fa640 1 -- 172.21.15.76:0/2746347885 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f8041070 con 0x7f11100b54f0 2024-04-25T19:17:40.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.658+0000 7f110d7fa640 1 -- 172.21.15.76:0/2746347885 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f110001a070 con 0x7f11100cab50 2024-04-25T19:17:40.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.658+0000 7f110d7fa640 1 --2- 172.21.15.76:0/2746347885 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f10f00743e0 0x7f10f00768f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.658+0000 7f110d7fa640 1 -- 172.21.15.76:0/2746347885 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f1100095580 con 0x7f11100cab50 2024-04-25T19:17:40.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.658+0000 7f11152d2640 1 --2- 172.21.15.76:0/2746347885 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f10f00743e0 0x7f10f00768f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.658+0000 7f110d7fa640 1 -- 172.21.15.76:0/2746347885 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f10f803c660 con 0x7f11100b54f0 2024-04-25T19:17:40.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.658+0000 7f110d7fa640 1 -- 172.21.15.76:0/2746347885 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11100b54f0 msgr2=0x7f111014af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.658+0000 7f110d7fa640 1 --2- 172.21.15.76:0/2746347885 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11100b54f0 0x7f111014af70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1110072730 tx=0x7f10f80046c0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.658+0000 7f110d7fa640 1 -- 172.21.15.76:0/2746347885 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11100b54f0 msgr2=0x7f111014af70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:17:40.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.658+0000 7f110d7fa640 1 --2- 172.21.15.76:0/2746347885 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f11100b54f0 0x7f111014af70 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1110072730 tx=0x7f10f80046c0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.659+0000 7f11152d2640 1 --2- 172.21.15.76:0/2746347885 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f10f00743e0 0x7f10f00768f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1108001560 tx=0x7f1108009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:40.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.665+0000 7f1116d35640 1 --2- 172.21.15.76:0/2746347885 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1110152800 0x7f11100b3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.665+0000 7f1116d35640 1 -- 172.21.15.76:0/2746347885 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f111015d950 con 0x7f1110152800 2024-04-25T19:17:40.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.665+0000 7f1114ad1640 1 --2- 172.21.15.76:0/2746347885 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1110152800 0x7f11100b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.666+0000 7f1114ad1640 1 -- 172.21.15.76:0/2746347885 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f11101534a0 con 0x7f1110152800 2024-04-25T19:17:40.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.666+0000 7f1114ad1640 1 --2- 172.21.15.76:0/2746347885 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1110152800 0x7f11100b3820 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f10f80097a0 tx=0x7f10f8008030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:40.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.666+0000 7f110d7fa640 1 -- 172.21.15.76:0/2746347885 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f8037040 con 0x7f1110152800 2024-04-25T19:17:40.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.666+0000 7f110d7fa640 1 -- 172.21.15.76:0/2746347885 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4725 (secure 0 0 0) 0x7f10f803ca10 con 0x7f1110152800 2024-04-25T19:17:40.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.666+0000 7f110d7fa640 1 -- 172.21.15.76:0/2746347885 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1110152800 msgr2=0x7f11100b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.666+0000 7f110d7fa640 1 --2- 172.21.15.76:0/2746347885 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1110152800 0x7f11100b3820 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f10f80097a0 tx=0x7f10f8008030 comp rx=0 tx=0).stop 2024-04-25T19:17:40.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.666+0000 7f110d7fa640 1 -- 172.21.15.76:0/2746347885 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1110152800 msgr2=0x7f11100b3820 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:17:40.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.666+0000 7f110d7fa640 1 --2- 172.21.15.76:0/2746347885 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1110152800 0x7f11100b3820 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f10f80097a0 tx=0x7f10f8008030 comp rx=0 tx=0).stop 2024-04-25T19:17:40.668 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:17:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:17:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:17:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "peon", 2024-04-25T19:17:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 16, 2024-04-25T19:17:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 64444, 2024-04-25T19:17:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:17:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:17:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-04-25T19:17:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:17:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 11, 2024-04-25T19:17:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:17:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:17:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:17:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:17:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:17:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:17:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:17:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:17:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:17:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:17:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:17:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:17:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:17:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:17:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:17:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:17:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:17:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:17:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:17:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:17:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:17:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:17:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:17:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:17:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:17:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:17:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:17:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:17:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:17:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:17:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:17:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:17:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:17:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:17:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:17:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:17:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:17:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:17:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:17:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:17:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:17:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:17:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:17:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:17:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:17:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:17:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:17:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:17:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:17:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:17:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:17:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:17:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:17:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:17:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:17:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:17:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:17:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:17:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:17:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:17:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:17:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:17:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:17:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:17:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:17:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:17:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:17:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:17:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:17:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:17:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:17:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:17:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:17:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:17:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:17:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:17:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:17:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:17:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:17:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:17:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:17:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:17:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:17:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:17:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:17:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:17:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:17:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:17:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:17:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:17:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:17:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:17:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:17:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:17:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:17:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:17:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:17:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:17:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:17:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:17:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": [ 2024-04-25T19:17:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:17:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:17:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:17:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:17:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:17:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:17:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 2 2024-04-25T19:17:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:17:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:17:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:17:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:17:40.681 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:17:40.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.667+0000 7f1116d35640 1 -- 172.21.15.76:0/2746347885 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f10f00743e0 msgr2=0x7f10f00768f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.667+0000 7f1116d35640 1 --2- 172.21.15.76:0/2746347885 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f10f00743e0 0x7f10f00768f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1108001560 tx=0x7f1108009040 comp rx=0 tx=0).stop 2024-04-25T19:17:40.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.667+0000 7f1116d35640 1 -- 172.21.15.76:0/2746347885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11100cab50 msgr2=0x7f11101e4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.667+0000 7f1116d35640 1 --2- 172.21.15.76:0/2746347885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11100cab50 0x7f11101e4160 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f110000e990 tx=0x7f110000ee50 comp rx=0 tx=0).stop 2024-04-25T19:17:40.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.667+0000 7f11152d2640 1 -- 172.21.15.76:0/2746347885 reap_dead start 2024-04-25T19:17:40.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.667+0000 7f11152d2640 1 -- 172.21.15.76:0/2746347885 reap_dead start 2024-04-25T19:17:40.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.667+0000 7f1116d35640 1 -- 172.21.15.76:0/2746347885 shutdown_connections 2024-04-25T19:17:40.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.667+0000 7f1116d35640 1 -- 172.21.15.76:0/2746347885 >> 172.21.15.76:0/2746347885 conn(0x7f1110072b60 msgr2=0x7f111014ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:40.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.667+0000 7f1116d35640 1 -- 172.21.15.76:0/2746347885 shutdown_connections 2024-04-25T19:17:40.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.667+0000 7f1116d35640 1 -- 172.21.15.76:0/2746347885 wait complete. 2024-04-25T19:17:40.684 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T19:17:40.684 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-04-25T19:17:40.684 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-25T19:17:40.684 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-04-25T19:17:40.684 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-04-25T19:17:40.684 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T19:17:40.684 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:17:40.696 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T19:17:40.697 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-04-25T19:17:40.697 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:17:40.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.842+0000 7fbfde496640 1 Processor -- start 2024-04-25T19:17:40.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.842+0000 7fbfde496640 1 -- start start 2024-04-25T19:17:40.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.843+0000 7fbfde496640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfd8072b60 0x7fbfd8072f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.843+0000 7fbfde496640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfd815af40 con 0x7fbfd815e740 2024-04-25T19:17:40.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.843+0000 7fbfde496640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfd815b220 con 0x7fbfd8154d60 2024-04-25T19:17:40.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.843+0000 7fbfde496640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fbfd815bac0 con 0x7fbfd8072b60 2024-04-25T19:17:40.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.843+0000 7fbfdca33640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7fbfd815e740 legacy=0x7fbfd8165480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2024-04-25T19:17:40.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.844+0000 7fbfd77fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfd8072b60 0x7fbfd8072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.844+0000 7fbfd7fff640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fbfd8154d60 0x7fbfd80ca9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.76:36634/0 (socket says 172.21.15.76:36634) 2024-04-25T19:17:40.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.844+0000 7fbfd7fff640 1 -- 172.21.15.76:0/2377303996 learned_addr learned my addr 172.21.15.76:0/2377303996 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:17:40.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.844+0000 7fbfd77fe640 1 -- 172.21.15.76:0/2377303996 >> v1:172.21.15.105:6789/0 conn(0x7fbfd8154d60 legacy=0x7fbfd80ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.844+0000 7fbfd77fe640 1 -- 172.21.15.76:0/2377303996 >> v1:172.21.15.76:6789/0 conn(0x7fbfd815e740 legacy=0x7fbfd8165480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:17:40.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.844+0000 7fbfd77fe640 1 -- 172.21.15.76:0/2377303996 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd815b500 con 0x7fbfd8072b60 2024-04-25T19:17:40.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.845+0000 7fbfd77fe640 1 --2- 172.21.15.76:0/2377303996 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfd8072b60 0x7fbfd8072f70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc0097d0 tx=0x7fbfcc02a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3acb4e9b0827d13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:40.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.845+0000 7fbfd6ffd640 1 -- 172.21.15.76:0/2377303996 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc037070 con 0x7fbfd8072b60 2024-04-25T19:17:40.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.845+0000 7fbfd6ffd640 1 -- 172.21.15.76:0/2377303996 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfcc02ae40 con 0x7fbfd8072b60 2024-04-25T19:17:40.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.845+0000 7fbfd6ffd640 1 -- 172.21.15.76:0/2377303996 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc032660 con 0x7fbfd8072b60 2024-04-25T19:17:40.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.846+0000 7fbfde496640 1 -- 172.21.15.76:0/2377303996 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfd8072b60 msgr2=0x7fbfd8072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.846+0000 7fbfde496640 1 --2- 172.21.15.76:0/2377303996 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfd8072b60 0x7fbfd8072f70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc0097d0 tx=0x7fbfcc02a130 comp rx=0 tx=0).stop 2024-04-25T19:17:40.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.846+0000 7fbfde496640 1 -- 172.21.15.76:0/2377303996 shutdown_connections 2024-04-25T19:17:40.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.846+0000 7fbfde496640 1 --2- 172.21.15.76:0/2377303996 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfd8072b60 0x7fbfd8072f70 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.846+0000 7fbfde496640 1 -- 172.21.15.76:0/2377303996 >> 172.21.15.76:0/2377303996 conn(0x7fbfd8070b00 msgr2=0x7fbfd8070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:40.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.846+0000 7fbfde496640 1 -- 172.21.15.76:0/2377303996 shutdown_connections 2024-04-25T19:17:40.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.846+0000 7fbfde496640 1 -- 172.21.15.76:0/2377303996 wait complete. 2024-04-25T19:17:40.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.847+0000 7fbfde496640 1 Processor -- start 2024-04-25T19:17:40.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.847+0000 7fbfde496640 1 -- start start 2024-04-25T19:17:40.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.849+0000 7fbfde496640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfd8072b60 0x7fbfd81f1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.849+0000 7fbfde496640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfd8154d60 0x7fbfd80d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.849+0000 7fbfde496640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfd815e740 0x7fbfd80c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.849+0000 7fbfde496640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd8155170 con 0x7fbfd8154d60 2024-04-25T19:17:40.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.849+0000 7fbfde496640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd815e2b0 con 0x7fbfd815e740 2024-04-25T19:17:40.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.849+0000 7fbfde496640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fbfd80cb0e0 con 0x7fbfd8072b60 2024-04-25T19:17:40.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.849+0000 7fbfd77fe640 1 -- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfd8154d60 msgr2=0x7fbfd80d00d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:3300/0 2024-04-25T19:17:40.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.849+0000 7fbfd77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfd8154d60 0x7fbfd80d00d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:17:40.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.849+0000 7fbfdca33640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfd815e740 0x7fbfd80c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.849+0000 7fbfd7fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfd8072b60 0x7fbfd81f1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.849+0000 7fbfd7fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfd8072b60 0x7fbfd81f1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:56078/0 (socket says 172.21.15.76:56078) 2024-04-25T19:17:40.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.849+0000 7fbfd7fff640 1 -- 172.21.15.76:0/1162779585 learned_addr learned my addr 172.21.15.76:0/1162779585 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:17:40.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.849+0000 7fbfd7fff640 1 -- 172.21.15.76:0/1162779585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfd815e740 msgr2=0x7fbfd80c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:40.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.849+0000 7fbfd7fff640 1 --2- 172.21.15.76:0/1162779585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfd815e740 0x7fbfd80c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.849+0000 7fbfd7fff640 1 -- 172.21.15.76:0/1162779585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfd8154d60 msgr2=0x7fbfd80d00d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:17:40.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.849+0000 7fbfd7fff640 1 --2- 172.21.15.76:0/1162779585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfd8154d60 0x7fbfd80d00d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:40.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.849+0000 7fbfd7fff640 1 -- 172.21.15.76:0/1162779585 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd80d0610 con 0x7fbfd8072b60 2024-04-25T19:17:40.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.850+0000 7fbfd7fff640 1 --2- 172.21.15.76:0/1162779585 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfd8072b60 0x7fbfd81f1a90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbfc800c3c0 tx=0x7fbfc800c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:40.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.850+0000 7fbfd4ff9640 1 -- 172.21.15.76:0/1162779585 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8017070 con 0x7fbfd8072b60 2024-04-25T19:17:40.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.850+0000 7fbfd4ff9640 1 -- 172.21.15.76:0/1162779585 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfc8005ce0 con 0x7fbfd8072b60 2024-04-25T19:17:40.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.850+0000 7fbfde496640 1 -- 172.21.15.76:0/1162779585 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd80c8810 con 0x7fbfd8072b60 2024-04-25T19:17:40.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.850+0000 7fbfd4ff9640 1 -- 172.21.15.76:0/1162779585 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc80072f0 con 0x7fbfd8072b60 2024-04-25T19:17:40.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.850+0000 7fbfde496640 1 -- 172.21.15.76:0/1162779585 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd80c8dd0 con 0x7fbfd8072b60 2024-04-25T19:17:40.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.851+0000 7fbfde496640 1 -- 172.21.15.76:0/1162779585 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfd806a390 con 0x7fbfd8072b60 2024-04-25T19:17:40.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.851+0000 7fbfd4ff9640 1 -- 172.21.15.76:0/1162779585 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbfc801a070 con 0x7fbfd8072b60 2024-04-25T19:17:40.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.852+0000 7fbfd4ff9640 1 --2- 172.21.15.76:0/1162779585 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbfc4074460 0x7fbfc4076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:40.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.852+0000 7fbfd4ff9640 1 -- 172.21.15.76:0/1162779585 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fbfc8094570 con 0x7fbfd8072b60 2024-04-25T19:17:40.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.852+0000 7fbfd77fe640 1 --2- 172.21.15.76:0/1162779585 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbfc4074460 0x7fbfc4076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:40.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.852+0000 7fbfd77fe640 1 --2- 172.21.15.76:0/1162779585 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbfc4074460 0x7fbfc4076970 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc034000 tx=0x7fbfcc02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:40.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:40.857+0000 7fbfd4ff9640 1 -- 172.21.15.76:0/1162779585 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbfc805e6b0 con 0x7fbfd8072b60 2024-04-25T19:17:41.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:41.101+0000 7fbfde496640 1 -- 172.21.15.76:0/1162779585 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fbfd80cb430 con 0x7fbfd8072b60 2024-04-25T19:17:41.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:41.101+0000 7fbfd4ff9640 1 -- 172.21.15.76:0/1162779585 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7fbfc80629b0 con 0x7fbfd8072b60 2024-04-25T19:17:41.103 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:17:41.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:41.103+0000 7fbfde496640 1 -- 172.21.15.76:0/1162779585 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbfc4074460 msgr2=0x7fbfc4076970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:41.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:41.103+0000 7fbfde496640 1 --2- 172.21.15.76:0/1162779585 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbfc4074460 0x7fbfc4076970 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc034000 tx=0x7fbfcc02e000 comp rx=0 tx=0).stop 2024-04-25T19:17:41.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:41.103+0000 7fbfde496640 1 -- 172.21.15.76:0/1162779585 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfd8072b60 msgr2=0x7fbfd81f1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:41.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:41.103+0000 7fbfde496640 1 --2- 172.21.15.76:0/1162779585 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfd8072b60 0x7fbfd81f1a90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbfc800c3c0 tx=0x7fbfc800c880 comp rx=0 tx=0).stop 2024-04-25T19:17:41.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:41.103+0000 7fbfde496640 1 -- 172.21.15.76:0/1162779585 shutdown_connections 2024-04-25T19:17:41.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:41.103+0000 7fbfde496640 1 --2- 172.21.15.76:0/1162779585 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbfc4074460 0x7fbfc4076970 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:41.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:41.103+0000 7fbfde496640 1 --2- 172.21.15.76:0/1162779585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbfd815e740 0x7fbfd80c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:41.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:41.103+0000 7fbfde496640 1 --2- 172.21.15.76:0/1162779585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfd8154d60 0x7fbfd80d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:41.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:41.103+0000 7fbfde496640 1 --2- 172.21.15.76:0/1162779585 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbfd8072b60 0x7fbfd81f1a90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:41.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:41.103+0000 7fbfde496640 1 -- 172.21.15.76:0/1162779585 >> 172.21.15.76:0/1162779585 conn(0x7fbfd8070b00 msgr2=0x7fbfd81506f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:41.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:41.103+0000 7fbfde496640 1 -- 172.21.15.76:0/1162779585 shutdown_connections 2024-04-25T19:17:41.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:41.103+0000 7fbfde496640 1 -- 172.21.15.76:0/1162779585 wait complete. 2024-04-25T19:17:44.117 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:17:44.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.243+0000 7f0ff49a8640 1 Processor -- start 2024-04-25T19:17:44.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.243+0000 7f0ff49a8640 1 -- start start 2024-04-25T19:17:44.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.245+0000 7f0ff49a8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0ff015a9f0 0x7f0ff00c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:44.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.245+0000 7f0ff49a8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0ff015df30 0x7f0ff0074130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:44.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.245+0000 7f0ff49a8640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ff015ae00 con 0x7f0ff0154ee0 2024-04-25T19:17:44.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.245+0000 7f0ff49a8640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f0ff0068bf0 con 0x7f0ff015a9f0 2024-04-25T19:17:44.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.245+0000 7f0ff49a8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f0ff00c8e60 con 0x7f0ff015df30 2024-04-25T19:17:44.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.245+0000 7f0fee59c640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f0ff0154ee0 legacy=0x7f0ff00684e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2024-04-25T19:17:44.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.245+0000 7f0fedd9b640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f0ff015a9f0 msgr2=0x7f0ff00c98b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2024-04-25T19:17:44.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.245+0000 7f0fedd9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0ff015a9f0 0x7f0ff00c98b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:17:44.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.245+0000 7f0feed9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0ff015df30 0x7f0ff0074130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:44.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.245+0000 7f0feed9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0ff015df30 0x7f0ff0074130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:52666/0 (socket says 172.21.15.76:52666) 2024-04-25T19:17:44.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.245+0000 7f0feed9d640 1 -- 172.21.15.76:0/4074523560 learned_addr learned my addr 172.21.15.76:0/4074523560 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:17:44.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.245+0000 7f0feed9d640 1 -- 172.21.15.76:0/4074523560 >> v2:172.21.15.76:3300/0 conn(0x7f0ff015a9f0 msgr2=0x7f0ff00c98b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:17:44.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.245+0000 7f0feed9d640 1 --2- 172.21.15.76:0/4074523560 >> v2:172.21.15.76:3300/0 conn(0x7f0ff015a9f0 0x7f0ff00c98b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.245+0000 7f0feed9d640 1 -- 172.21.15.76:0/4074523560 >> v1:172.21.15.76:6789/0 conn(0x7f0ff0154ee0 legacy=0x7f0ff00684e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:17:44.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.245+0000 7f0feed9d640 1 -- 172.21.15.76:0/4074523560 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff015e6c0 con 0x7f0ff015df30 2024-04-25T19:17:44.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.246+0000 7f0feed9d640 1 --2- 172.21.15.76:0/4074523560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ff015df30 0x7f0ff0074130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0070bc0 tx=0x7f0fe802a7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e80c700070ec61ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:44.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.246+0000 7f0fed59a640 1 -- 172.21.15.76:0/4074523560 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8036070 con 0x7f0ff015df30 2024-04-25T19:17:44.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.246+0000 7f0fed59a640 1 -- 172.21.15.76:0/4074523560 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fe8031600 con 0x7f0ff015df30 2024-04-25T19:17:44.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.246+0000 7f0fed59a640 1 -- 172.21.15.76:0/4074523560 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe80319d0 con 0x7f0ff015df30 2024-04-25T19:17:44.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.247+0000 7f0ff49a8640 1 -- 172.21.15.76:0/4074523560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ff015df30 msgr2=0x7f0ff0074130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:44.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.247+0000 7f0ff49a8640 1 --2- 172.21.15.76:0/4074523560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ff015df30 0x7f0ff0074130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0070bc0 tx=0x7f0fe802a7a0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.247+0000 7f0ff49a8640 1 -- 172.21.15.76:0/4074523560 shutdown_connections 2024-04-25T19:17:44.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.247+0000 7f0ff49a8640 1 --2- 172.21.15.76:0/4074523560 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ff015df30 0x7f0ff0074130 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.247+0000 7f0ff49a8640 1 --2- 172.21.15.76:0/4074523560 >> v2:172.21.15.76:3300/0 conn(0x7f0ff015a9f0 0x7f0ff00c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.247+0000 7f0ff49a8640 1 -- 172.21.15.76:0/4074523560 >> 172.21.15.76:0/4074523560 conn(0x7f0ff006c5c0 msgr2=0x7f0ff006c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:44.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.247+0000 7f0ff49a8640 1 -- 172.21.15.76:0/4074523560 shutdown_connections 2024-04-25T19:17:44.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.247+0000 7f0ff49a8640 1 -- 172.21.15.76:0/4074523560 wait complete. 2024-04-25T19:17:44.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.249+0000 7f0ff49a8640 1 Processor -- start 2024-04-25T19:17:44.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.249+0000 7f0ff49a8640 1 -- start start 2024-04-25T19:17:44.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.251+0000 7f0ff49a8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ff0154ee0 0x7f0ff015bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:44.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.251+0000 7f0ff49a8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ff015a9f0 0x7f0ff015cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:44.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.251+0000 7f0ff49a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ff015df30 0x7f0ff0073450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:44.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.251+0000 7f0ff49a8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff00c90b0 con 0x7f0ff015a9f0 2024-04-25T19:17:44.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.251+0000 7f0ff49a8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff015ec80 con 0x7f0ff015df30 2024-04-25T19:17:44.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.251+0000 7f0ff49a8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f0ff00c9390 con 0x7f0ff0154ee0 2024-04-25T19:17:44.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.251+0000 7f0fedd9b640 1 -- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ff015a9f0 msgr2=0x7f0ff015cb60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:3300/0 2024-04-25T19:17:44.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.251+0000 7f0fedd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ff015a9f0 0x7f0ff015cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:17:44.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.251+0000 7f0fee59c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ff0154ee0 0x7f0ff015bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:44.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.251+0000 7f0feed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ff015df30 0x7f0ff0073450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:44.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.251+0000 7f0fee59c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ff0154ee0 0x7f0ff015bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:56082/0 (socket says 172.21.15.76:56082) 2024-04-25T19:17:44.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.251+0000 7f0feed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ff015df30 0x7f0ff0073450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:52670/0 (socket says 172.21.15.76:52670) 2024-04-25T19:17:44.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.251+0000 7f0fee59c640 1 -- 172.21.15.76:0/2744032995 learned_addr learned my addr 172.21.15.76:0/2744032995 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:17:44.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.251+0000 7f0fee59c640 1 -- 172.21.15.76:0/2744032995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ff015df30 msgr2=0x7f0ff0073450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:44.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.251+0000 7f0fee59c640 1 --2- 172.21.15.76:0/2744032995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ff015df30 0x7f0ff0073450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.251+0000 7f0fee59c640 1 -- 172.21.15.76:0/2744032995 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ff015a9f0 msgr2=0x7f0ff015cb60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:17:44.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.251+0000 7f0fee59c640 1 --2- 172.21.15.76:0/2744032995 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ff015a9f0 0x7f0ff015cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.251+0000 7f0fee59c640 1 -- 172.21.15.76:0/2744032995 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff0152380 con 0x7f0ff0154ee0 2024-04-25T19:17:44.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.251+0000 7f0feed9d640 1 --2- 172.21.15.76:0/2744032995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ff015df30 0x7f0ff0073450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:17:44.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.252+0000 7f0fee59c640 1 --2- 172.21.15.76:0/2744032995 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ff0154ee0 0x7f0ff015bef0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0fe000b580 tx=0x7f0fe000ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:44.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.252+0000 7f0fdeffd640 1 -- 172.21.15.76:0/2744032995 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0013020 con 0x7f0ff0154ee0 2024-04-25T19:17:44.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.252+0000 7f0fdeffd640 1 -- 172.21.15.76:0/2744032995 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fe0003de0 con 0x7f0ff0154ee0 2024-04-25T19:17:44.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.252+0000 7f0fdeffd640 1 -- 172.21.15.76:0/2744032995 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe00052b0 con 0x7f0ff0154ee0 2024-04-25T19:17:44.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.252+0000 7f0ff49a8640 1 -- 172.21.15.76:0/2744032995 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ff01f83a0 con 0x7f0ff0154ee0 2024-04-25T19:17:44.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.252+0000 7f0ff49a8640 1 -- 172.21.15.76:0/2744032995 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ff0151e00 con 0x7f0ff0154ee0 2024-04-25T19:17:44.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.253+0000 7f0ff49a8640 1 -- 172.21.15.76:0/2744032995 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ff006d6a0 con 0x7f0ff0154ee0 2024-04-25T19:17:44.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.254+0000 7f0fdeffd640 1 -- 172.21.15.76:0/2744032995 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0fe0016020 con 0x7f0ff0154ee0 2024-04-25T19:17:44.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.254+0000 7f0fdeffd640 1 --2- 172.21.15.76:0/2744032995 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0fd0074460 0x7f0fd0076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:44.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.254+0000 7f0fdeffd640 1 -- 172.21.15.76:0/2744032995 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f0fe0094f40 con 0x7f0ff0154ee0 2024-04-25T19:17:44.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.254+0000 7f0fedd9b640 1 --2- 172.21.15.76:0/2744032995 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0fd0074460 0x7f0fd0076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:44.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.255+0000 7f0fedd9b640 1 --2- 172.21.15.76:0/2744032995 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0fd0074460 0x7f0fd0076970 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0ff01585a0 tx=0x7f0fe4006d30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:44.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.259+0000 7f0fdeffd640 1 -- 172.21.15.76:0/2744032995 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0fe001d020 con 0x7f0ff0154ee0 2024-04-25T19:17:44.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.574+0000 7f0ff49a8640 1 -- 172.21.15.76:0/2744032995 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0ff0076950 con 0x7f0ff0154ee0 2024-04-25T19:17:44.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.575+0000 7f0fdeffd640 1 -- 172.21.15.76:0/2744032995 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7f0fe005f770 con 0x7f0ff0154ee0 2024-04-25T19:17:44.576 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:17:44.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.577+0000 7f0ff49a8640 1 -- 172.21.15.76:0/2744032995 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0fd0074460 msgr2=0x7f0fd0076970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:44.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.577+0000 7f0ff49a8640 1 --2- 172.21.15.76:0/2744032995 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0fd0074460 0x7f0fd0076970 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0ff01585a0 tx=0x7f0fe4006d30 comp rx=0 tx=0).stop 2024-04-25T19:17:44.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.577+0000 7f0ff49a8640 1 -- 172.21.15.76:0/2744032995 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ff0154ee0 msgr2=0x7f0ff015bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:44.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.577+0000 7f0ff49a8640 1 --2- 172.21.15.76:0/2744032995 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ff0154ee0 0x7f0ff015bef0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0fe000b580 tx=0x7f0fe000ba40 comp rx=0 tx=0).stop 2024-04-25T19:17:44.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.577+0000 7f0ff49a8640 1 -- 172.21.15.76:0/2744032995 shutdown_connections 2024-04-25T19:17:44.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.577+0000 7f0ff49a8640 1 --2- 172.21.15.76:0/2744032995 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0fd0074460 0x7f0fd0076970 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.577+0000 7f0ff49a8640 1 --2- 172.21.15.76:0/2744032995 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0ff015df30 0x7f0ff0073450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.577+0000 7f0ff49a8640 1 --2- 172.21.15.76:0/2744032995 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ff015a9f0 0x7f0ff015cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.577+0000 7f0ff49a8640 1 --2- 172.21.15.76:0/2744032995 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0ff0154ee0 0x7f0ff015bef0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:44.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.577+0000 7f0ff49a8640 1 -- 172.21.15.76:0/2744032995 >> 172.21.15.76:0/2744032995 conn(0x7f0ff006c5c0 msgr2=0x7f0ff00684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:44.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.577+0000 7f0ff49a8640 1 -- 172.21.15.76:0/2744032995 shutdown_connections 2024-04-25T19:17:44.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:44.577+0000 7f0ff49a8640 1 -- 172.21.15.76:0/2744032995 wait complete. 2024-04-25T19:17:47.593 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:17:47.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.723+0000 7fbe9f684640 1 Processor -- start 2024-04-25T19:17:47.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.724+0000 7fbe9f684640 1 -- start start 2024-04-25T19:17:47.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.725+0000 7fbe9f684640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbe98156f60 0x7fbe980cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:47.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.725+0000 7fbe9f684640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbe980c95c0 0x7fbe980cf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:47.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.725+0000 7fbe9f684640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe98073c30 con 0x7fbe980c9ea0 2024-04-25T19:17:47.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.725+0000 7fbe9f684640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fbe9815d730 con 0x7fbe980c95c0 2024-04-25T19:17:47.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.725+0000 7fbe9f684640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fbe980746b0 con 0x7fbe98156f60 2024-04-25T19:17:47.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.725+0000 7fbe9cc1f640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7fbe980c95c0 msgr2=0x7fbe980cf520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2024-04-25T19:17:47.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.725+0000 7fbe9cc1f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbe980c95c0 0x7fbe980cf520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:17:47.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.725+0000 7fbe9d420640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbe98156f60 0x7fbe980cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:47.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.725+0000 7fbe9dc21640 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fbe980c9ea0 0x7fbe980c7790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.76:36638/0 (socket says 172.21.15.76:36638) 2024-04-25T19:17:47.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.726+0000 7fbe9d420640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbe98156f60 0x7fbe980cefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:52682/0 (socket says 172.21.15.76:52682) 2024-04-25T19:17:47.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.726+0000 7fbe9dc21640 1 -- 172.21.15.76:0/2319562896 learned_addr learned my addr 172.21.15.76:0/2319562896 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:17:47.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.726+0000 7fbe9d420640 1 -- 172.21.15.76:0/2319562896 >> v2:172.21.15.76:3300/0 conn(0x7fbe980c95c0 msgr2=0x7fbe980cf520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:17:47.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.726+0000 7fbe9d420640 1 --2- 172.21.15.76:0/2319562896 >> v2:172.21.15.76:3300/0 conn(0x7fbe980c95c0 0x7fbe980cf520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:47.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.726+0000 7fbe9d420640 1 -- 172.21.15.76:0/2319562896 >> v1:172.21.15.105:6789/0 conn(0x7fbe980c9ea0 legacy=0x7fbe980c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:47.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.726+0000 7fbe9d420640 1 -- 172.21.15.76:0/2319562896 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe98073940 con 0x7fbe98156f60 2024-04-25T19:17:47.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.727+0000 7fbe9d420640 1 --2- 172.21.15.76:0/2319562896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe98156f60 0x7fbe980cefe0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbe9806c720 tx=0x7fbe9402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=552a3f27bc891785 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:47.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.727+0000 7fbe8ffff640 1 -- 172.21.15.76:0/2319562896 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe94037070 con 0x7fbe98156f60 2024-04-25T19:17:47.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.727+0000 7fbe8ffff640 1 -- 172.21.15.76:0/2319562896 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe94032440 con 0x7fbe98156f60 2024-04-25T19:17:47.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.727+0000 7fbe8ffff640 1 -- 172.21.15.76:0/2319562896 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe94032810 con 0x7fbe98156f60 2024-04-25T19:17:47.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.728+0000 7fbe9f684640 1 -- 172.21.15.76:0/2319562896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe98156f60 msgr2=0x7fbe980cefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:47.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.728+0000 7fbe9f684640 1 --2- 172.21.15.76:0/2319562896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe98156f60 0x7fbe980cefe0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbe9806c720 tx=0x7fbe9402a1c0 comp rx=0 tx=0).stop 2024-04-25T19:17:47.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.728+0000 7fbe9f684640 1 -- 172.21.15.76:0/2319562896 shutdown_connections 2024-04-25T19:17:47.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.728+0000 7fbe9f684640 1 --2- 172.21.15.76:0/2319562896 >> v2:172.21.15.76:3300/0 conn(0x7fbe980c95c0 0x7fbe980cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:47.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.728+0000 7fbe9f684640 1 --2- 172.21.15.76:0/2319562896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe98156f60 0x7fbe980cefe0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:47.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.728+0000 7fbe9f684640 1 -- 172.21.15.76:0/2319562896 >> 172.21.15.76:0/2319562896 conn(0x7fbe98071a30 msgr2=0x7fbe98071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:47.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.728+0000 7fbe9f684640 1 -- 172.21.15.76:0/2319562896 shutdown_connections 2024-04-25T19:17:47.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.728+0000 7fbe9f684640 1 -- 172.21.15.76:0/2319562896 wait complete. 2024-04-25T19:17:47.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.730+0000 7fbe9f684640 1 Processor -- start 2024-04-25T19:17:47.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.730+0000 7fbe9f684640 1 -- start start 2024-04-25T19:17:47.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.731+0000 7fbe9f684640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe980c95c0 0x7fbe9815b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:47.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.731+0000 7fbe9f684640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe980c9ea0 0x7fbe9815ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:47.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.731+0000 7fbe9f684640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe98156f60 0x7fbe9814a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:47.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.731+0000 7fbe9f684640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbe980740e0 con 0x7fbe980c9ea0 2024-04-25T19:17:47.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.731+0000 7fbe9f684640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9815da10 con 0x7fbe980c95c0 2024-04-25T19:17:47.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.731+0000 7fbe9f684640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fbe980c7ea0 con 0x7fbe98156f60 2024-04-25T19:17:47.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.731+0000 7fbe9cc1f640 1 -- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe980c9ea0 msgr2=0x7fbe9815ab60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:3300/0 2024-04-25T19:17:47.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.731+0000 7fbe9cc1f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe980c9ea0 0x7fbe9815ab60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:17:47.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.731+0000 7fbe9d420640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe980c95c0 0x7fbe9815b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:47.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.731+0000 7fbe9dc21640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe98156f60 0x7fbe9814a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:47.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.731+0000 7fbe9d420640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe980c95c0 0x7fbe9815b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:52694/0 (socket says 172.21.15.76:52694) 2024-04-25T19:17:47.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.731+0000 7fbe9d420640 1 -- 172.21.15.76:0/4099733898 learned_addr learned my addr 172.21.15.76:0/4099733898 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:17:47.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.732+0000 7fbe9d420640 1 -- 172.21.15.76:0/4099733898 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe98156f60 msgr2=0x7fbe9814a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:47.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.732+0000 7fbe9d420640 1 --2- 172.21.15.76:0/4099733898 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe98156f60 0x7fbe9814a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:47.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.732+0000 7fbe9d420640 1 -- 172.21.15.76:0/4099733898 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe980c9ea0 msgr2=0x7fbe9815ab60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:17:47.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.732+0000 7fbe9d420640 1 --2- 172.21.15.76:0/4099733898 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe980c9ea0 0x7fbe9815ab60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:47.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.732+0000 7fbe9d420640 1 -- 172.21.15.76:0/4099733898 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe980ccf60 con 0x7fbe980c95c0 2024-04-25T19:17:47.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.732+0000 7fbe9d420640 1 --2- 172.21.15.76:0/4099733898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe980c95c0 0x7fbe9815b520 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbe94000c00 tx=0x7fbe940046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:47.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.732+0000 7fbe8dffb640 1 -- 172.21.15.76:0/4099733898 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe94041070 con 0x7fbe980c95c0 2024-04-25T19:17:47.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.732+0000 7fbe8dffb640 1 -- 172.21.15.76:0/4099733898 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe9403c950 con 0x7fbe980c95c0 2024-04-25T19:17:47.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.732+0000 7fbe8dffb640 1 -- 172.21.15.76:0/4099733898 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe94037040 con 0x7fbe980c95c0 2024-04-25T19:17:47.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.732+0000 7fbe9f684640 1 -- 172.21.15.76:0/4099733898 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe980cdd00 con 0x7fbe980c95c0 2024-04-25T19:17:47.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.732+0000 7fbe9f684640 1 -- 172.21.15.76:0/4099733898 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe980ce2c0 con 0x7fbe980c95c0 2024-04-25T19:17:47.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.734+0000 7fbe9f684640 1 -- 172.21.15.76:0/4099733898 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe9806a390 con 0x7fbe980c95c0 2024-04-25T19:17:47.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.734+0000 7fbe8dffb640 1 -- 172.21.15.76:0/4099733898 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbe94044070 con 0x7fbe980c95c0 2024-04-25T19:17:47.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.735+0000 7fbe8dffb640 1 --2- 172.21.15.76:0/4099733898 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbe7c074460 0x7fbe7c076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:47.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.735+0000 7fbe8dffb640 1 -- 172.21.15.76:0/4099733898 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fbe940b5e20 con 0x7fbe980c95c0 2024-04-25T19:17:47.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.735+0000 7fbe9cc1f640 1 --2- 172.21.15.76:0/4099733898 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbe7c074460 0x7fbe7c076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:47.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.736+0000 7fbe9cc1f640 1 --2- 172.21.15.76:0/4099733898 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbe7c074460 0x7fbe7c076970 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbe980ccd50 tx=0x7fbe88006d50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:17:47.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:47.740+0000 7fbe8dffb640 1 -- 172.21.15.76:0/4099733898 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbe9407fee0 con 0x7fbe980c95c0 2024-04-25T19:17:48.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:48.082+0000 7fbe9f684640 1 -- 172.21.15.76:0/4099733898 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fbe980c9a70 con 0x7fbe980c95c0 2024-04-25T19:17:48.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:48.083+0000 7fbe8dffb640 1 -- 172.21.15.76:0/4099733898 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7fbe940841e0 con 0x7fbe980c95c0 2024-04-25T19:17:48.084 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:17:48.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:48.085+0000 7fbe9f684640 1 -- 172.21.15.76:0/4099733898 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbe7c074460 msgr2=0x7fbe7c076970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:48.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:48.085+0000 7fbe9f684640 1 --2- 172.21.15.76:0/4099733898 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbe7c074460 0x7fbe7c076970 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbe980ccd50 tx=0x7fbe88006d50 comp rx=0 tx=0).stop 2024-04-25T19:17:48.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:48.085+0000 7fbe9f684640 1 -- 172.21.15.76:0/4099733898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe980c95c0 msgr2=0x7fbe9815b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:48.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:48.085+0000 7fbe9f684640 1 --2- 172.21.15.76:0/4099733898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe980c95c0 0x7fbe9815b520 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbe94000c00 tx=0x7fbe940046c0 comp rx=0 tx=0).stop 2024-04-25T19:17:48.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:48.085+0000 7fbe9f684640 1 -- 172.21.15.76:0/4099733898 shutdown_connections 2024-04-25T19:17:48.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:48.085+0000 7fbe9f684640 1 --2- 172.21.15.76:0/4099733898 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbe7c074460 0x7fbe7c076970 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:48.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:48.085+0000 7fbe9f684640 1 --2- 172.21.15.76:0/4099733898 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbe98156f60 0x7fbe9814a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:48.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:48.085+0000 7fbe9f684640 1 --2- 172.21.15.76:0/4099733898 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe980c9ea0 0x7fbe9815ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:48.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:48.085+0000 7fbe9f684640 1 --2- 172.21.15.76:0/4099733898 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbe980c95c0 0x7fbe9815b520 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:48.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:48.085+0000 7fbe9f684640 1 -- 172.21.15.76:0/4099733898 >> 172.21.15.76:0/4099733898 conn(0x7fbe98071a30 msgr2=0x7fbe98155ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:17:48.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:48.085+0000 7fbe9f684640 1 -- 172.21.15.76:0/4099733898 shutdown_connections 2024-04-25T19:17:48.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:48.085+0000 7fbe9f684640 1 -- 172.21.15.76:0/4099733898 wait complete. 2024-04-25T19:17:50.980 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:17:50.980 INFO:tasks.workunit.client.0.smithi076.stdout: Children see throughput for 1 rewriters = 124551.88 kB/sec 2024-04-25T19:17:50.980 INFO:tasks.workunit.client.0.smithi076.stdout: Parent sees throughput for 1 rewriters = 124549.41 kB/sec 2024-04-25T19:17:50.980 INFO:tasks.workunit.client.0.smithi076.stdout: Min throughput per process = 124551.88 kB/sec 2024-04-25T19:17:50.980 INFO:tasks.workunit.client.0.smithi076.stdout: Max throughput per process = 124551.88 kB/sec 2024-04-25T19:17:50.980 INFO:tasks.workunit.client.0.smithi076.stdout: Avg throughput per process = 124551.88 kB/sec 2024-04-25T19:17:50.980 INFO:tasks.workunit.client.0.smithi076.stdout: Min xfer = 1048576.00 kB 2024-04-25T19:17:51.101 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:17:51.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.207+0000 7f80a5892640 1 Processor -- start 2024-04-25T19:17:51.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.207+0000 7f80a5892640 1 -- start start 2024-04-25T19:17:51.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.208+0000 7f80a5892640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f80a0157200 0x7f80a00d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:51.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.208+0000 7f80a5892640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f80a00cfd50 0x7f80a00cee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:51.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.208+0000 7f80a5892640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80a00d0160 0x7f80a00cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:51.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.208+0000 7f80a5892640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f80a015dd00 con 0x7f80a00cfd50 2024-04-25T19:17:51.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.208+0000 7f80a5892640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f80a015cea0 con 0x7f80a0157200 2024-04-25T19:17:51.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.208+0000 7f80a5892640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f80a015d180 con 0x7f80a00d0160 2024-04-25T19:17:51.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.208+0000 7f809e7fc640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f80a00cfd50 msgr2=0x7f80a00cee90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2024-04-25T19:17:51.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.208+0000 7f809e7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f80a00cfd50 0x7f80a00cee90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:17:51.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.208+0000 7f809f7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80a00d0160 0x7f80a00cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:51.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.209+0000 7f809effd640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f80a0157200 0x7f80a00d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:51.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.209+0000 7f809f7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80a00d0160 0x7f80a00cf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:44206/0 (socket says 172.21.15.76:44206) 2024-04-25T19:17:51.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.209+0000 7f809f7fe640 1 -- 172.21.15.76:0/2185088306 learned_addr learned my addr 172.21.15.76:0/2185088306 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:17:51.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.209+0000 7f809f7fe640 1 -- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80a00d0160 msgr2=0x7f80a00cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T19:17:51.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.209+0000 7f809f7fe640 1 -- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80a00d0160 msgr2=0x7f80a00cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:17:51.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.209+0000 7f809f7fe640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80a00d0160 0x7f80a00cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:51.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.209+0000 7f809f7fe640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80a00d0160 0x7f80a00cf3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:17:51.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.209+0000 7f809effd640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f80a0157200 msgr2=0x7f80a00d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T19:17:51.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.209+0000 7f809effd640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f80a0157200 msgr2=0x7f80a00d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:17:51.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.209+0000 7f809effd640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f80a0157200 0x7f80a00d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:51.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.209+0000 7f809effd640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f80a0157200 0x7f80a00d0b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:17:51.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.409+0000 7f809e7fc640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.76:3300/0 conn(0x7f80a00cfd50 msgr2=0x7f80a00cee90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2024-04-25T19:17:51.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.409+0000 7f809e7fc640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.76:3300/0 conn(0x7f80a00cfd50 0x7f80a00cee90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T19:17:51.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.410+0000 7f809f7fe640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80a00d0160 0x7f80a00cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:51.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.410+0000 7f809effd640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f80a0157200 0x7f80a00d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:51.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.410+0000 7f809f7fe640 1 -- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80a00d0160 msgr2=0x7f80a00cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T19:17:51.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.410+0000 7f809f7fe640 1 -- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80a00d0160 msgr2=0x7f80a00cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:17:51.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.410+0000 7f809f7fe640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80a00d0160 0x7f80a00cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:51.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.410+0000 7f809effd640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f80a0157200 msgr2=0x7f80a00d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T19:17:51.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.410+0000 7f809f7fe640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80a00d0160 0x7f80a00cf3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T19:17:51.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.410+0000 7f809effd640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f80a0157200 msgr2=0x7f80a00d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:17:51.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.410+0000 7f809effd640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f80a0157200 0x7f80a00d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:51.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.410+0000 7f809effd640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f80a0157200 0x7f80a00d0b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T19:17:51.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.810+0000 7f809e7fc640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.76:3300/0 conn(0x7f80a00cfd50 msgr2=0x7f80a00cee90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2024-04-25T19:17:51.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.810+0000 7f809e7fc640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.76:3300/0 conn(0x7f80a00cfd50 0x7f80a00cee90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T19:17:51.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.811+0000 7f809effd640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f80a0157200 0x7f80a00d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:51.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.811+0000 7f809f7fe640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80a00d0160 0x7f80a00cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:51.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.811+0000 7f809f7fe640 1 -- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80a00d0160 msgr2=0x7f80a00cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T19:17:51.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.811+0000 7f809f7fe640 1 -- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80a00d0160 msgr2=0x7f80a00cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:17:51.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.811+0000 7f809f7fe640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80a00d0160 0x7f80a00cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:51.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.811+0000 7f809effd640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f80a0157200 msgr2=0x7f80a00d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T19:17:51.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.811+0000 7f809effd640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f80a0157200 msgr2=0x7f80a00d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:17:51.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.811+0000 7f809effd640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f80a0157200 0x7f80a00d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:51.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.811+0000 7f809f7fe640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80a00d0160 0x7f80a00cf3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T19:17:51.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:51.811+0000 7f809effd640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f80a0157200 0x7f80a00d0b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T19:17:52.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:52.611+0000 7f809e7fc640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.76:3300/0 conn(0x7f80a00cfd50 msgr2=0x7f80a00cee90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2024-04-25T19:17:52.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:52.611+0000 7f809e7fc640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.76:3300/0 conn(0x7f80a00cfd50 0x7f80a00cee90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T19:17:52.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:52.612+0000 7f809effd640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f80a0157200 0x7f80a00d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:52.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:52.612+0000 7f809effd640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f80a0157200 msgr2=0x7f80a00d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T19:17:52.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:52.612+0000 7f809effd640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f80a0157200 msgr2=0x7f80a00d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:17:52.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:52.612+0000 7f809effd640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f80a0157200 0x7f80a00d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:52.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:52.612+0000 7f809effd640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f80a0157200 0x7f80a00d0b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T19:17:52.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:52.612+0000 7f809f7fe640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80a00d0160 0x7f80a00cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:52.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:52.612+0000 7f809f7fe640 1 -- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80a00d0160 msgr2=0x7f80a00cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T19:17:52.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:52.612+0000 7f809f7fe640 1 -- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80a00d0160 msgr2=0x7f80a00cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:17:52.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:52.612+0000 7f809f7fe640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80a00d0160 0x7f80a00cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:52.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:52.612+0000 7f809f7fe640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80a00d0160 0x7f80a00cf3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T19:17:54.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:54.209+0000 7f809cff9640 1 -- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80a00d0160 msgr2=0x7f80a00cf3d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:17:54.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:54.209+0000 7f809cff9640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f80a00d0160 0x7f80a00cf3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:54.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:54.209+0000 7f809cff9640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f80a0157200 msgr2=0x7f80a00d0b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:17:54.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:54.209+0000 7f809cff9640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f80a0157200 0x7f80a00d0b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:54.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:54.209+0000 7f809cff9640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.76:3300/0 conn(0x7f80a00cfd50 msgr2=0x7f80a00cee90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:17:54.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:54.209+0000 7f809cff9640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.76:3300/0 conn(0x7f80a00cfd50 0x7f80a00cee90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:17:54.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:54.209+0000 7f809cff9640 1 -- 172.21.15.76:0/2185088306 --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80a015d460 con 0x7f808c003690 2024-04-25T19:17:54.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:54.209+0000 7f809cff9640 1 -- 172.21.15.76:0/2185088306 --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f808c00c1c0 con 0x7f808c007400 2024-04-25T19:17:54.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:54.209+0000 7f809effd640 1 -- 172.21.15.76:0/2185088306 >> v1:172.21.15.76:6789/0 conn(0x7f808c003690 legacy=0x7f808c005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2024-04-25T19:17:54.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:54.410+0000 7f809effd640 1 -- 172.21.15.76:0/2185088306 >> v1:172.21.15.76:6789/0 conn(0x7f808c003690 legacy=0x7f808c005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2024-04-25T19:17:54.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:54.811+0000 7f809effd640 1 -- 172.21.15.76:0/2185088306 >> v1:172.21.15.76:6789/0 conn(0x7f808c003690 legacy=0x7f808c005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2024-04-25T19:17:54.954 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:17:54.954 INFO:tasks.workunit.client.0.smithi076.stdout: Children see throughput for 1 readers = 7557796.50 kB/sec 2024-04-25T19:17:54.954 INFO:tasks.workunit.client.0.smithi076.stdout: Parent sees throughput for 1 readers = 7547187.39 kB/sec 2024-04-25T19:17:54.954 INFO:tasks.workunit.client.0.smithi076.stdout: Min throughput per process = 7557796.50 kB/sec 2024-04-25T19:17:54.954 INFO:tasks.workunit.client.0.smithi076.stdout: Max throughput per process = 7557796.50 kB/sec 2024-04-25T19:17:54.954 INFO:tasks.workunit.client.0.smithi076.stdout: Avg throughput per process = 7557796.50 kB/sec 2024-04-25T19:17:54.954 INFO:tasks.workunit.client.0.smithi076.stdout: Min xfer = 1048576.00 kB 2024-04-25T19:17:55.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:55.612+0000 7f809effd640 1 -- 172.21.15.76:0/2185088306 >> v1:172.21.15.76:6789/0 conn(0x7f808c003690 legacy=0x7f808c005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2024-04-25T19:17:56.101 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:17:56.104 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:17:57.184 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:17:57.184 INFO:tasks.workunit.client.0.smithi076.stdout: Children see throughput for 1 re-readers = 7664681.50 kB/sec 2024-04-25T19:17:57.184 INFO:tasks.workunit.client.0.smithi076.stdout: Parent sees throughput for 1 re-readers = 7653957.20 kB/sec 2024-04-25T19:17:57.184 INFO:tasks.workunit.client.0.smithi076.stdout: Min throughput per process = 7664681.50 kB/sec 2024-04-25T19:17:57.185 INFO:tasks.workunit.client.0.smithi076.stdout: Max throughput per process = 7664681.50 kB/sec 2024-04-25T19:17:57.185 INFO:tasks.workunit.client.0.smithi076.stdout: Avg throughput per process = 7664681.50 kB/sec 2024-04-25T19:17:57.185 INFO:tasks.workunit.client.0.smithi076.stdout: Min xfer = 1048576.00 kB 2024-04-25T19:17:57.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.209+0000 7f809cff9640 1 -- 172.21.15.76:0/2185088306 >> v1:172.21.15.105:6789/0 conn(0x7f808c007400 legacy=0x7f808c009910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:17:57.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.209+0000 7f809cff9640 1 -- 172.21.15.76:0/2185088306 >> v1:172.21.15.76:6789/0 conn(0x7f808c003690 legacy=0x7f808c005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:17:57.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.209+0000 7f809f7fe640 1 -- 172.21.15.76:0/2185088306 reap_dead start 2024-04-25T19:17:57.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.209+0000 7f809cff9640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.76:3300/0 conn(0x7f808c00c4e0 0x7f808c009840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:57.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.209+0000 7f809cff9640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f808c00c8f0 0x7f808c00cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:57.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.209+0000 7f809cff9640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f808c00b170 0x7f808c00b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:17:57.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.209+0000 7f809cff9640 1 -- 172.21.15.76:0/2185088306 --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f80a015dd00 con 0x7f808c00c4e0 2024-04-25T19:17:57.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.209+0000 7f809cff9640 1 -- 172.21.15.76:0/2185088306 --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f80a015cea0 con 0x7f808c00c8f0 2024-04-25T19:17:57.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.209+0000 7f809cff9640 1 -- 172.21.15.76:0/2185088306 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f80a015d180 con 0x7f808c00b170 2024-04-25T19:17:57.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.209+0000 7f809effd640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.76:3300/0 conn(0x7f808c00c4e0 msgr2=0x7f808c009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2024-04-25T19:17:57.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.209+0000 7f809effd640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.76:3300/0 conn(0x7f808c00c4e0 0x7f808c009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:17:57.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.209+0000 7f809f7fe640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f808c00b170 0x7f808c00b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:57.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.209+0000 7f809e7fc640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f808c00c8f0 0x7f808c00cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:57.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.209+0000 7f809f7fe640 1 -- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f808c00b170 msgr2=0x7f808c00b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T19:17:57.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.209+0000 7f809f7fe640 1 -- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f808c00b170 msgr2=0x7f808c00b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:17:57.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.209+0000 7f809e7fc640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f808c00c8f0 msgr2=0x7f808c00cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T19:17:57.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.209+0000 7f809f7fe640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f808c00b170 0x7f808c00b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:57.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.209+0000 7f809e7fc640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f808c00c8f0 msgr2=0x7f808c00cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:17:57.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.209+0000 7f809e7fc640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f808c00c8f0 0x7f808c00cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:57.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.209+0000 7f809e7fc640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f808c00c8f0 0x7f808c00cd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:17:57.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.209+0000 7f809f7fe640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f808c00b170 0x7f808c00b580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:17:57.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.410+0000 7f809effd640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.76:3300/0 conn(0x7f808c00c4e0 msgr2=0x7f808c009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2024-04-25T19:17:57.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.410+0000 7f809effd640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.76:3300/0 conn(0x7f808c00c4e0 0x7f808c009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T19:17:57.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.410+0000 7f809f7fe640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f808c00b170 0x7f808c00b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:57.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.410+0000 7f809e7fc640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f808c00c8f0 0x7f808c00cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:57.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.410+0000 7f809f7fe640 1 -- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f808c00b170 msgr2=0x7f808c00b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T19:17:57.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.410+0000 7f809f7fe640 1 -- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f808c00b170 msgr2=0x7f808c00b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:17:57.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.410+0000 7f809f7fe640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f808c00b170 0x7f808c00b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:57.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.410+0000 7f809e7fc640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f808c00c8f0 msgr2=0x7f808c00cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T19:17:57.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.410+0000 7f809e7fc640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f808c00c8f0 msgr2=0x7f808c00cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:17:57.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.410+0000 7f809e7fc640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f808c00c8f0 0x7f808c00cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:57.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.410+0000 7f809f7fe640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f808c00b170 0x7f808c00b580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T19:17:57.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.410+0000 7f809e7fc640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f808c00c8f0 0x7f808c00cd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T19:17:57.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.811+0000 7f809effd640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.76:3300/0 conn(0x7f808c00c4e0 msgr2=0x7f808c009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2024-04-25T19:17:57.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.811+0000 7f809effd640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.76:3300/0 conn(0x7f808c00c4e0 0x7f808c009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T19:17:57.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.811+0000 7f809e7fc640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f808c00c8f0 0x7f808c00cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:57.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.811+0000 7f809f7fe640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f808c00b170 0x7f808c00b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:57.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.811+0000 7f809f7fe640 1 -- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f808c00b170 msgr2=0x7f808c00b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T19:17:57.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.811+0000 7f809f7fe640 1 -- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f808c00b170 msgr2=0x7f808c00b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:17:57.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.811+0000 7f809f7fe640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f808c00b170 0x7f808c00b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:57.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.811+0000 7f809f7fe640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f808c00b170 0x7f808c00b580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T19:17:57.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.811+0000 7f809e7fc640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f808c00c8f0 msgr2=0x7f808c00cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T19:17:57.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.811+0000 7f809e7fc640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f808c00c8f0 msgr2=0x7f808c00cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:17:57.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.811+0000 7f809e7fc640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f808c00c8f0 0x7f808c00cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:57.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:57.811+0000 7f809e7fc640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f808c00c8f0 0x7f808c00cd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T19:17:58.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:58.612+0000 7f809effd640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.76:3300/0 conn(0x7f808c00c4e0 msgr2=0x7f808c009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2024-04-25T19:17:58.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:58.612+0000 7f809effd640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.76:3300/0 conn(0x7f808c00c4e0 0x7f808c009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T19:17:58.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:58.612+0000 7f809f7fe640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f808c00b170 0x7f808c00b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:58.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:58.612+0000 7f809e7fc640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f808c00c8f0 0x7f808c00cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:17:58.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:58.612+0000 7f809f7fe640 1 -- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f808c00b170 msgr2=0x7f808c00b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T19:17:58.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:58.612+0000 7f809f7fe640 1 -- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f808c00b170 msgr2=0x7f808c00b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:17:58.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:58.612+0000 7f809f7fe640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f808c00b170 0x7f808c00b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:58.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:58.612+0000 7f809e7fc640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f808c00c8f0 msgr2=0x7f808c00cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T19:17:58.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:58.612+0000 7f809e7fc640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f808c00c8f0 msgr2=0x7f808c00cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:17:58.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:58.613+0000 7f809e7fc640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f808c00c8f0 0x7f808c00cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:17:58.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:58.613+0000 7f809f7fe640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f808c00b170 0x7f808c00b580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T19:17:58.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:17:58.613+0000 7f809e7fc640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f808c00c8f0 0x7f808c00cd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T19:18:00.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.210+0000 7f809cff9640 1 -- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f808c00b170 msgr2=0x7f808c00b580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:18:00.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.210+0000 7f809cff9640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f808c00b170 0x7f808c00b580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.210+0000 7f809cff9640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f808c00c8f0 msgr2=0x7f808c00cd00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:18:00.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.210+0000 7f809cff9640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.105:3300/0 conn(0x7f808c00c8f0 0x7f808c00cd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.210+0000 7f809cff9640 1 -- 172.21.15.76:0/2185088306 >> v2:172.21.15.76:3300/0 conn(0x7f808c00c4e0 msgr2=0x7f808c009840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:18:00.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.210+0000 7f809cff9640 1 --2- 172.21.15.76:0/2185088306 >> v2:172.21.15.76:3300/0 conn(0x7f808c00c4e0 0x7f808c009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.210+0000 7f809cff9640 1 -- 172.21.15.76:0/2185088306 --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f808c00d4c0 con 0x7f808c00a040 2024-04-25T19:18:00.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.210+0000 7f809cff9640 1 -- 172.21.15.76:0/2185088306 --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80a015d460 con 0x7f808c00ab60 2024-04-25T19:18:00.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.210+0000 7f809effd640 1 -- 172.21.15.76:0/2185088306 >> v1:172.21.15.76:6789/0 conn(0x7f808c00a040 legacy=0x7f808c00a450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2024-04-25T19:18:00.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.210+0000 7f809dffb640 1 -- 172.21.15.76:0/2185088306 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3718874117 0 0) 0x7f80a015d460 con 0x7f808c00ab60 2024-04-25T19:18:00.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.210+0000 7f809dffb640 1 -- 172.21.15.76:0/2185088306 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8088003520 con 0x7f808c00ab60 2024-04-25T19:18:00.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.210+0000 7f809dffb640 1 -- 172.21.15.76:0/2185088306 <== mon.1 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 186393933 0 0) 0x7f8088003520 con 0x7f808c00ab60 2024-04-25T19:18:00.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.210+0000 7f809dffb640 1 -- 172.21.15.76:0/2185088306 >> v1:172.21.15.76:6789/0 conn(0x7f808c00a040 legacy=0x7f808c00a450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:18:00.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.210+0000 7f809dffb640 1 -- 172.21.15.76:0/2185088306 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808c016850 con 0x7f808c00ab60 2024-04-25T19:18:00.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.210+0000 7f809dffb640 1 -- 172.21.15.76:0/2185088306 <== mon.1 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f8094009870 con 0x7f808c00ab60 2024-04-25T19:18:00.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.210+0000 7f809dffb640 1 -- 172.21.15.76:0/2185088306 >> v1:172.21.15.105:6789/0 conn(0x7f808c00ab60 legacy=0x7f808c006b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:00.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.210+0000 7f809f7fe640 1 -- 172.21.15.76:0/2185088306 reap_dead start 2024-04-25T19:18:00.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.210+0000 7f809dffb640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80880039b0 0x7f8088003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:00.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.210+0000 7f809dffb640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8088004800 0x7f8088004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:00.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.210+0000 7f809dffb640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8088005350 0x7f808800ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:00.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.210+0000 7f809dffb640 1 -- 172.21.15.76:0/2185088306 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f808800c2b0 con 0x7f8088004800 2024-04-25T19:18:00.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.210+0000 7f809dffb640 1 -- 172.21.15.76:0/2185088306 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f808800c460 con 0x7f80880039b0 2024-04-25T19:18:00.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.210+0000 7f809dffb640 1 -- 172.21.15.76:0/2185088306 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f808800c660 con 0x7f8088005350 2024-04-25T19:18:00.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.210+0000 7f809e7fc640 1 -- 172.21.15.76:0/2185088306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8088004800 msgr2=0x7f8088004c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2024-04-25T19:18:00.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.210+0000 7f809e7fc640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8088004800 0x7f8088004c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:18:00.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.211+0000 7f809effd640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80880039b0 0x7f8088003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:00.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.211+0000 7f809f7fe640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8088005350 0x7f808800ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:00.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.211+0000 7f809effd640 1 -- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8088005350 msgr2=0x7f808800ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:00.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.211+0000 7f809effd640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8088005350 0x7f808800ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.211+0000 7f809effd640 1 -- 172.21.15.76:0/2185088306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8088004800 msgr2=0x7f8088004c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:18:00.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.211+0000 7f809effd640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8088004800 0x7f8088004c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.211+0000 7f809effd640 1 -- 172.21.15.76:0/2185088306 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8088003520 con 0x7f80880039b0 2024-04-25T19:18:00.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.212+0000 7f809effd640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80880039b0 0x7f8088003de0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8090006fd0 tx=0x7f8088005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa97f41e11daa6d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:00.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.212+0000 7f809dffb640 1 -- 172.21.15.76:0/2185088306 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8090031070 con 0x7f80880039b0 2024-04-25T19:18:00.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.212+0000 7f809dffb640 1 -- 172.21.15.76:0/2185088306 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f809002b7f0 con 0x7f80880039b0 2024-04-25T19:18:00.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.212+0000 7f809dffb640 1 -- 172.21.15.76:0/2185088306 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80900044b0 con 0x7f80880039b0 2024-04-25T19:18:00.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.213+0000 7f80a5892640 1 -- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80880039b0 msgr2=0x7f8088003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:00.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.213+0000 7f80a5892640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80880039b0 0x7f8088003de0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8090006fd0 tx=0x7f8088005830 comp rx=0 tx=0).stop 2024-04-25T19:18:00.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.213+0000 7f80a5892640 1 -- 172.21.15.76:0/2185088306 shutdown_connections 2024-04-25T19:18:00.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.213+0000 7f80a5892640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8088005350 0x7f808800ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.213+0000 7f80a5892640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8088004800 0x7f8088004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.213+0000 7f80a5892640 1 --2- 172.21.15.76:0/2185088306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80880039b0 0x7f8088003de0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.213+0000 7f80a5892640 1 -- 172.21.15.76:0/2185088306 >> 172.21.15.76:0/2185088306 conn(0x7f80a00d2370 msgr2=0x7f80a00d2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:00.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.213+0000 7f80a5892640 1 -- 172.21.15.76:0/2185088306 shutdown_connections 2024-04-25T19:18:00.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.213+0000 7f80a5892640 1 -- 172.21.15.76:0/2185088306 wait complete. 2024-04-25T19:18:00.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.215+0000 7f80a5892640 1 Processor -- start 2024-04-25T19:18:00.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.216+0000 7f80a5892640 1 -- start start 2024-04-25T19:18:00.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.217+0000 7f80a5892640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80880039b0 0x7f80a01f6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:00.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.217+0000 7f80a5892640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8088004800 0x7f80a01e6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:00.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.217+0000 7f80a5892640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8088005350 0x7f80a0152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:00.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.217+0000 7f80a5892640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f80a00d0b10 con 0x7f80880039b0 2024-04-25T19:18:00.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.217+0000 7f80a5892640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f80a015da20 con 0x7f8088004800 2024-04-25T19:18:00.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.217+0000 7f80a5892640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f80a00d0c90 con 0x7f8088005350 2024-04-25T19:18:00.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.217+0000 7f809effd640 1 -- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80880039b0 msgr2=0x7f80a01f6280 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:3300/0 2024-04-25T19:18:00.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.217+0000 7f809effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80880039b0 0x7f80a01f6280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:18:00.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.217+0000 7f809e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8088004800 0x7f80a01e6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:00.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.217+0000 7f809f7fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8088005350 0x7f80a0152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:00.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.217+0000 7f809e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8088004800 0x7f80a01e6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:59222/0 (socket says 172.21.15.76:59222) 2024-04-25T19:18:00.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.217+0000 7f809e7fc640 1 -- 172.21.15.76:0/1983579403 learned_addr learned my addr 172.21.15.76:0/1983579403 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:00.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.218+0000 7f809e7fc640 1 -- 172.21.15.76:0/1983579403 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8088005350 msgr2=0x7f80a0152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:00.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.218+0000 7f809e7fc640 1 --2- 172.21.15.76:0/1983579403 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8088005350 0x7f80a0152950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.218+0000 7f809e7fc640 1 -- 172.21.15.76:0/1983579403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80880039b0 msgr2=0x7f80a01f6280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:18:00.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.218+0000 7f809e7fc640 1 --2- 172.21.15.76:0/1983579403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80880039b0 0x7f80a01f6280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.218+0000 7f809e7fc640 1 -- 172.21.15.76:0/1983579403 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a01f67c0 con 0x7f8088004800 2024-04-25T19:18:00.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.218+0000 7f809e7fc640 1 --2- 172.21.15.76:0/1983579403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8088004800 0x7f80a01e6270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f809400cdd0 tx=0x7f809400b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:00.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.218+0000 7f8087fff640 1 -- 172.21.15.76:0/1983579403 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809401c070 con 0x7f8088004800 2024-04-25T19:18:00.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.218+0000 7f8087fff640 1 -- 172.21.15.76:0/1983579403 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8094017600 con 0x7f8088004800 2024-04-25T19:18:00.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.218+0000 7f80a5892640 1 -- 172.21.15.76:0/1983579403 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a0154ce0 con 0x7f8088004800 2024-04-25T19:18:00.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.218+0000 7f8087fff640 1 -- 172.21.15.76:0/1983579403 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80940179f0 con 0x7f8088004800 2024-04-25T19:18:00.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.218+0000 7f80a5892640 1 -- 172.21.15.76:0/1983579403 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a01f3450 con 0x7f8088004800 2024-04-25T19:18:00.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.220+0000 7f80a5892640 1 -- 172.21.15.76:0/1983579403 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80a006a390 con 0x7f8088004800 2024-04-25T19:18:00.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.220+0000 7f8087fff640 1 -- 172.21.15.76:0/1983579403 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f809401f070 con 0x7f8088004800 2024-04-25T19:18:00.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.221+0000 7f8087fff640 1 --2- 172.21.15.76:0/1983579403 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f808c075e40 0x7f808c078350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:00.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.221+0000 7f8087fff640 1 -- 172.21.15.76:0/1983579403 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f8094095650 con 0x7f8088004800 2024-04-25T19:18:00.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.221+0000 7f809effd640 1 --2- 172.21.15.76:0/1983579403 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f808c075e40 0x7f808c078350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:00.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.221+0000 7f809effd640 1 --2- 172.21.15.76:0/1983579403 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f808c075e40 0x7f808c078350 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8090036000 tx=0x7f8090031000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:00.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.226+0000 7f8087fff640 1 -- 172.21.15.76:0/1983579403 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f809405f710 con 0x7f8088004800 2024-04-25T19:18:00.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.474+0000 7f80a5892640 1 -- 172.21.15.76:0/1983579403 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f80a015ce00 con 0x7f8088004800 2024-04-25T19:18:00.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.474+0000 7f8087fff640 1 -- 172.21.15.76:0/1983579403 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f8094063a10 con 0x7f8088004800 2024-04-25T19:18:00.476 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":20,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:18:00.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.476+0000 7f80a5892640 1 -- 172.21.15.76:0/1983579403 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f808c075e40 msgr2=0x7f808c078350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:00.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.476+0000 7f80a5892640 1 --2- 172.21.15.76:0/1983579403 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f808c075e40 0x7f808c078350 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8090036000 tx=0x7f8090031000 comp rx=0 tx=0).stop 2024-04-25T19:18:00.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.476+0000 7f80a5892640 1 -- 172.21.15.76:0/1983579403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8088004800 msgr2=0x7f80a01e6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:00.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.476+0000 7f80a5892640 1 --2- 172.21.15.76:0/1983579403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8088004800 0x7f80a01e6270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f809400cdd0 tx=0x7f809400b7a0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.476+0000 7f80a5892640 1 -- 172.21.15.76:0/1983579403 shutdown_connections 2024-04-25T19:18:00.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.476+0000 7f80a5892640 1 --2- 172.21.15.76:0/1983579403 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f808c075e40 0x7f808c078350 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.476+0000 7f80a5892640 1 --2- 172.21.15.76:0/1983579403 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8088005350 0x7f80a0152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.476+0000 7f80a5892640 1 --2- 172.21.15.76:0/1983579403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8088004800 0x7f80a01e6270 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.476+0000 7f80a5892640 1 --2- 172.21.15.76:0/1983579403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80880039b0 0x7f80a01f6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.476+0000 7f80a5892640 1 -- 172.21.15.76:0/1983579403 >> 172.21.15.76:0/1983579403 conn(0x7f80a00d2370 msgr2=0x7f80a015bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:00.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.477+0000 7f80a5892640 1 -- 172.21.15.76:0/1983579403 shutdown_connections 2024-04-25T19:18:00.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.477+0000 7f80a5892640 1 -- 172.21.15.76:0/1983579403 wait complete. 2024-04-25T19:18:00.489 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-04-25T19:18:00.489 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-04-25T19:18:00.489 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T19:18:00.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.597+0000 7ff767146640 1 Processor -- start 2024-04-25T19:18:00.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.597+0000 7ff767146640 1 -- start start 2024-04-25T19:18:00.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.598+0000 7ff767146640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff7600d0810 0x7ff76006f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:00.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.598+0000 7ff767146640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7600cea70 0x7ff7600cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:00.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.598+0000 7ff767146640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff76015ce80 con 0x7ff76006e160 2024-04-25T19:18:00.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.598+0000 7ff767146640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7ff76015d160 con 0x7ff7600d0810 2024-04-25T19:18:00.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.598+0000 7ff767146640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff76015d440 con 0x7ff7600cea70 2024-04-25T19:18:00.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.599+0000 7ff764ee2640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff7600d0810 0x7ff76006f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:00.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.599+0000 7ff757fff640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7600cea70 0x7ff7600cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:00.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.599+0000 7ff764ee2640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff7600d0810 0x7ff76006f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:59234/0 (socket says 172.21.15.76:59234) 2024-04-25T19:18:00.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.599+0000 7ff764ee2640 1 -- 172.21.15.76:0/4209036631 learned_addr learned my addr 172.21.15.76:0/4209036631 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:00.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.599+0000 7ff757fff640 1 -- 172.21.15.76:0/4209036631 >> v2:172.21.15.105:3300/0 conn(0x7ff7600d0810 msgr2=0x7ff76006f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:00.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.599+0000 7ff757fff640 1 --2- 172.21.15.76:0/4209036631 >> v2:172.21.15.105:3300/0 conn(0x7ff7600d0810 0x7ff76006f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.599+0000 7ff757fff640 1 -- 172.21.15.76:0/4209036631 >> v1:172.21.15.105:6789/0 conn(0x7ff76006e160 legacy=0x7ff76006e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:00.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.599+0000 7ff757fff640 1 -- 172.21.15.76:0/4209036631 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76015d720 con 0x7ff7600cea70 2024-04-25T19:18:00.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.599+0000 7ff764ee2640 1 --2- 172.21.15.76:0/4209036631 >> v2:172.21.15.105:3300/0 conn(0x7ff7600d0810 0x7ff76006f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:18:00.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.600+0000 7ff757fff640 1 --2- 172.21.15.76:0/4209036631 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7600cea70 0x7ff7600cee80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff7500097a0 tx=0x7ff75002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53f5bb890400ad52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:00.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.600+0000 7ff7577fe640 1 -- 172.21.15.76:0/4209036631 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750037070 con 0x7ff7600cea70 2024-04-25T19:18:00.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.600+0000 7ff7577fe640 1 -- 172.21.15.76:0/4209036631 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff750032440 con 0x7ff7600cea70 2024-04-25T19:18:00.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.600+0000 7ff7577fe640 1 -- 172.21.15.76:0/4209036631 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750032810 con 0x7ff7600cea70 2024-04-25T19:18:00.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.601+0000 7ff767146640 1 -- 172.21.15.76:0/4209036631 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7600cea70 msgr2=0x7ff7600cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:00.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.601+0000 7ff767146640 1 --2- 172.21.15.76:0/4209036631 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7600cea70 0x7ff7600cee80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff7500097a0 tx=0x7ff75002a180 comp rx=0 tx=0).stop 2024-04-25T19:18:00.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.601+0000 7ff767146640 1 -- 172.21.15.76:0/4209036631 shutdown_connections 2024-04-25T19:18:00.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.601+0000 7ff767146640 1 --2- 172.21.15.76:0/4209036631 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7600cea70 0x7ff7600cee80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.601+0000 7ff767146640 1 --2- 172.21.15.76:0/4209036631 >> v2:172.21.15.105:3300/0 conn(0x7ff7600d0810 0x7ff76006f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.601+0000 7ff767146640 1 -- 172.21.15.76:0/4209036631 >> 172.21.15.76:0/4209036631 conn(0x7ff7600b3c00 msgr2=0x7ff7600b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:00.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.601+0000 7ff767146640 1 -- 172.21.15.76:0/4209036631 shutdown_connections 2024-04-25T19:18:00.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.601+0000 7ff767146640 1 -- 172.21.15.76:0/4209036631 wait complete. 2024-04-25T19:18:00.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.602+0000 7ff767146640 1 Processor -- start 2024-04-25T19:18:00.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.602+0000 7ff767146640 1 -- start start 2024-04-25T19:18:00.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.604+0000 7ff767146640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff76006e160 0x7ff760150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:00.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.604+0000 7ff767146640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7600cea70 0x7ff760157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:00.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.604+0000 7ff767146640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7600d0810 0x7ff760157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:00.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.604+0000 7ff767146640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff76015cb70 con 0x7ff76006e160 2024-04-25T19:18:00.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.604+0000 7ff767146640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff76006dc40 con 0x7ff7600cea70 2024-04-25T19:18:00.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.604+0000 7ff767146640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff76006f540 con 0x7ff7600d0810 2024-04-25T19:18:00.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.604+0000 7ff764ee2640 1 -- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff76006e160 msgr2=0x7ff760150a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:3300/0 2024-04-25T19:18:00.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.604+0000 7ff764ee2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff76006e160 0x7ff760150a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:18:00.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.604+0000 7ff757fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7600cea70 0x7ff760157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:00.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.604+0000 7ff7656e3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7600d0810 0x7ff760157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:00.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.604+0000 7ff7656e3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7600d0810 0x7ff760157980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:48930/0 (socket says 172.21.15.76:48930) 2024-04-25T19:18:00.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.604+0000 7ff7656e3640 1 -- 172.21.15.76:0/828485530 learned_addr learned my addr 172.21.15.76:0/828485530 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:00.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.604+0000 7ff7656e3640 1 -- 172.21.15.76:0/828485530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7600cea70 msgr2=0x7ff760157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:00.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.604+0000 7ff7656e3640 1 --2- 172.21.15.76:0/828485530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7600cea70 0x7ff760157440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.604+0000 7ff7656e3640 1 -- 172.21.15.76:0/828485530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff76006e160 msgr2=0x7ff760150a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:18:00.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.604+0000 7ff7656e3640 1 --2- 172.21.15.76:0/828485530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff76006e160 0x7ff760150a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.604+0000 7ff7656e3640 1 -- 172.21.15.76:0/828485530 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76015dd30 con 0x7ff7600d0810 2024-04-25T19:18:00.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.604+0000 7ff757fff640 1 --2- 172.21.15.76:0/828485530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7600cea70 0x7ff760157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:18:00.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.604+0000 7ff7656e3640 1 --2- 172.21.15.76:0/828485530 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7600d0810 0x7ff760157980 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff75800c730 tx=0x7ff75800cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:00.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.605+0000 7ff7557fa640 1 -- 172.21.15.76:0/828485530 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758003b90 con 0x7ff7600d0810 2024-04-25T19:18:00.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.605+0000 7ff7557fa640 1 -- 172.21.15.76:0/828485530 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff758003d30 con 0x7ff7600d0810 2024-04-25T19:18:00.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.605+0000 7ff767146640 1 -- 172.21.15.76:0/828485530 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff76015ebd0 con 0x7ff7600d0810 2024-04-25T19:18:00.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.605+0000 7ff7557fa640 1 -- 172.21.15.76:0/828485530 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758005310 con 0x7ff7600d0810 2024-04-25T19:18:00.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.605+0000 7ff767146640 1 -- 172.21.15.76:0/828485530 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7601f3480 con 0x7ff7600d0810 2024-04-25T19:18:00.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.605+0000 7ff767146640 1 --2- 172.21.15.76:0/828485530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7600b7e50 0x7ff76015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:00.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.606+0000 7ff767146640 1 -- 172.21.15.76:0/828485530 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff7601f3000 con 0x7ff7600b7e50 2024-04-25T19:18:00.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.606+0000 7ff764ee2640 1 --2- 172.21.15.76:0/828485530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7600b7e50 0x7ff76015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:00.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.606+0000 7ff764ee2640 1 -- 172.21.15.76:0/828485530 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff76015e960 con 0x7ff7600b7e50 2024-04-25T19:18:00.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.606+0000 7ff764ee2640 1 --2- 172.21.15.76:0/828485530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7600b7e50 0x7ff76015f5a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff75c006c80 tx=0x7ff75c00eb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:00.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.606+0000 7ff7557fa640 1 -- 172.21.15.76:0/828485530 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c017070 con 0x7ff7600b7e50 2024-04-25T19:18:00.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.606+0000 7ff7557fa640 1 -- 172.21.15.76:0/828485530 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff7580056c0 con 0x7ff7600d0810 2024-04-25T19:18:00.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.607+0000 7ff7557fa640 1 --2- 172.21.15.76:0/828485530 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff74c0743b0 0x7ff74c0768c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:00.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.607+0000 7ff7557fa640 1 -- 172.21.15.76:0/828485530 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7ff758095270 con 0x7ff7600d0810 2024-04-25T19:18:00.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.607+0000 7ff757fff640 1 --2- 172.21.15.76:0/828485530 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff74c0743b0 0x7ff74c0768c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:00.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.607+0000 7ff7557fa640 1 -- 172.21.15.76:0/828485530 <== mon.1 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff75c00c770 con 0x7ff7600b7e50 2024-04-25T19:18:00.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.607+0000 7ff7557fa640 1 -- 172.21.15.76:0/828485530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7600b7e50 msgr2=0x7ff76015f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:00.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.607+0000 7ff7557fa640 1 --2- 172.21.15.76:0/828485530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7600b7e50 0x7ff76015f5a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff75c006c80 tx=0x7ff75c00eb50 comp rx=0 tx=0).stop 2024-04-25T19:18:00.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.607+0000 7ff7557fa640 1 -- 172.21.15.76:0/828485530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7600b7e50 msgr2=0x7ff76015f5a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:18:00.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.607+0000 7ff7557fa640 1 --2- 172.21.15.76:0/828485530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7600b7e50 0x7ff76015f5a0 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff75c006c80 tx=0x7ff75c00eb50 comp rx=0 tx=0).stop 2024-04-25T19:18:00.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.608+0000 7ff757fff640 1 --2- 172.21.15.76:0/828485530 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff74c0743b0 0x7ff74c0768c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff750034000 tx=0x7ff75002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:00.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.613+0000 7ff767146640 1 --2- 172.21.15.76:0/828485530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7601f3c80 0x7ff7601f6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:00.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.613+0000 7ff767146640 1 -- 172.21.15.76:0/828485530 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff76015eeb0 con 0x7ff7601f3c80 2024-04-25T19:18:00.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.614+0000 7ff764ee2640 1 --2- 172.21.15.76:0/828485530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7601f3c80 0x7ff7601f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:00.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.614+0000 7ff764ee2640 1 -- 172.21.15.76:0/828485530 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff76015fae0 con 0x7ff7601f3c80 2024-04-25T19:18:00.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.614+0000 7ff764ee2640 1 --2- 172.21.15.76:0/828485530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7601f3c80 0x7ff7601f6540 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff75c019000 tx=0x7ff75c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:00.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.615+0000 7ff7557fa640 1 -- 172.21.15.76:0/828485530 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c006370 con 0x7ff7601f3c80 2024-04-25T19:18:00.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.615+0000 7ff7557fa640 1 -- 172.21.15.76:0/828485530 <== mon.1 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5055 (secure 0 0 0) 0x7ff75c006370 con 0x7ff7601f3c80 2024-04-25T19:18:00.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.615+0000 7ff7557fa640 1 -- 172.21.15.76:0/828485530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7601f3c80 msgr2=0x7ff7601f6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:00.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.615+0000 7ff7557fa640 1 --2- 172.21.15.76:0/828485530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7601f3c80 0x7ff7601f6540 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff75c019000 tx=0x7ff75c004730 comp rx=0 tx=0).stop 2024-04-25T19:18:00.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.615+0000 7ff7557fa640 1 -- 172.21.15.76:0/828485530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7601f3c80 msgr2=0x7ff7601f6540 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:18:00.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.615+0000 7ff7557fa640 1 --2- 172.21.15.76:0/828485530 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7601f3c80 0x7ff7601f6540 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.617 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:18:00.617 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:18:00.617 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:18:00.617 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "leader", 2024-04-25T19:18:00.617 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 20, 2024-04-25T19:18:00.617 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 32106, 2024-04-25T19:18:00.617 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:18:00.617 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-04-25T19:18:00.617 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:18:00.617 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:00.617 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 0, 2024-04-25T19:18:00.617 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:18:00.618 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:18:00.618 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:18:00.618 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:00.618 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:00.618 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:00.618 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:00.618 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:00.618 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:00.618 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:00.618 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:00.618 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:00.618 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:00.618 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:00.618 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:00.618 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:18:00.618 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:18:00.619 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:00.619 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:00.619 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:00.619 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:00.619 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:00.619 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:00.619 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:00.619 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:00.619 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:00.619 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:00.619 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:00.619 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:00.619 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:00.619 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:18:00.619 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:18:00.620 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:18:00.620 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:18:00.620 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:18:00.620 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:18:00.620 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:18:00.620 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:18:00.620 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:18:00.620 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:18:00.620 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:18:00.620 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:18:00.620 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:18:00.620 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:18:00.620 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:18:00.620 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:18:00.620 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:18:00.620 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:00.621 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:00.621 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:00.621 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:00.621 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:00.621 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:00.621 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:00.621 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:00.621 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:00.621 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:00.621 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:00.621 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:00.621 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:18:00.621 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:00.621 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:18:00.621 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.622 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:18:00.622 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:18:00.622 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:00.622 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:00.622 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.622 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:00.622 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:18:00.622 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:00.622 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:00.622 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.622 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:00.622 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:18:00.622 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:00.622 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:00.622 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:00.623 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:00.623 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:18:00.623 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:18:00.623 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:00.623 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:00.623 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:00.623 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:00.623 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.623 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:18:00.623 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:18:00.623 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:00.623 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:00.623 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.623 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:00.623 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:18:00.624 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:00.624 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:00.624 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.624 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:00.624 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:18:00.624 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:00.624 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:00.624 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:00.624 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:00.624 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:18:00.624 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:18:00.624 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:00.624 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:00.624 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:00.624 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:00.625 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.625 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:18:00.625 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:18:00.625 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:00.625 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:00.625 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.625 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:00.625 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:18:00.625 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:00.625 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:00.625 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.625 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:00.625 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:18:00.625 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:00.625 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:00.625 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:00.626 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:00.626 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:18:00.626 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:18:00.626 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:00.626 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:00.626 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:00.626 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:00.626 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:00.626 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:00.626 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:18:00.626 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:18:00.626 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.626 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:00.627 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:00.627 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:18:00.627 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:00.627 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:00.627 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": [ 2024-04-25T19:18:00.627 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.627 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:00.627 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:00.627 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 3 2024-04-25T19:18:00.628 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:00.628 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:00.628 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": [ 2024-04-25T19:18:00.628 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.628 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:00.628 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:00.628 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 3 2024-04-25T19:18:00.628 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:00.628 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:00.628 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:18:00.628 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.628 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:00.628 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:00.628 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 3 2024-04-25T19:18:00.628 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:00.628 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:00.629 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": [ 2024-04-25T19:18:00.629 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.629 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:00.629 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:00.629 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 2 2024-04-25T19:18:00.629 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:00.629 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:00.629 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:00.629 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:18:00.629 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:18:00.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.615+0000 7ff767146640 1 -- 172.21.15.76:0/828485530 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff74c0743b0 msgr2=0x7ff74c0768c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:00.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.615+0000 7ff767146640 1 --2- 172.21.15.76:0/828485530 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff74c0743b0 0x7ff74c0768c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff750034000 tx=0x7ff75002e000 comp rx=0 tx=0).stop 2024-04-25T19:18:00.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.615+0000 7ff767146640 1 -- 172.21.15.76:0/828485530 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7600d0810 msgr2=0x7ff760157980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:00.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.615+0000 7ff767146640 1 --2- 172.21.15.76:0/828485530 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff7600d0810 0x7ff760157980 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff75800c730 tx=0x7ff75800cbf0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.615+0000 7ff7656e3640 1 -- 172.21.15.76:0/828485530 reap_dead start 2024-04-25T19:18:00.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.615+0000 7ff7656e3640 1 -- 172.21.15.76:0/828485530 reap_dead start 2024-04-25T19:18:00.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.615+0000 7ff767146640 1 -- 172.21.15.76:0/828485530 shutdown_connections 2024-04-25T19:18:00.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.615+0000 7ff767146640 1 -- 172.21.15.76:0/828485530 >> 172.21.15.76:0/828485530 conn(0x7ff7600b3c00 msgr2=0x7ff7600b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:00.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.615+0000 7ff767146640 1 -- 172.21.15.76:0/828485530 shutdown_connections 2024-04-25T19:18:00.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.615+0000 7ff767146640 1 -- 172.21.15.76:0/828485530 wait complete. 2024-04-25T19:18:00.631 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T19:18:00.631 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-04-25T19:18:00.631 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T19:18:00.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.738+0000 7f63cc97a640 1 Processor -- start 2024-04-25T19:18:00.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.738+0000 7f63cc97a640 1 -- start start 2024-04-25T19:18:00.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.739+0000 7f63cc97a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f63c80d0810 0x7f63c806f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:00.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.739+0000 7f63cc97a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f63c806e870 0x7f63c80cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:00.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.739+0000 7f63cc97a640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63c815ce80 con 0x7f63c80cea70 2024-04-25T19:18:00.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.739+0000 7f63cc97a640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f63c815d160 con 0x7f63c80d0810 2024-04-25T19:18:00.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.739+0000 7f63cc97a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f63c815d440 con 0x7f63c806e870 2024-04-25T19:18:00.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.739+0000 7f63c5d9b640 1 -- >> v1:172.21.15.76:6789/0 conn(0x7f63c80cea70 legacy=0x7f63c806e160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.76:6789/0 2024-04-25T19:18:00.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.739+0000 7f63c659c640 1 -- >> v2:172.21.15.76:3300/0 conn(0x7f63c80d0810 msgr2=0x7f63c806f000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.76:3300/0 2024-04-25T19:18:00.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.739+0000 7f63c659c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f63c80d0810 0x7f63c806f000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:18:00.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.739+0000 7f63c6d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f63c806e870 0x7f63c80cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:00.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.740+0000 7f63c6d9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f63c806e870 0x7f63c80cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:59278/0 (socket says 172.21.15.76:59278) 2024-04-25T19:18:00.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.740+0000 7f63c6d9d640 1 -- 172.21.15.76:0/229071518 learned_addr learned my addr 172.21.15.76:0/229071518 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:00.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.740+0000 7f63c6d9d640 1 -- 172.21.15.76:0/229071518 >> v2:172.21.15.76:3300/0 conn(0x7f63c80d0810 msgr2=0x7f63c806f000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:18:00.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.740+0000 7f63c6d9d640 1 --2- 172.21.15.76:0/229071518 >> v2:172.21.15.76:3300/0 conn(0x7f63c80d0810 0x7f63c806f000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.740+0000 7f63c6d9d640 1 -- 172.21.15.76:0/229071518 >> v1:172.21.15.76:6789/0 conn(0x7f63c80cea70 legacy=0x7f63c806e160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:18:00.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.740+0000 7f63c6d9d640 1 -- 172.21.15.76:0/229071518 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c815d720 con 0x7f63c806e870 2024-04-25T19:18:00.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.741+0000 7f63c6d9d640 1 --2- 172.21.15.76:0/229071518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63c806e870 0x7f63c80cee80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f63c8090090 tx=0x7f63c002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69b97d167aa260b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:00.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.741+0000 7f63c559a640 1 -- 172.21.15.76:0/229071518 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c0036070 con 0x7f63c806e870 2024-04-25T19:18:00.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.741+0000 7f63c559a640 1 -- 172.21.15.76:0/229071518 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63c0031600 con 0x7f63c806e870 2024-04-25T19:18:00.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.741+0000 7f63c559a640 1 -- 172.21.15.76:0/229071518 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c00319d0 con 0x7f63c806e870 2024-04-25T19:18:00.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.742+0000 7f63cc97a640 1 -- 172.21.15.76:0/229071518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63c806e870 msgr2=0x7f63c80cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:00.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.742+0000 7f63cc97a640 1 --2- 172.21.15.76:0/229071518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63c806e870 0x7f63c80cee80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f63c8090090 tx=0x7f63c002a710 comp rx=0 tx=0).stop 2024-04-25T19:18:00.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.742+0000 7f63cc97a640 1 -- 172.21.15.76:0/229071518 shutdown_connections 2024-04-25T19:18:00.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.742+0000 7f63cc97a640 1 --2- 172.21.15.76:0/229071518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63c806e870 0x7f63c80cee80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.742+0000 7f63cc97a640 1 --2- 172.21.15.76:0/229071518 >> v2:172.21.15.76:3300/0 conn(0x7f63c80d0810 0x7f63c806f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.742+0000 7f63cc97a640 1 -- 172.21.15.76:0/229071518 >> 172.21.15.76:0/229071518 conn(0x7f63c80b3c00 msgr2=0x7f63c80b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:00.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.742+0000 7f63cc97a640 1 -- 172.21.15.76:0/229071518 shutdown_connections 2024-04-25T19:18:00.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.742+0000 7f63cc97a640 1 -- 172.21.15.76:0/229071518 wait complete. 2024-04-25T19:18:00.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.744+0000 7f63cc97a640 1 Processor -- start 2024-04-25T19:18:00.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.744+0000 7f63cc97a640 1 -- start start 2024-04-25T19:18:00.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.745+0000 7f63cc97a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63c806e870 0x7f63c8152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:00.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.745+0000 7f63cc97a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63c80cea70 0x7f63c8158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:00.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.745+0000 7f63cc97a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63c80d0810 0x7f63c81516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:00.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.745+0000 7f63cc97a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f63c815cb70 con 0x7f63c806e870 2024-04-25T19:18:00.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.745+0000 7f63cc97a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f63c806dc40 con 0x7f63c80d0810 2024-04-25T19:18:00.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.745+0000 7f63cc97a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f63c806f540 con 0x7f63c80cea70 2024-04-25T19:18:00.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.745+0000 7f63c659c640 1 -- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63c806e870 msgr2=0x7f63c8152900 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.76:3300/0 2024-04-25T19:18:00.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.745+0000 7f63c659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63c806e870 0x7f63c8152900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:18:00.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.745+0000 7f63c6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63c80d0810 0x7f63c81516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:00.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.745+0000 7f63c5d9b640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63c80cea70 0x7f63c8158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:00.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.745+0000 7f63c6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63c80d0810 0x7f63c81516b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:59282/0 (socket says 172.21.15.76:59282) 2024-04-25T19:18:00.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.745+0000 7f63c6d9d640 1 -- 172.21.15.76:0/3405094844 learned_addr learned my addr 172.21.15.76:0/3405094844 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:00.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.745+0000 7f63c6d9d640 1 -- 172.21.15.76:0/3405094844 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63c80cea70 msgr2=0x7f63c8158290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:00.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.745+0000 7f63c6d9d640 1 --2- 172.21.15.76:0/3405094844 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63c80cea70 0x7f63c8158290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.745+0000 7f63c6d9d640 1 -- 172.21.15.76:0/3405094844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63c806e870 msgr2=0x7f63c8152900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:18:00.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.745+0000 7f63c6d9d640 1 --2- 172.21.15.76:0/3405094844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63c806e870 0x7f63c8152900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.745+0000 7f63c6d9d640 1 -- 172.21.15.76:0/3405094844 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c81587d0 con 0x7f63c80d0810 2024-04-25T19:18:00.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.746+0000 7f63c6d9d640 1 --2- 172.21.15.76:0/3405094844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63c80d0810 0x7f63c81516b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009f40 tx=0x7f63c0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:00.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.746+0000 7f63b6ffd640 1 -- 172.21.15.76:0/3405094844 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c0041020 con 0x7f63c80d0810 2024-04-25T19:18:00.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.746+0000 7f63b6ffd640 1 -- 172.21.15.76:0/3405094844 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63c0007780 con 0x7f63c80d0810 2024-04-25T19:18:00.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.746+0000 7f63cc97a640 1 -- 172.21.15.76:0/3405094844 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c81536b0 con 0x7f63c80d0810 2024-04-25T19:18:00.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.746+0000 7f63b6ffd640 1 -- 172.21.15.76:0/3405094844 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c0036040 con 0x7f63c80d0810 2024-04-25T19:18:00.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.746+0000 7f63cc97a640 1 -- 172.21.15.76:0/3405094844 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c8153c30 con 0x7f63c80d0810 2024-04-25T19:18:00.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.747+0000 7f63cc97a640 1 --2- 172.21.15.76:0/3405094844 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63c81589a0 0x7f63c8159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:00.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.747+0000 7f63cc97a640 1 -- 172.21.15.76:0/3405094844 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f63c8153460 con 0x7f63c81589a0 2024-04-25T19:18:00.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.747+0000 7f63c659c640 1 --2- 172.21.15.76:0/3405094844 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63c81589a0 0x7f63c8159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:00.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.747+0000 7f63c659c640 1 -- 172.21.15.76:0/3405094844 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f63c806a390 con 0x7f63c81589a0 2024-04-25T19:18:00.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.747+0000 7f63c659c640 1 --2- 172.21.15.76:0/3405094844 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63c81589a0 0x7f63c8159bc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f63b800b450 tx=0x7f63b800b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:00.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.748+0000 7f63b6ffd640 1 -- 172.21.15.76:0/3405094844 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f63c0044020 con 0x7f63c80d0810 2024-04-25T19:18:00.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.748+0000 7f63b6ffd640 1 --2- 172.21.15.76:0/3405094844 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f63a8074460 0x7f63a8076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:00.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.748+0000 7f63b6ffd640 1 -- 172.21.15.76:0/3405094844 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b8003d40 con 0x7f63c81589a0 2024-04-25T19:18:00.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.748+0000 7f63b6ffd640 1 -- 172.21.15.76:0/3405094844 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f63c00b5c30 con 0x7f63c80d0810 2024-04-25T19:18:00.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.748+0000 7f63c5d9b640 1 --2- 172.21.15.76:0/3405094844 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f63a8074460 0x7f63a8076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:00.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.748+0000 7f63b6ffd640 1 -- 172.21.15.76:0/3405094844 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f63b8015070 con 0x7f63c81589a0 2024-04-25T19:18:00.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.748+0000 7f63b6ffd640 1 -- 172.21.15.76:0/3405094844 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63c81589a0 msgr2=0x7f63c8159bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:00.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.748+0000 7f63b6ffd640 1 --2- 172.21.15.76:0/3405094844 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63c81589a0 0x7f63c8159bc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f63b800b450 tx=0x7f63b800b910 comp rx=0 tx=0).stop 2024-04-25T19:18:00.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.748+0000 7f63b6ffd640 1 -- 172.21.15.76:0/3405094844 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63c81589a0 msgr2=0x7f63c8159bc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:18:00.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.748+0000 7f63b6ffd640 1 --2- 172.21.15.76:0/3405094844 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63c81589a0 0x7f63c8159bc0 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f63b800b450 tx=0x7f63b800b910 comp rx=0 tx=0).stop 2024-04-25T19:18:00.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.749+0000 7f63c5d9b640 1 --2- 172.21.15.76:0/3405094844 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f63a8074460 0x7f63a8076970 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f63c81f0630 tx=0x7f63bc0090e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:00.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.755+0000 7f63cc97a640 1 --2- 172.21.15.76:0/3405094844 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63c815a100 0x7f63c81501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:00.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.755+0000 7f63cc97a640 1 -- 172.21.15.76:0/3405094844 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f63c80b8070 con 0x7f63c815a100 2024-04-25T19:18:00.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.755+0000 7f63c659c640 1 --2- 172.21.15.76:0/3405094844 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63c815a100 0x7f63c81501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:00.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.755+0000 7f63c659c640 1 -- 172.21.15.76:0/3405094844 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f63c8153e00 con 0x7f63c815a100 2024-04-25T19:18:00.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.756+0000 7f63c659c640 1 --2- 172.21.15.76:0/3405094844 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63c815a100 0x7f63c81501e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f63b800beb0 tx=0x7f63b8005990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:00.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.757+0000 7f63b6ffd640 1 -- 172.21.15.76:0/3405094844 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b80054b0 con 0x7f63c815a100 2024-04-25T19:18:00.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.758+0000 7f63b6ffd640 1 -- 172.21.15.76:0/3405094844 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4883 (secure 0 0 0) 0x7f63b80054b0 con 0x7f63c815a100 2024-04-25T19:18:00.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.758+0000 7f63b6ffd640 1 -- 172.21.15.76:0/3405094844 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63c815a100 msgr2=0x7f63c81501e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:00.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.758+0000 7f63b6ffd640 1 --2- 172.21.15.76:0/3405094844 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63c815a100 0x7f63c81501e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f63b800beb0 tx=0x7f63b8005990 comp rx=0 tx=0).stop 2024-04-25T19:18:00.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.758+0000 7f63b6ffd640 1 -- 172.21.15.76:0/3405094844 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63c815a100 msgr2=0x7f63c81501e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:18:00.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.758+0000 7f63b6ffd640 1 --2- 172.21.15.76:0/3405094844 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63c815a100 0x7f63c81501e0 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f63b800beb0 tx=0x7f63b8005990 comp rx=0 tx=0).stop 2024-04-25T19:18:00.760 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:18:00.760 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:18:00.760 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:18:00.760 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "peon", 2024-04-25T19:18:00.760 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 20, 2024-04-25T19:18:00.760 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 84535, 2024-04-25T19:18:00.760 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:18:00.760 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-04-25T19:18:00.760 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:18:00.760 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:00.760 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 1, 2024-04-25T19:18:00.760 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:18:00.760 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:18:00.761 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:18:00.761 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:00.761 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:00.761 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:00.761 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:00.761 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:00.761 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:00.761 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:00.761 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:00.761 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:00.761 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:00.761 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:00.761 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:00.761 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:18:00.761 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:18:00.762 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:00.762 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:00.762 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:00.762 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:00.762 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:00.762 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:00.762 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:00.762 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:00.762 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:00.762 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:00.763 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:00.763 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:00.763 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:00.763 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:18:00.763 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:18:00.763 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:18:00.763 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:18:00.763 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:18:00.763 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:18:00.763 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:18:00.763 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:18:00.763 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:18:00.763 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:18:00.763 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:18:00.764 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:18:00.764 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:18:00.764 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:18:00.764 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:18:00.764 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:18:00.764 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:18:00.764 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:00.764 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:00.764 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:00.764 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:00.764 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:00.764 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:00.764 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:00.764 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:00.764 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:00.765 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:00.765 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:00.765 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:00.765 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:18:00.765 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:00.765 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:18:00.765 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.765 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:18:00.765 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:18:00.765 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:00.765 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:00.765 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.765 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:00.765 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:18:00.765 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:00.766 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:00.766 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.766 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:00.766 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:18:00.766 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:00.766 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:00.766 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:00.766 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:00.766 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:18:00.766 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:18:00.766 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:00.766 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:00.766 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:00.767 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:00.767 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.767 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:18:00.767 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:18:00.767 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:00.767 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:00.767 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.767 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:00.767 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:18:00.767 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:00.767 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:00.767 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.767 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:00.767 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:18:00.768 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:00.768 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:00.768 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:00.768 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:00.768 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:18:00.768 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:18:00.768 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:00.768 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:00.768 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:00.768 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:00.768 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.768 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:18:00.768 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:18:00.768 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:00.768 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:00.769 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.769 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:00.769 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:18:00.769 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:00.769 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:00.769 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.769 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:00.769 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:18:00.769 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:00.769 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:00.769 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:00.769 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:00.769 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:18:00.769 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:18:00.770 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:00.770 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:00.770 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:00.770 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:00.770 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:00.770 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:00.770 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:18:00.770 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:18:00.770 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.770 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:00.770 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:00.770 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:18:00.770 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:00.770 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:00.770 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": [ 2024-04-25T19:18:00.771 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.771 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:00.771 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:00.771 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 3 2024-04-25T19:18:00.771 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:00.771 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:00.771 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": [ 2024-04-25T19:18:00.771 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.771 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:00.771 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:00.771 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 5 2024-04-25T19:18:00.771 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:00.771 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:00.771 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:18:00.771 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:00.772 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:00.772 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:00.772 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 4 2024-04-25T19:18:00.772 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:00.772 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:00.772 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:00.772 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:18:00.772 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:18:00.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.758+0000 7f63cc97a640 1 -- 172.21.15.76:0/3405094844 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f63a8074460 msgr2=0x7f63a8076970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:00.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.758+0000 7f63cc97a640 1 --2- 172.21.15.76:0/3405094844 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f63a8074460 0x7f63a8076970 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f63c81f0630 tx=0x7f63bc0090e0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.758+0000 7f63cc97a640 1 -- 172.21.15.76:0/3405094844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63c80d0810 msgr2=0x7f63c81516b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:00.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.758+0000 7f63c6d9d640 1 -- 172.21.15.76:0/3405094844 reap_dead start 2024-04-25T19:18:00.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.758+0000 7f63cc97a640 1 --2- 172.21.15.76:0/3405094844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63c80d0810 0x7f63c81516b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009f40 tx=0x7f63c0004a40 comp rx=0 tx=0).stop 2024-04-25T19:18:00.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.758+0000 7f63cc97a640 1 -- 172.21.15.76:0/3405094844 shutdown_connections 2024-04-25T19:18:00.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.758+0000 7f63cc97a640 1 --2- 172.21.15.76:0/3405094844 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63c80d0810 0x7f63c81516b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:00.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.758+0000 7f63cc97a640 1 -- 172.21.15.76:0/3405094844 >> 172.21.15.76:0/3405094844 conn(0x7f63c80b3c00 msgr2=0x7f63c80ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:00.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.758+0000 7f63cc97a640 1 -- 172.21.15.76:0/3405094844 shutdown_connections 2024-04-25T19:18:00.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:00.758+0000 7f63cc97a640 1 -- 172.21.15.76:0/3405094844 wait complete. 2024-04-25T19:18:00.774 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T19:18:00.774 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-04-25T19:18:03.627 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:18:03.627 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:18:03.627 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:18:03.627 INFO:tasks.workunit.client.0.smithi076.stdout:iozone test complete. 2024-04-25T19:18:03.627 INFO:tasks.workunit.client.0.smithi076.stderr:+ iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-04-25T19:18:03.692 INFO:tasks.workunit.client.0.smithi076.stdout: Iozone: Performance Test of File I/O 2024-04-25T19:18:03.692 INFO:tasks.workunit.client.0.smithi076.stdout: Version $Revision: 3.494 $ 2024-04-25T19:18:03.692 INFO:tasks.workunit.client.0.smithi076.stdout: Compiled for 64 bit mode. 2024-04-25T19:18:03.693 INFO:tasks.workunit.client.0.smithi076.stdout: Build: linux-AMD64 2024-04-25T19:18:03.693 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:18:03.693 INFO:tasks.workunit.client.0.smithi076.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-04-25T19:18:03.693 INFO:tasks.workunit.client.0.smithi076.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-04-25T19:18:03.693 INFO:tasks.workunit.client.0.smithi076.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-04-25T19:18:03.693 INFO:tasks.workunit.client.0.smithi076.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-04-25T19:18:03.693 INFO:tasks.workunit.client.0.smithi076.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-04-25T19:18:03.693 INFO:tasks.workunit.client.0.smithi076.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-04-25T19:18:03.693 INFO:tasks.workunit.client.0.smithi076.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-04-25T19:18:03.693 INFO:tasks.workunit.client.0.smithi076.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-04-25T19:18:03.693 INFO:tasks.workunit.client.0.smithi076.stdout: Alexey Skidanov, Sudhir Kumar. 2024-04-25T19:18:03.693 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:18:03.693 INFO:tasks.workunit.client.0.smithi076.stdout: Run began: Thu Apr 25 19:18:03 2024 2024-04-25T19:18:03.693 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:18:03.693 INFO:tasks.workunit.client.0.smithi076.stdout: Include close in write timing 2024-04-25T19:18:03.694 INFO:tasks.workunit.client.0.smithi076.stdout: Include fsync in write timing 2024-04-25T19:18:03.694 INFO:tasks.workunit.client.0.smithi076.stdout: File size set to 1048576 kB 2024-04-25T19:18:03.694 INFO:tasks.workunit.client.0.smithi076.stdout: Record Size 1024 kB 2024-04-25T19:18:03.694 INFO:tasks.workunit.client.0.smithi076.stdout: Command line used: iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-04-25T19:18:03.694 INFO:tasks.workunit.client.0.smithi076.stdout: Output is in kBytes/sec 2024-04-25T19:18:03.694 INFO:tasks.workunit.client.0.smithi076.stdout: Time Resolution = 0.000001 seconds. 2024-04-25T19:18:03.694 INFO:tasks.workunit.client.0.smithi076.stdout: Processor cache size set to 1024 kBytes. 2024-04-25T19:18:03.694 INFO:tasks.workunit.client.0.smithi076.stdout: Processor cache line size set to 32 bytes. 2024-04-25T19:18:03.694 INFO:tasks.workunit.client.0.smithi076.stdout: File stride size set to 17 * record size. 2024-04-25T19:18:03.694 INFO:tasks.workunit.client.0.smithi076.stdout: Throughput test with 1 process 2024-04-25T19:18:03.694 INFO:tasks.workunit.client.0.smithi076.stdout: Each process writes a 1048576 kByte file in 1024 kByte records 2024-04-25T19:18:06.170 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:18:06.171 INFO:tasks.workunit.client.0.smithi076.stdout: Children see throughput for 1 initial writers = 428720.09 kB/sec 2024-04-25T19:18:06.171 INFO:tasks.workunit.client.0.smithi076.stdout: Parent sees throughput for 1 initial writers = 428689.05 kB/sec 2024-04-25T19:18:06.171 INFO:tasks.workunit.client.0.smithi076.stdout: Min throughput per process = 428720.09 kB/sec 2024-04-25T19:18:06.171 INFO:tasks.workunit.client.0.smithi076.stdout: Max throughput per process = 428720.09 kB/sec 2024-04-25T19:18:06.171 INFO:tasks.workunit.client.0.smithi076.stdout: Avg throughput per process = 428720.09 kB/sec 2024-04-25T19:18:06.171 INFO:tasks.workunit.client.0.smithi076.stdout: Min xfer = 1048576.00 kB 2024-04-25T19:18:10.265 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:18:10.265 INFO:tasks.workunit.client.0.smithi076.stdout: Children see throughput for 1 rewriters = 791667.81 kB/sec 2024-04-25T19:18:10.265 INFO:tasks.workunit.client.0.smithi076.stdout: Parent sees throughput for 1 rewriters = 791547.89 kB/sec 2024-04-25T19:18:10.265 INFO:tasks.workunit.client.0.smithi076.stdout: Min throughput per process = 791667.81 kB/sec 2024-04-25T19:18:10.265 INFO:tasks.workunit.client.0.smithi076.stdout: Max throughput per process = 791667.81 kB/sec 2024-04-25T19:18:10.265 INFO:tasks.workunit.client.0.smithi076.stdout: Avg throughput per process = 791667.81 kB/sec 2024-04-25T19:18:10.265 INFO:tasks.workunit.client.0.smithi076.stdout: Min xfer = 1048576.00 kB 2024-04-25T19:18:13.058 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:18:13.059 INFO:tasks.workunit.client.0.smithi076.stdout: Children see throughput for 1 readers = 7343224.00 kB/sec 2024-04-25T19:18:13.059 INFO:tasks.workunit.client.0.smithi076.stdout: Parent sees throughput for 1 readers = 7330898.51 kB/sec 2024-04-25T19:18:13.059 INFO:tasks.workunit.client.0.smithi076.stdout: Min throughput per process = 7343224.00 kB/sec 2024-04-25T19:18:13.059 INFO:tasks.workunit.client.0.smithi076.stdout: Max throughput per process = 7343224.00 kB/sec 2024-04-25T19:18:13.059 INFO:tasks.workunit.client.0.smithi076.stdout: Avg throughput per process = 7343224.00 kB/sec 2024-04-25T19:18:13.059 INFO:tasks.workunit.client.0.smithi076.stdout: Min xfer = 1048576.00 kB 2024-04-25T19:18:15.270 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:18:15.270 INFO:tasks.workunit.client.0.smithi076.stdout: Children see throughput for 1 re-readers = 7158887.00 kB/sec 2024-04-25T19:18:15.270 INFO:tasks.workunit.client.0.smithi076.stdout: Parent sees throughput for 1 re-readers = 7147613.18 kB/sec 2024-04-25T19:18:15.270 INFO:tasks.workunit.client.0.smithi076.stdout: Min throughput per process = 7158887.00 kB/sec 2024-04-25T19:18:15.270 INFO:tasks.workunit.client.0.smithi076.stdout: Max throughput per process = 7158887.00 kB/sec 2024-04-25T19:18:15.270 INFO:tasks.workunit.client.0.smithi076.stdout: Avg throughput per process = 7158887.00 kB/sec 2024-04-25T19:18:15.270 INFO:tasks.workunit.client.0.smithi076.stdout: Min xfer = 1048576.00 kB 2024-04-25T19:18:20.776 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-04-25T19:18:20.776 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-04-25T19:18:20.776 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T19:18:20.776 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T19:18:20.779 INFO:tasks.ceph.mon.a:Started 2024-04-25T19:18:20.780 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T19:18:20.780 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:18:20.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.900+0000 7fe2ebe77640 1 Processor -- start 2024-04-25T19:18:20.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.900+0000 7fe2ebe77640 1 -- start start 2024-04-25T19:18:20.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.901+0000 7fe2ebe77640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe2e4156fa0 0x7fe2e40cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:20.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.901+0000 7fe2ebe77640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe2e40d19f0 0x7fe2e406d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:20.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.901+0000 7fe2ebe77640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2e415cc60 con 0x7fe2e406d560 2024-04-25T19:18:20.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.901+0000 7fe2ebe77640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe2e415d870 con 0x7fe2e4156fa0 2024-04-25T19:18:20.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.901+0000 7fe2ebe77640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fe2e415cf40 con 0x7fe2e40d19f0 2024-04-25T19:18:20.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.902+0000 7fe2e9c13640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe2e4156fa0 0x7fe2e40cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:20.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.902+0000 7fe2e9c13640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe2e4156fa0 0x7fe2e40cfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34280/0 (socket says 172.21.15.76:34280) 2024-04-25T19:18:20.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.902+0000 7fe2e9c13640 1 -- 172.21.15.76:0/1513165109 learned_addr learned my addr 172.21.15.76:0/1513165109 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:20.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.902+0000 7fe2e9c13640 1 -- 172.21.15.76:0/1513165109 >> v2:172.21.15.76:3300/0 conn(0x7fe2e4156fa0 msgr2=0x7fe2e40cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T19:18:20.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.902+0000 7fe2e9c13640 1 -- 172.21.15.76:0/1513165109 >> v2:172.21.15.76:3300/0 conn(0x7fe2e4156fa0 msgr2=0x7fe2e40cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:18:20.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.902+0000 7fe2e9c13640 1 --2- 172.21.15.76:0/1513165109 >> v2:172.21.15.76:3300/0 conn(0x7fe2e4156fa0 0x7fe2e40cfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:18:20.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.902+0000 7fe2e9c13640 1 --2- 172.21.15.76:0/1513165109 >> v2:172.21.15.76:3300/0 conn(0x7fe2e4156fa0 0x7fe2e40cfdb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:18:20.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.902+0000 7fe2e9412640 1 --2- 172.21.15.76:0/1513165109 >> v2:172.21.15.105:3300/0 conn(0x7fe2e40d19f0 0x7fe2e406d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:20.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.902+0000 7fe2e9412640 1 -- 172.21.15.76:0/1513165109 >> v2:172.21.15.76:3300/0 conn(0x7fe2e4156fa0 msgr2=0x7fe2e40cfdb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:18:20.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.902+0000 7fe2e9412640 1 --2- 172.21.15.76:0/1513165109 >> v2:172.21.15.76:3300/0 conn(0x7fe2e4156fa0 0x7fe2e40cfdb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:20.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.902+0000 7fe2e9412640 1 -- 172.21.15.76:0/1513165109 >> v1:172.21.15.105:6789/0 conn(0x7fe2e406d560 legacy=0x7fe2e406e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:20.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.902+0000 7fe2e9412640 1 -- 172.21.15.76:0/1513165109 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e415d220 con 0x7fe2e40d19f0 2024-04-25T19:18:20.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.903+0000 7fe2e9412640 1 --2- 172.21.15.76:0/1513165109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2e40d19f0 0x7fe2e406d020 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4069de0 tx=0x7fe2e002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9347cf1851f112f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:20.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.903+0000 7fe2e8c11640 1 -- 172.21.15.76:0/1513165109 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0037070 con 0x7fe2e40d19f0 2024-04-25T19:18:20.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.903+0000 7fe2e8c11640 1 -- 172.21.15.76:0/1513165109 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e0032440 con 0x7fe2e40d19f0 2024-04-25T19:18:20.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.903+0000 7fe2e8c11640 1 -- 172.21.15.76:0/1513165109 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0032810 con 0x7fe2e40d19f0 2024-04-25T19:18:20.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.904+0000 7fe2ebe77640 1 -- 172.21.15.76:0/1513165109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2e40d19f0 msgr2=0x7fe2e406d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:20.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.904+0000 7fe2ebe77640 1 --2- 172.21.15.76:0/1513165109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2e40d19f0 0x7fe2e406d020 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4069de0 tx=0x7fe2e002a230 comp rx=0 tx=0).stop 2024-04-25T19:18:20.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.904+0000 7fe2ebe77640 1 -- 172.21.15.76:0/1513165109 shutdown_connections 2024-04-25T19:18:20.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.904+0000 7fe2ebe77640 1 --2- 172.21.15.76:0/1513165109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2e40d19f0 0x7fe2e406d020 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:20.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.904+0000 7fe2ebe77640 1 --2- 172.21.15.76:0/1513165109 >> v2:172.21.15.76:3300/0 conn(0x7fe2e4156fa0 0x7fe2e40cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:20.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.904+0000 7fe2ebe77640 1 -- 172.21.15.76:0/1513165109 >> 172.21.15.76:0/1513165109 conn(0x7fe2e40b3c80 msgr2=0x7fe2e40b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:20.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.904+0000 7fe2ebe77640 1 -- 172.21.15.76:0/1513165109 shutdown_connections 2024-04-25T19:18:20.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.904+0000 7fe2ebe77640 1 -- 172.21.15.76:0/1513165109 wait complete. 2024-04-25T19:18:20.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.906+0000 7fe2ebe77640 1 Processor -- start 2024-04-25T19:18:20.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.906+0000 7fe2ebe77640 1 -- start start 2024-04-25T19:18:20.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.907+0000 7fe2ebe77640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2e406d560 0x7fe2e4150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:20.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.907+0000 7fe2ebe77640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2e40d19f0 0x7fe2e4155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:20.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.907+0000 7fe2ebe77640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2e4156fa0 0x7fe2e414f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:20.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.907+0000 7fe2ebe77640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e40cd710 con 0x7fe2e40d19f0 2024-04-25T19:18:20.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.907+0000 7fe2ebe77640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e406e080 con 0x7fe2e4156fa0 2024-04-25T19:18:20.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.907+0000 7fe2ebe77640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fe2e40cd340 con 0x7fe2e406d560 2024-04-25T19:18:20.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.907+0000 7fe2e9412640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2e40d19f0 0x7fe2e4155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:20.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.907+0000 7fe2e9412640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2e40d19f0 0x7fe2e4155050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34294/0 (socket says 172.21.15.76:34294) 2024-04-25T19:18:20.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.907+0000 7fe2e9412640 1 -- 172.21.15.76:0/304412387 learned_addr learned my addr 172.21.15.76:0/304412387 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:20.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.907+0000 7fe2e9412640 1 -- 172.21.15.76:0/304412387 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2e40d19f0 msgr2=0x7fe2e4155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-04-25T19:18:20.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.907+0000 7fe2e9412640 1 -- 172.21.15.76:0/304412387 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2e40d19f0 msgr2=0x7fe2e4155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T19:18:20.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.907+0000 7fe2e9412640 1 --2- 172.21.15.76:0/304412387 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2e40d19f0 0x7fe2e4155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:18:20.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.907+0000 7fe2e9c13640 1 --2- 172.21.15.76:0/304412387 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2e406d560 0x7fe2e4150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:20.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.907+0000 7fe2e9412640 1 --2- 172.21.15.76:0/304412387 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2e40d19f0 0x7fe2e4155050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:18:20.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.907+0000 7fe2ea414640 1 --2- 172.21.15.76:0/304412387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2e4156fa0 0x7fe2e414f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:20.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.908+0000 7fe2e9c13640 1 -- 172.21.15.76:0/304412387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2e4156fa0 msgr2=0x7fe2e414f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:20.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.908+0000 7fe2e9c13640 1 --2- 172.21.15.76:0/304412387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2e4156fa0 0x7fe2e414f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:20.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.908+0000 7fe2e9c13640 1 -- 172.21.15.76:0/304412387 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2e40d19f0 msgr2=0x7fe2e4155050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:18:20.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.908+0000 7fe2e9c13640 1 --2- 172.21.15.76:0/304412387 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2e40d19f0 0x7fe2e4155050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:20.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.908+0000 7fe2e9c13640 1 -- 172.21.15.76:0/304412387 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e4150b80 con 0x7fe2e406d560 2024-04-25T19:18:20.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.908+0000 7fe2e9c13640 1 --2- 172.21.15.76:0/304412387 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2e406d560 0x7fe2e4150640 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe2d400e810 tx=0x7fe2d400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:20.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.908+0000 7fe2da7fc640 1 -- 172.21.15.76:0/304412387 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d4017070 con 0x7fe2e406d560 2024-04-25T19:18:20.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.908+0000 7fe2da7fc640 1 -- 172.21.15.76:0/304412387 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2d4004020 con 0x7fe2e406d560 2024-04-25T19:18:20.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.908+0000 7fe2da7fc640 1 -- 172.21.15.76:0/304412387 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d400f040 con 0x7fe2e406d560 2024-04-25T19:18:20.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.908+0000 7fe2ebe77640 1 -- 172.21.15.76:0/304412387 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2e41f2d50 con 0x7fe2e406d560 2024-04-25T19:18:20.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.908+0000 7fe2ebe77640 1 -- 172.21.15.76:0/304412387 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2e41f3310 con 0x7fe2e406d560 2024-04-25T19:18:20.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.909+0000 7fe2ebe77640 1 -- 172.21.15.76:0/304412387 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2e406a390 con 0x7fe2e406d560 2024-04-25T19:18:20.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.910+0000 7fe2da7fc640 1 -- 172.21.15.76:0/304412387 <== mon.2 v2:172.21.15.105:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe2d4005990 con 0x7fe2e406d560 2024-04-25T19:18:20.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.911+0000 7fe2da7fc640 1 --2- 172.21.15.76:0/304412387 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe2c80744b0 0x7fe2c80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:20.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.911+0000 7fe2da7fc640 1 -- 172.21.15.76:0/304412387 <== mon.2 v2:172.21.15.105:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fe2d403be00 con 0x7fe2e406d560 2024-04-25T19:18:20.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.911+0000 7fe2e9412640 1 --2- 172.21.15.76:0/304412387 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe2c80744b0 0x7fe2c80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:20.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.911+0000 7fe2e9412640 1 --2- 172.21.15.76:0/304412387 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe2c80744b0 0x7fe2c80769c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe2e40b7da0 tx=0x7fe2e002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:20.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:20.915+0000 7fe2da7fc640 1 -- 172.21.15.76:0/304412387 <== mon.2 v2:172.21.15.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe2d405efd0 con 0x7fe2e406d560 2024-04-25T19:18:21.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:21.159+0000 7fe2ebe77640 1 -- 172.21.15.76:0/304412387 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe2e41503a0 con 0x7fe2e406d560 2024-04-25T19:18:21.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:21.160+0000 7fe2da7fc640 1 -- 172.21.15.76:0/304412387 <== mon.2 v2:172.21.15.105:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1559 (secure 0 0 0) 0x7fe2d40632d0 con 0x7fe2e406d560 2024-04-25T19:18:21.161 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":20,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:18:21.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:21.162+0000 7fe2ebe77640 1 -- 172.21.15.76:0/304412387 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe2c80744b0 msgr2=0x7fe2c80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:21.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:21.162+0000 7fe2ebe77640 1 --2- 172.21.15.76:0/304412387 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe2c80744b0 0x7fe2c80769c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe2e40b7da0 tx=0x7fe2e002e000 comp rx=0 tx=0).stop 2024-04-25T19:18:21.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:21.162+0000 7fe2ebe77640 1 -- 172.21.15.76:0/304412387 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2e406d560 msgr2=0x7fe2e4150640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:21.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:21.162+0000 7fe2ebe77640 1 --2- 172.21.15.76:0/304412387 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2e406d560 0x7fe2e4150640 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe2d400e810 tx=0x7fe2d400ecd0 comp rx=0 tx=0).stop 2024-04-25T19:18:21.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:21.162+0000 7fe2ebe77640 1 -- 172.21.15.76:0/304412387 shutdown_connections 2024-04-25T19:18:21.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:21.162+0000 7fe2ebe77640 1 --2- 172.21.15.76:0/304412387 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe2c80744b0 0x7fe2c80769c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:21.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:21.162+0000 7fe2ebe77640 1 --2- 172.21.15.76:0/304412387 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2e4156fa0 0x7fe2e414f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:21.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:21.162+0000 7fe2ebe77640 1 --2- 172.21.15.76:0/304412387 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2e40d19f0 0x7fe2e4155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:21.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:21.162+0000 7fe2ebe77640 1 --2- 172.21.15.76:0/304412387 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe2e406d560 0x7fe2e4150640 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:21.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:21.162+0000 7fe2ebe77640 1 -- 172.21.15.76:0/304412387 >> 172.21.15.76:0/304412387 conn(0x7fe2e40b3c80 msgr2=0x7fe2e40d0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:21.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:21.162+0000 7fe2ebe77640 1 -- 172.21.15.76:0/304412387 shutdown_connections 2024-04-25T19:18:21.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:21.162+0000 7fe2ebe77640 1 -- 172.21.15.76:0/304412387 wait complete. 2024-04-25T19:18:21.699 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:18:21.699 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:18:21.699 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:18:21.699 INFO:tasks.workunit.client.0.smithi076.stdout:iozone test complete. 2024-04-25T19:18:21.699 INFO:tasks.workunit.client.0.smithi076.stderr:+ iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-04-25T19:18:21.764 INFO:tasks.workunit.client.0.smithi076.stdout: Iozone: Performance Test of File I/O 2024-04-25T19:18:21.764 INFO:tasks.workunit.client.0.smithi076.stdout: Version $Revision: 3.494 $ 2024-04-25T19:18:21.764 INFO:tasks.workunit.client.0.smithi076.stdout: Compiled for 64 bit mode. 2024-04-25T19:18:21.764 INFO:tasks.workunit.client.0.smithi076.stdout: Build: linux-AMD64 2024-04-25T19:18:21.764 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:18:21.764 INFO:tasks.workunit.client.0.smithi076.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-04-25T19:18:21.764 INFO:tasks.workunit.client.0.smithi076.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-04-25T19:18:21.764 INFO:tasks.workunit.client.0.smithi076.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-04-25T19:18:21.765 INFO:tasks.workunit.client.0.smithi076.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-04-25T19:18:21.765 INFO:tasks.workunit.client.0.smithi076.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-04-25T19:18:21.765 INFO:tasks.workunit.client.0.smithi076.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-04-25T19:18:21.765 INFO:tasks.workunit.client.0.smithi076.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-04-25T19:18:21.765 INFO:tasks.workunit.client.0.smithi076.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-04-25T19:18:21.765 INFO:tasks.workunit.client.0.smithi076.stdout: Alexey Skidanov, Sudhir Kumar. 2024-04-25T19:18:21.765 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:18:21.765 INFO:tasks.workunit.client.0.smithi076.stdout: Run began: Thu Apr 25 19:18:21 2024 2024-04-25T19:18:21.765 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:18:21.765 INFO:tasks.workunit.client.0.smithi076.stdout: Include close in write timing 2024-04-25T19:18:21.765 INFO:tasks.workunit.client.0.smithi076.stdout: Include fsync in write timing 2024-04-25T19:18:21.765 INFO:tasks.workunit.client.0.smithi076.stdout: File size set to 10485760 kB 2024-04-25T19:18:21.765 INFO:tasks.workunit.client.0.smithi076.stdout: Record Size 1024 kB 2024-04-25T19:18:21.765 INFO:tasks.workunit.client.0.smithi076.stdout: Command line used: iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-04-25T19:18:21.765 INFO:tasks.workunit.client.0.smithi076.stdout: Output is in kBytes/sec 2024-04-25T19:18:21.766 INFO:tasks.workunit.client.0.smithi076.stdout: Time Resolution = 0.000001 seconds. 2024-04-25T19:18:21.766 INFO:tasks.workunit.client.0.smithi076.stdout: Processor cache size set to 1024 kBytes. 2024-04-25T19:18:21.766 INFO:tasks.workunit.client.0.smithi076.stdout: Processor cache line size set to 32 bytes. 2024-04-25T19:18:21.766 INFO:tasks.workunit.client.0.smithi076.stdout: File stride size set to 17 * record size. 2024-04-25T19:18:21.766 INFO:tasks.workunit.client.0.smithi076.stdout: Throughput test with 1 process 2024-04-25T19:18:21.766 INFO:tasks.workunit.client.0.smithi076.stdout: Each process writes a 10485760 kByte file in 1024 kByte records 2024-04-25T19:18:24.176 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:18:24.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.341+0000 7fac8d9a8640 1 Processor -- start 2024-04-25T19:18:24.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.342+0000 7fac8d9a8640 1 -- start start 2024-04-25T19:18:24.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.343+0000 7fac8d9a8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fac88160e20 0x7fac880c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:24.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.343+0000 7fac8d9a8640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac880c8fe0 con 0x7fac880c9780 2024-04-25T19:18:24.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.343+0000 7fac8d9a8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac880c92c0 con 0x7fac88156f50 2024-04-25T19:18:24.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.343+0000 7fac8d9a8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fac88160340 con 0x7fac88160e20 2024-04-25T19:18:24.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.344+0000 7fac86ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fac880c9780 0x7fac88068520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:49422/0 (socket says 172.21.15.76:49422) 2024-04-25T19:18:24.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.344+0000 7fac86ffd640 1 -- 172.21.15.76:0/84709401 learned_addr learned my addr 172.21.15.76:0/84709401 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:18:24.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.344+0000 7fac867fc640 1 -- 172.21.15.76:0/84709401 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 534978938 0 0) 0x7fac880c8fe0 con 0x7fac880c9780 2024-04-25T19:18:24.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.344+0000 7fac7ffff640 1 --2- 172.21.15.76:0/84709401 >> v2:172.21.15.105:3300/0 conn(0x7fac88160e20 0x7fac880c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:24.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.344+0000 7fac867fc640 1 -- 172.21.15.76:0/84709401 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac74003520 con 0x7fac880c9780 2024-04-25T19:18:24.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.344+0000 7fac867fc640 1 -- 172.21.15.76:0/84709401 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 65396401 0 0) 0x7fac74003520 con 0x7fac880c9780 2024-04-25T19:18:24.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.344+0000 7fac867fc640 1 -- 172.21.15.76:0/84709401 >> v2:172.21.15.105:3300/0 conn(0x7fac88160e20 msgr2=0x7fac880c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:24.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.344+0000 7fac867fc640 1 --2- 172.21.15.76:0/84709401 >> v2:172.21.15.105:3300/0 conn(0x7fac88160e20 0x7fac880c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:24.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.344+0000 7fac867fc640 1 -- 172.21.15.76:0/84709401 >> v1:172.21.15.105:6789/0 conn(0x7fac88156f50 legacy=0x7fac88074130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:24.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.344+0000 7fac867fc640 1 -- 172.21.15.76:0/84709401 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac88160620 con 0x7fac880c9780 2024-04-25T19:18:24.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.344+0000 7fac867fc640 1 -- 172.21.15.76:0/84709401 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7fac78001670 con 0x7fac880c9780 2024-04-25T19:18:24.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.344+0000 7fac867fc640 1 -- 172.21.15.76:0/84709401 >> v1:172.21.15.76:6789/0 conn(0x7fac880c9780 legacy=0x7fac88068520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:24.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.344+0000 7fac867fc640 1 --2- 172.21.15.76:0/84709401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac740039b0 0x7fac74003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:24.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.344+0000 7fac867fc640 1 --2- 172.21.15.76:0/84709401 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fac74004800 0x7fac74004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:24.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.344+0000 7fac867fc640 1 --2- 172.21.15.76:0/84709401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac74005270 0x7fac7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:24.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.344+0000 7fac867fc640 1 -- 172.21.15.76:0/84709401 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fac88160340 con 0x7fac740039b0 2024-04-25T19:18:24.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.344+0000 7fac867fc640 1 -- 172.21.15.76:0/84709401 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fac7400c1e0 con 0x7fac74005270 2024-04-25T19:18:24.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.344+0000 7fac867fc640 1 -- 172.21.15.76:0/84709401 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fac7400c3e0 con 0x7fac74004800 2024-04-25T19:18:24.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.345+0000 7fac86ffd640 1 --2- 172.21.15.76:0/84709401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac740039b0 0x7fac74003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:24.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.345+0000 7fac7ffff640 1 --2- 172.21.15.76:0/84709401 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fac74004800 0x7fac74004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:24.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.345+0000 7fac877fe640 1 --2- 172.21.15.76:0/84709401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac74005270 0x7fac7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:24.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.345+0000 7fac86ffd640 1 -- 172.21.15.76:0/84709401 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fac74004800 msgr2=0x7fac74004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:24.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.345+0000 7fac86ffd640 1 --2- 172.21.15.76:0/84709401 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fac74004800 0x7fac74004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:24.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.345+0000 7fac86ffd640 1 -- 172.21.15.76:0/84709401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac74005270 msgr2=0x7fac7400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:24.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.345+0000 7fac86ffd640 1 --2- 172.21.15.76:0/84709401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac74005270 0x7fac7400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:24.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.345+0000 7fac86ffd640 1 -- 172.21.15.76:0/84709401 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac74003520 con 0x7fac740039b0 2024-04-25T19:18:24.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.345+0000 7fac877fe640 1 --2- 172.21.15.76:0/84709401 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac74005270 0x7fac7400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:18:24.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.345+0000 7fac877fe640 1 -- 172.21.15.76:0/84709401 reap_dead start 2024-04-25T19:18:24.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.346+0000 7fac86ffd640 1 --2- 172.21.15.76:0/84709401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac740039b0 0x7fac74003e00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fac7800c240 tx=0x7fac74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204edb02b0de3aa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:24.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.346+0000 7fac867fc640 1 -- 172.21.15.76:0/84709401 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7802e070 con 0x7fac740039b0 2024-04-25T19:18:24.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.346+0000 7fac867fc640 1 -- 172.21.15.76:0/84709401 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac78005af0 con 0x7fac740039b0 2024-04-25T19:18:24.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.346+0000 7fac867fc640 1 -- 172.21.15.76:0/84709401 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78007d30 con 0x7fac740039b0 2024-04-25T19:18:24.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.347+0000 7fac8d9a8640 1 -- 172.21.15.76:0/84709401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac740039b0 msgr2=0x7fac74003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:24.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.347+0000 7fac8d9a8640 1 --2- 172.21.15.76:0/84709401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac740039b0 0x7fac74003e00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fac7800c240 tx=0x7fac74005830 comp rx=0 tx=0).stop 2024-04-25T19:18:24.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.348+0000 7fac8d9a8640 1 -- 172.21.15.76:0/84709401 shutdown_connections 2024-04-25T19:18:24.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.348+0000 7fac8d9a8640 1 --2- 172.21.15.76:0/84709401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac740039b0 0x7fac74003e00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:24.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.348+0000 7fac8d9a8640 1 -- 172.21.15.76:0/84709401 >> 172.21.15.76:0/84709401 conn(0x7fac8806c5c0 msgr2=0x7fac8806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:24.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.348+0000 7fac8d9a8640 1 -- 172.21.15.76:0/84709401 shutdown_connections 2024-04-25T19:18:24.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.348+0000 7fac8d9a8640 1 -- 172.21.15.76:0/84709401 wait complete. 2024-04-25T19:18:24.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.350+0000 7fac8d9a8640 1 Processor -- start 2024-04-25T19:18:24.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.350+0000 7fac8d9a8640 1 -- start start 2024-04-25T19:18:24.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.352+0000 7fac8d9a8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fac740039b0 0x7fac88155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:24.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.352+0000 7fac8d9a8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac88151d20 0x7fac8815f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:24.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.352+0000 7fac8d9a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac88152850 0x7fac88154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:24.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.352+0000 7fac8d9a8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fac880c5590 con 0x7fac88151d20 2024-04-25T19:18:24.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.352+0000 7fac8d9a8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fac88157360 con 0x7fac88152850 2024-04-25T19:18:24.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.352+0000 7fac8d9a8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fac880c9cd0 con 0x7fac740039b0 2024-04-25T19:18:24.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.352+0000 7fac7ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac88151d20 0x7fac8815f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:24.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.353+0000 7fac7ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac88151d20 0x7fac8815f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34320/0 (socket says 172.21.15.76:34320) 2024-04-25T19:18:24.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.353+0000 7fac7ffff640 1 -- 172.21.15.76:0/2632680244 learned_addr learned my addr 172.21.15.76:0/2632680244 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:24.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.353+0000 7fac86ffd640 1 --2- 172.21.15.76:0/2632680244 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fac740039b0 0x7fac88155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:24.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.353+0000 7fac877fe640 1 --2- 172.21.15.76:0/2632680244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac88152850 0x7fac88154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:24.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.353+0000 7fac7ffff640 1 -- 172.21.15.76:0/2632680244 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fac740039b0 msgr2=0x7fac88155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:24.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.353+0000 7fac7ffff640 1 --2- 172.21.15.76:0/2632680244 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fac740039b0 0x7fac88155f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:24.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.353+0000 7fac7ffff640 1 -- 172.21.15.76:0/2632680244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac88152850 msgr2=0x7fac88154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:24.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.353+0000 7fac7ffff640 1 --2- 172.21.15.76:0/2632680244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac88152850 0x7fac88154760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:24.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.353+0000 7fac7ffff640 1 -- 172.21.15.76:0/2632680244 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8815f8d0 con 0x7fac88151d20 2024-04-25T19:18:24.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.353+0000 7fac7ffff640 1 --2- 172.21.15.76:0/2632680244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac88151d20 0x7fac8815f390 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fac7000f680 tx=0x7fac7000fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:24.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.354+0000 7fac84ff9640 1 -- 172.21.15.76:0/2632680244 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70019030 con 0x7fac88151d20 2024-04-25T19:18:24.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.354+0000 7fac8d9a8640 1 -- 172.21.15.76:0/2632680244 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac88152e40 con 0x7fac88151d20 2024-04-25T19:18:24.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.354+0000 7fac84ff9640 1 -- 172.21.15.76:0/2632680244 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac70004020 con 0x7fac88151d20 2024-04-25T19:18:24.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.354+0000 7fac84ff9640 1 -- 172.21.15.76:0/2632680244 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac700043f0 con 0x7fac88151d20 2024-04-25T19:18:24.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.354+0000 7fac8d9a8640 1 -- 172.21.15.76:0/2632680244 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac881e8b40 con 0x7fac88151d20 2024-04-25T19:18:24.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.355+0000 7fac8d9a8640 1 -- 172.21.15.76:0/2632680244 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac8806d6a0 con 0x7fac88151d20 2024-04-25T19:18:24.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.356+0000 7fac84ff9640 1 -- 172.21.15.76:0/2632680244 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fac7001c070 con 0x7fac88151d20 2024-04-25T19:18:24.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.356+0000 7fac84ff9640 1 --2- 172.21.15.76:0/2632680244 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fac680744b0 0x7fac680769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:24.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.356+0000 7fac84ff9640 1 -- 172.21.15.76:0/2632680244 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fac70095680 con 0x7fac88151d20 2024-04-25T19:18:24.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.357+0000 7fac86ffd640 1 --2- 172.21.15.76:0/2632680244 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fac680744b0 0x7fac680769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:24.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.357+0000 7fac86ffd640 1 --2- 172.21.15.76:0/2632680244 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fac680744b0 0x7fac680769c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fac78033000 tx=0x7fac7802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:24.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.364+0000 7fac84ff9640 1 -- 172.21.15.76:0/2632680244 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fac7005feb0 con 0x7fac88151d20 2024-04-25T19:18:24.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.736+0000 7fac8d9a8640 1 -- 172.21.15.76:0/2632680244 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fac8806a390 con 0x7fac88151d20 2024-04-25T19:18:24.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.737+0000 7fac84ff9640 1 -- 172.21.15.76:0/2632680244 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7fac7005f7b0 con 0x7fac88151d20 2024-04-25T19:18:24.738 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":24,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:18:24.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.739+0000 7fac8d9a8640 1 -- 172.21.15.76:0/2632680244 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fac680744b0 msgr2=0x7fac680769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:24.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.739+0000 7fac8d9a8640 1 --2- 172.21.15.76:0/2632680244 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fac680744b0 0x7fac680769c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fac78033000 tx=0x7fac7802e000 comp rx=0 tx=0).stop 2024-04-25T19:18:24.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.739+0000 7fac8d9a8640 1 -- 172.21.15.76:0/2632680244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac88151d20 msgr2=0x7fac8815f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:24.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.739+0000 7fac8d9a8640 1 --2- 172.21.15.76:0/2632680244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac88151d20 0x7fac8815f390 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fac7000f680 tx=0x7fac7000fb40 comp rx=0 tx=0).stop 2024-04-25T19:18:24.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.739+0000 7fac8d9a8640 1 -- 172.21.15.76:0/2632680244 shutdown_connections 2024-04-25T19:18:24.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.739+0000 7fac8d9a8640 1 --2- 172.21.15.76:0/2632680244 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fac680744b0 0x7fac680769c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:24.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.739+0000 7fac8d9a8640 1 --2- 172.21.15.76:0/2632680244 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac88152850 0x7fac88154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:24.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.739+0000 7fac8d9a8640 1 --2- 172.21.15.76:0/2632680244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac88151d20 0x7fac8815f390 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:24.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.739+0000 7fac8d9a8640 1 --2- 172.21.15.76:0/2632680244 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fac740039b0 0x7fac88155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:24.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.739+0000 7fac8d9a8640 1 -- 172.21.15.76:0/2632680244 >> 172.21.15.76:0/2632680244 conn(0x7fac8806c5c0 msgr2=0x7fac880c4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:24.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.740+0000 7fac8d9a8640 1 -- 172.21.15.76:0/2632680244 shutdown_connections 2024-04-25T19:18:24.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.740+0000 7fac8d9a8640 1 -- 172.21.15.76:0/2632680244 wait complete. 2024-04-25T19:18:24.758 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T19:18:24.759 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T19:18:24.759 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T19:18:24.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.928+0000 7f86081f3640 1 Processor -- start 2024-04-25T19:18:24.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.928+0000 7f86081f3640 1 -- start start 2024-04-25T19:18:24.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.930+0000 7f86081f3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f86000c9e20 0x7f86000c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:24.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.930+0000 7f86081f3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f860006fc50 0x7f8600070060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:24.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.930+0000 7f86081f3640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86000c3c10 con 0x7f86000c3480 2024-04-25T19:18:24.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.930+0000 7f86081f3640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f86000c8e60 con 0x7f860006fc50 2024-04-25T19:18:24.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.930+0000 7f86081f3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8600153920 con 0x7f86000c9e20 2024-04-25T19:18:24.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.930+0000 7f8605f8f640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f86000c3480 0x7f8600153210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:49432/0 (socket says 172.21.15.76:49432) 2024-04-25T19:18:24.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.930+0000 7f8605f8f640 1 -- 172.21.15.76:0/2948946380 learned_addr learned my addr 172.21.15.76:0/2948946380 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:18:24.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.930+0000 7f8606790640 1 --2- 172.21.15.76:0/2948946380 >> v2:172.21.15.105:3300/0 conn(0x7f860006fc50 0x7f8600070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:24.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.930+0000 7f860578e640 1 --2- 172.21.15.76:0/2948946380 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f86000c9e20 0x7f86000c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:24.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.930+0000 7f8604f8d640 1 -- 172.21.15.76:0/2948946380 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 651795442 0 0) 0x7f86000c3c10 con 0x7f86000c3480 2024-04-25T19:18:24.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.930+0000 7f8604f8d640 1 -- 172.21.15.76:0/2948946380 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85ec003520 con 0x7f86000c3480 2024-04-25T19:18:24.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.931+0000 7f8604f8d640 1 -- 172.21.15.76:0/2948946380 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 955463803 0 0) 0x7f85ec003520 con 0x7f86000c3480 2024-04-25T19:18:24.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.931+0000 7f8604f8d640 1 -- 172.21.15.76:0/2948946380 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f86000c9e20 msgr2=0x7f86000c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:24.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.931+0000 7f8604f8d640 1 --2- 172.21.15.76:0/2948946380 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f86000c9e20 0x7f86000c5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:24.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.931+0000 7f8604f8d640 1 -- 172.21.15.76:0/2948946380 >> v2:172.21.15.105:3300/0 conn(0x7f860006fc50 msgr2=0x7f8600070060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:24.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.931+0000 7f8604f8d640 1 --2- 172.21.15.76:0/2948946380 >> v2:172.21.15.105:3300/0 conn(0x7f860006fc50 0x7f8600070060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:24.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.931+0000 7f8604f8d640 1 -- 172.21.15.76:0/2948946380 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86000c3950 con 0x7f86000c3480 2024-04-25T19:18:24.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.931+0000 7f8604f8d640 1 -- 172.21.15.76:0/2948946380 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f85fc001670 con 0x7f86000c3480 2024-04-25T19:18:24.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.931+0000 7f8604f8d640 1 -- 172.21.15.76:0/2948946380 >> v1:172.21.15.76:6789/0 conn(0x7f86000c3480 legacy=0x7f8600153210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:24.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.931+0000 7f8604f8d640 1 --2- 172.21.15.76:0/2948946380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85ec0039b0 0x7f85ec003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:24.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.931+0000 7f8604f8d640 1 --2- 172.21.15.76:0/2948946380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ec004800 0x7f85ec004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:24.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.931+0000 7f8604f8d640 1 --2- 172.21.15.76:0/2948946380 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f85ec005150 0x7f85ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:24.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.931+0000 7f8604f8d640 1 -- 172.21.15.76:0/2948946380 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec00bf00 con 0x7f85ec004800 2024-04-25T19:18:24.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.931+0000 7f8604f8d640 1 -- 172.21.15.76:0/2948946380 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec00c080 con 0x7f85ec0039b0 2024-04-25T19:18:24.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.931+0000 7f8604f8d640 1 -- 172.21.15.76:0/2948946380 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f85ec00c280 con 0x7f85ec005150 2024-04-25T19:18:24.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.931+0000 7f8606790640 1 --2- 172.21.15.76:0/2948946380 >> v2:172.21.15.105:3300/0 conn(0x7f860006fc50 0x7f8600070060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:18:24.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.931+0000 7f860578e640 1 --2- 172.21.15.76:0/2948946380 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f86000c9e20 0x7f86000c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:18:24.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.931+0000 7f860578e640 1 --2- 172.21.15.76:0/2948946380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ec004800 0x7f85ec004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:24.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.931+0000 7f8605f8f640 1 --2- 172.21.15.76:0/2948946380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85ec0039b0 0x7f85ec003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:24.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.931+0000 7f8606790640 1 --2- 172.21.15.76:0/2948946380 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f85ec005150 0x7f85ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:24.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.931+0000 7f860578e640 1 -- 172.21.15.76:0/2948946380 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f85ec005150 msgr2=0x7f85ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:24.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.931+0000 7f860578e640 1 --2- 172.21.15.76:0/2948946380 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f85ec005150 0x7f85ec00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:24.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.931+0000 7f860578e640 1 -- 172.21.15.76:0/2948946380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85ec0039b0 msgr2=0x7f85ec003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:24.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.931+0000 7f860578e640 1 --2- 172.21.15.76:0/2948946380 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85ec0039b0 0x7f85ec003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:24.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.931+0000 7f860578e640 1 -- 172.21.15.76:0/2948946380 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ec003520 con 0x7f85ec004800 2024-04-25T19:18:24.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.932+0000 7f860578e640 1 --2- 172.21.15.76:0/2948946380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ec004800 0x7f85ec004c10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f86000c8210 tx=0x7f85ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66c8097a12e70c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:24.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.932+0000 7f8604f8d640 1 -- 172.21.15.76:0/2948946380 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0036070 con 0x7f85ec004800 2024-04-25T19:18:24.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.932+0000 7f8604f8d640 1 -- 172.21.15.76:0/2948946380 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85f0004030 con 0x7f85ec004800 2024-04-25T19:18:24.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.932+0000 7f8604f8d640 1 -- 172.21.15.76:0/2948946380 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f00097d0 con 0x7f85ec004800 2024-04-25T19:18:24.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.933+0000 7f86081f3640 1 -- 172.21.15.76:0/2948946380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ec004800 msgr2=0x7f85ec004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:24.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.933+0000 7f86081f3640 1 --2- 172.21.15.76:0/2948946380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ec004800 0x7f85ec004c10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f86000c8210 tx=0x7f85ec005830 comp rx=0 tx=0).stop 2024-04-25T19:18:24.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.933+0000 7f8606790640 1 -- 172.21.15.76:0/2948946380 reap_dead start 2024-04-25T19:18:24.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.933+0000 7f86081f3640 1 -- 172.21.15.76:0/2948946380 shutdown_connections 2024-04-25T19:18:24.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.933+0000 7f86081f3640 1 -- 172.21.15.76:0/2948946380 >> 172.21.15.76:0/2948946380 conn(0x7f860006c5c0 msgr2=0x7f860006c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:24.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.933+0000 7f86081f3640 1 -- 172.21.15.76:0/2948946380 shutdown_connections 2024-04-25T19:18:24.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.934+0000 7f86081f3640 1 -- 172.21.15.76:0/2948946380 wait complete. 2024-04-25T19:18:24.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.936+0000 7f86081f3640 1 Processor -- start 2024-04-25T19:18:24.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.936+0000 7f86081f3640 1 -- start start 2024-04-25T19:18:24.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.938+0000 7f86081f3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ec004800 0x7f8600146910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:24.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.938+0000 7f86081f3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8600165e40 0x7f8600150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:24.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.938+0000 7f86081f3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86000c5ec0 0x7f8600141010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:24.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.938+0000 7f86081f3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f86000caad0 con 0x7f85ec004800 2024-04-25T19:18:24.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.938+0000 7f86081f3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f86000ca7f0 con 0x7f86000c5ec0 2024-04-25T19:18:24.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.938+0000 7f86081f3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f86000c5180 con 0x7f8600165e40 2024-04-25T19:18:24.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.938+0000 7f8605f8f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ec004800 0x7f8600146910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:24.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.938+0000 7f8605f8f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ec004800 0x7f8600146910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34342/0 (socket says 172.21.15.76:34342) 2024-04-25T19:18:24.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.938+0000 7f8605f8f640 1 -- 172.21.15.76:0/3282456735 learned_addr learned my addr 172.21.15.76:0/3282456735 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:24.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.938+0000 7f8606790640 1 --2- 172.21.15.76:0/3282456735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86000c5ec0 0x7f8600141010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:24.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.938+0000 7f860578e640 1 --2- 172.21.15.76:0/3282456735 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8600165e40 0x7f8600150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:24.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.938+0000 7f8605f8f640 1 -- 172.21.15.76:0/3282456735 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8600165e40 msgr2=0x7f8600150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:24.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.938+0000 7f8605f8f640 1 --2- 172.21.15.76:0/3282456735 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8600165e40 0x7f8600150fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:24.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.938+0000 7f8605f8f640 1 -- 172.21.15.76:0/3282456735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86000c5ec0 msgr2=0x7f8600141010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:24.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.938+0000 7f8605f8f640 1 --2- 172.21.15.76:0/3282456735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86000c5ec0 0x7f8600141010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:24.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.938+0000 7f8605f8f640 1 -- 172.21.15.76:0/3282456735 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8600146e50 con 0x7f85ec004800 2024-04-25T19:18:24.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.938+0000 7f8606790640 1 --2- 172.21.15.76:0/3282456735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86000c5ec0 0x7f8600141010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:18:24.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.939+0000 7f8605f8f640 1 --2- 172.21.15.76:0/3282456735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ec004800 0x7f8600146910 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f85fc00e7b0 tx=0x7f85fc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:24.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.939+0000 7f85f67fc640 1 -- 172.21.15.76:0/3282456735 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc00c870 con 0x7f85ec004800 2024-04-25T19:18:24.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.939+0000 7f85f67fc640 1 -- 172.21.15.76:0/3282456735 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85fc018070 con 0x7f85ec004800 2024-04-25T19:18:24.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.939+0000 7f86081f3640 1 -- 172.21.15.76:0/3282456735 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8600147060 con 0x7f85ec004800 2024-04-25T19:18:24.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.939+0000 7f85f67fc640 1 -- 172.21.15.76:0/3282456735 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc00cc30 con 0x7f85ec004800 2024-04-25T19:18:24.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.939+0000 7f86081f3640 1 -- 172.21.15.76:0/3282456735 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8600146110 con 0x7f85ec004800 2024-04-25T19:18:24.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.940+0000 7f86081f3640 1 --2- 172.21.15.76:0/3282456735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8600141550 0x7f86001f5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:24.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.940+0000 7f86081f3640 1 -- 172.21.15.76:0/3282456735 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f86001664d0 con 0x7f8600141550 2024-04-25T19:18:24.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.940+0000 7f860578e640 1 --2- 172.21.15.76:0/3282456735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8600141550 0x7f86001f5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:24.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.941+0000 7f860578e640 1 -- 172.21.15.76:0/3282456735 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f860006d6a0 con 0x7f8600141550 2024-04-25T19:18:24.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.941+0000 7f860578e640 1 --2- 172.21.15.76:0/3282456735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8600141550 0x7f86001f5b90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f860006e700 tx=0x7f85f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:24.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.941+0000 7f85f67fc640 1 -- 172.21.15.76:0/3282456735 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0042070 con 0x7f8600141550 2024-04-25T19:18:24.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.942+0000 7f85f67fc640 1 -- 172.21.15.76:0/3282456735 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f85fc01d460 con 0x7f85ec004800 2024-04-25T19:18:24.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.942+0000 7f85f67fc640 1 --2- 172.21.15.76:0/3282456735 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f85e40743b0 0x7f85e40768c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:24.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.943+0000 7f85f67fc640 1 -- 172.21.15.76:0/3282456735 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f85fc0950d0 con 0x7f85ec004800 2024-04-25T19:18:24.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.943+0000 7f85f67fc640 1 -- 172.21.15.76:0/3282456735 <== mon.0 v2:172.21.15.76:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f85f0003a20 con 0x7f8600141550 2024-04-25T19:18:24.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.943+0000 7f85f67fc640 1 -- 172.21.15.76:0/3282456735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8600141550 msgr2=0x7f86001f5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:24.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.943+0000 7f85f67fc640 1 --2- 172.21.15.76:0/3282456735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8600141550 0x7f86001f5b90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f860006e700 tx=0x7f85f0004730 comp rx=0 tx=0).stop 2024-04-25T19:18:24.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.943+0000 7f85f67fc640 1 -- 172.21.15.76:0/3282456735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8600141550 msgr2=0x7f86001f5b90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:18:24.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.943+0000 7f85f67fc640 1 --2- 172.21.15.76:0/3282456735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8600141550 0x7f86001f5b90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:24.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.943+0000 7f8606790640 1 --2- 172.21.15.76:0/3282456735 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f85e40743b0 0x7f85e40768c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:24.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.944+0000 7f8606790640 1 --2- 172.21.15.76:0/3282456735 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f85e40743b0 0x7f85e40768c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8600072050 tx=0x7f85f800b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:24.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.951+0000 7f86081f3640 1 --2- 172.21.15.76:0/3282456735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86001f60d0 0x7f86001f64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:24.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.951+0000 7f86081f3640 1 -- 172.21.15.76:0/3282456735 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f86001f6a20 con 0x7f86001f60d0 2024-04-25T19:18:24.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.952+0000 7f860578e640 1 --2- 172.21.15.76:0/3282456735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86001f60d0 0x7f86001f64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:24.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.952+0000 7f860578e640 1 -- 172.21.15.76:0/3282456735 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f860006a390 con 0x7f86001f60d0 2024-04-25T19:18:24.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.952+0000 7f860578e640 1 --2- 172.21.15.76:0/3282456735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86001f60d0 0x7f86001f64e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f85f0006f80 tx=0x7f85f0008000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:24.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.952+0000 7f85f67fc640 1 -- 172.21.15.76:0/3282456735 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0036040 con 0x7f86001f60d0 2024-04-25T19:18:24.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.953+0000 7f85f67fc640 1 -- 172.21.15.76:0/3282456735 <== mon.0 v2:172.21.15.76:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4555 (secure 0 0 0) 0x7f85f00066b0 con 0x7f86001f60d0 2024-04-25T19:18:24.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.953+0000 7f85f67fc640 1 -- 172.21.15.76:0/3282456735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86001f60d0 msgr2=0x7f86001f64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:24.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.953+0000 7f85f67fc640 1 --2- 172.21.15.76:0/3282456735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86001f60d0 0x7f86001f64e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f85f0006f80 tx=0x7f85f0008000 comp rx=0 tx=0).stop 2024-04-25T19:18:24.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.953+0000 7f85f67fc640 1 -- 172.21.15.76:0/3282456735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86001f60d0 msgr2=0x7f86001f64e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:18:24.955 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:18:24.955 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:18:24.955 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:18:24.955 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "leader", 2024-04-25T19:18:24.955 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 24, 2024-04-25T19:18:24.955 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 4063, 2024-04-25T19:18:24.956 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:18:24.956 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:18:24.956 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-04-25T19:18:24.956 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:18:24.956 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:24.956 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 2, 2024-04-25T19:18:24.956 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:18:24.956 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:18:24.956 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:18:24.956 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:24.956 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:24.956 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:24.956 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:24.956 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:24.956 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:24.957 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:24.957 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:24.957 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:24.957 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:24.957 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:24.957 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:24.957 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:18:24.957 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:18:24.957 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:24.957 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:24.957 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:24.957 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:24.957 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:24.957 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:24.957 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:24.957 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:24.958 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:24.958 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:24.958 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:24.958 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:24.958 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:24.958 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:18:24.958 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:18:24.958 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:18:24.958 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:18:24.958 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:18:24.958 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:18:24.958 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:18:24.958 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:18:24.958 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:18:24.959 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:18:24.959 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:18:24.959 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:18:24.959 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:18:24.959 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:18:24.959 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:18:24.959 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:18:24.959 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:18:24.959 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:24.959 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:24.959 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:24.959 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:24.959 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:24.959 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:24.960 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:24.960 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:24.960 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:24.960 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:24.960 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:24.961 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:24.961 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:18:24.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:24.961 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:18:24.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:24.962 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:18:24.962 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:18:24.962 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:24.962 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:24.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:24.962 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:24.962 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:18:24.962 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:24.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:24.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:24.962 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:24.962 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:18:24.962 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:24.962 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:24.963 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:24.963 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:24.963 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:18:24.963 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:18:24.963 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:24.963 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:24.963 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:24.963 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:24.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:24.963 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:18:24.963 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:18:24.963 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:24.963 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:24.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:24.963 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:24.964 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:18:24.964 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:24.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:24.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:24.964 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:24.964 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:18:24.964 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:24.964 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:24.964 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:24.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:24.964 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:18:24.964 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:18:24.964 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:24.964 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:24.964 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:24.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:24.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:24.965 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:18:24.965 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:18:24.965 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:24.965 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:24.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:24.965 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:24.965 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:18:24.965 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:24.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:24.966 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:24.966 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:24.966 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:18:24.966 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:24.966 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:24.966 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:24.966 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:24.966 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:18:24.966 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:18:24.966 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:24.966 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:24.967 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:24.967 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:24.967 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:24.967 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:24.967 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:18:24.967 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:18:24.967 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:24.967 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:24.967 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:24.967 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:18:24.967 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:24.967 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:24.968 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:18:24.968 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:24.968 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:24.968 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:24.968 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 2 2024-04-25T19:18:24.968 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:24.968 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:24.968 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:24.968 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:18:24.968 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:18:24.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.953+0000 7f85f67fc640 1 --2- 172.21.15.76:0/3282456735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86001f60d0 0x7f86001f64e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:24.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.954+0000 7f86081f3640 1 -- 172.21.15.76:0/3282456735 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f85e40743b0 msgr2=0x7f85e40768c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:24.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.954+0000 7f86081f3640 1 --2- 172.21.15.76:0/3282456735 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f85e40743b0 0x7f85e40768c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8600072050 tx=0x7f85f800b040 comp rx=0 tx=0).stop 2024-04-25T19:18:24.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.954+0000 7f86081f3640 1 -- 172.21.15.76:0/3282456735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ec004800 msgr2=0x7f8600146910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:24.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.954+0000 7f86081f3640 1 --2- 172.21.15.76:0/3282456735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ec004800 0x7f8600146910 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f85fc00e7b0 tx=0x7f85fc00ec70 comp rx=0 tx=0).stop 2024-04-25T19:18:24.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.954+0000 7f8606790640 1 -- 172.21.15.76:0/3282456735 reap_dead start 2024-04-25T19:18:24.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.954+0000 7f8606790640 1 -- 172.21.15.76:0/3282456735 reap_dead start 2024-04-25T19:18:24.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.954+0000 7f86081f3640 1 -- 172.21.15.76:0/3282456735 shutdown_connections 2024-04-25T19:18:24.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.954+0000 7f86081f3640 1 -- 172.21.15.76:0/3282456735 >> 172.21.15.76:0/3282456735 conn(0x7f860006c5c0 msgr2=0x7f8600075d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:24.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.954+0000 7f86081f3640 1 -- 172.21.15.76:0/3282456735 shutdown_connections 2024-04-25T19:18:24.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:24.954+0000 7f86081f3640 1 -- 172.21.15.76:0/3282456735 wait complete. 2024-04-25T19:18:24.973 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T19:18:24.973 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T19:18:24.973 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T19:18:25.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.132+0000 7fb8ccef9640 1 Processor -- start 2024-04-25T19:18:25.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.132+0000 7fb8ccef9640 1 -- start start 2024-04-25T19:18:25.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.134+0000 7fb8ccef9640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8c8159840 0x7fb8c8153b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:25.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.134+0000 7fb8ccef9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb8c80c5b20 0x7fb8c815d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:25.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.134+0000 7fb8ccef9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb8c80c96c0 0x7fb8c80c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:25.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.134+0000 7fb8ccef9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb8c815ce30 con 0x7fb8c80c5b20 2024-04-25T19:18:25.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.134+0000 7fb8ccef9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fb8c815ddc0 con 0x7fb8c80c96c0 2024-04-25T19:18:25.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.134+0000 7fb8ccef9640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb8c80715d0 con 0x7fb8c8159840 2024-04-25T19:18:25.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.134+0000 7fb8bffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb8c80c5b20 0x7fb8c815d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:25.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.135+0000 7fb8bffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb8c80c5b20 0x7fb8c815d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34362/0 (socket says 172.21.15.76:34362) 2024-04-25T19:18:25.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.135+0000 7fb8bffff640 1 -- 172.21.15.76:0/38860194 learned_addr learned my addr 172.21.15.76:0/38860194 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:25.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.135+0000 7fb8bffff640 1 -- 172.21.15.76:0/38860194 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8c8159840 msgr2=0x7fb8c8153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:25.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.135+0000 7fb8bffff640 1 --2- 172.21.15.76:0/38860194 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8c8159840 0x7fb8c8153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.135+0000 7fb8bffff640 1 -- 172.21.15.76:0/38860194 >> v2:172.21.15.105:3300/0 conn(0x7fb8c80c96c0 msgr2=0x7fb8c80c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:25.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.135+0000 7fb8bffff640 1 --2- 172.21.15.76:0/38860194 >> v2:172.21.15.105:3300/0 conn(0x7fb8c80c96c0 0x7fb8c80c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.135+0000 7fb8bffff640 1 -- 172.21.15.76:0/38860194 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c80718b0 con 0x7fb8c80c5b20 2024-04-25T19:18:25.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.136+0000 7fb8bffff640 1 --2- 172.21.15.76:0/38860194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8c80c5b20 0x7fb8c815d880 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8072510 tx=0x7fb8b002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de97189429448d3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:25.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.136+0000 7fb8c5d9b640 1 -- 172.21.15.76:0/38860194 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0037070 con 0x7fb8c80c5b20 2024-04-25T19:18:25.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.136+0000 7fb8c5d9b640 1 -- 172.21.15.76:0/38860194 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b0032440 con 0x7fb8c80c5b20 2024-04-25T19:18:25.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.136+0000 7fb8c5d9b640 1 -- 172.21.15.76:0/38860194 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0032810 con 0x7fb8c80c5b20 2024-04-25T19:18:25.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.137+0000 7fb8ccef9640 1 -- 172.21.15.76:0/38860194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8c80c5b20 msgr2=0x7fb8c815d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:25.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.137+0000 7fb8ccef9640 1 --2- 172.21.15.76:0/38860194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8c80c5b20 0x7fb8c815d880 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8072510 tx=0x7fb8b002a1e0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.137+0000 7fb8ccef9640 1 -- 172.21.15.76:0/38860194 shutdown_connections 2024-04-25T19:18:25.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.137+0000 7fb8ccef9640 1 --2- 172.21.15.76:0/38860194 >> v2:172.21.15.105:3300/0 conn(0x7fb8c80c96c0 0x7fb8c80c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.137+0000 7fb8ccef9640 1 --2- 172.21.15.76:0/38860194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8c80c5b20 0x7fb8c815d880 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.137+0000 7fb8ccef9640 1 --2- 172.21.15.76:0/38860194 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8c8159840 0x7fb8c8153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.137+0000 7fb8ccef9640 1 -- 172.21.15.76:0/38860194 >> 172.21.15.76:0/38860194 conn(0x7fb8c80cecc0 msgr2=0x7fb8c80cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:25.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.137+0000 7fb8ccef9640 1 -- 172.21.15.76:0/38860194 shutdown_connections 2024-04-25T19:18:25.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.137+0000 7fb8ccef9640 1 -- 172.21.15.76:0/38860194 wait complete. 2024-04-25T19:18:25.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.139+0000 7fb8ccef9640 1 Processor -- start 2024-04-25T19:18:25.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.139+0000 7fb8ccef9640 1 -- start start 2024-04-25T19:18:25.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.140+0000 7fb8ccef9640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8c80c5b20 0x7fb8c8152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:25.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.141+0000 7fb8ccef9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8c80c96c0 0x7fb8c81e6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:25.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.141+0000 7fb8ccef9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8c8159840 0x7fb8c815c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:25.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.141+0000 7fb8ccef9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c8071270 con 0x7fb8c8159840 2024-04-25T19:18:25.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.141+0000 7fb8ccef9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c815d080 con 0x7fb8c80c96c0 2024-04-25T19:18:25.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.141+0000 7fb8ccef9640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb8c80c9ad0 con 0x7fb8c80c5b20 2024-04-25T19:18:25.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.141+0000 7fb8c6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8c8159840 0x7fb8c815c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:25.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.141+0000 7fb8bffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8c80c96c0 0x7fb8c81e6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:25.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.141+0000 7fb8bffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8c80c96c0 0x7fb8c81e6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:45310/0 (socket says 172.21.15.76:45310) 2024-04-25T19:18:25.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.141+0000 7fb8c6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8c8159840 0x7fb8c815c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34374/0 (socket says 172.21.15.76:34374) 2024-04-25T19:18:25.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.141+0000 7fb8bffff640 1 -- 172.21.15.76:0/3500194097 learned_addr learned my addr 172.21.15.76:0/3500194097 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:25.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.141+0000 7fb8c659c640 1 --2- 172.21.15.76:0/3500194097 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8c80c5b20 0x7fb8c8152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:25.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.141+0000 7fb8c6d9d640 1 -- 172.21.15.76:0/3500194097 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8c80c5b20 msgr2=0x7fb8c8152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:25.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.141+0000 7fb8c6d9d640 1 --2- 172.21.15.76:0/3500194097 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb8c80c5b20 0x7fb8c8152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.141+0000 7fb8c6d9d640 1 -- 172.21.15.76:0/3500194097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8c80c96c0 msgr2=0x7fb8c81e6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:25.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.141+0000 7fb8c6d9d640 1 --2- 172.21.15.76:0/3500194097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8c80c96c0 0x7fb8c81e6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.141+0000 7fb8c6d9d640 1 -- 172.21.15.76:0/3500194097 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c814dd30 con 0x7fb8c8159840 2024-04-25T19:18:25.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.141+0000 7fb8c6d9d640 1 --2- 172.21.15.76:0/3500194097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8c8159840 0x7fb8c815c290 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000bdf0 tx=0x7fb8c000bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:25.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.142+0000 7fb8beffd640 1 -- 172.21.15.76:0/3500194097 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0019070 con 0x7fb8c8159840 2024-04-25T19:18:25.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.142+0000 7fb8beffd640 1 -- 172.21.15.76:0/3500194097 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8c0014440 con 0x7fb8c8159840 2024-04-25T19:18:25.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.142+0000 7fb8ccef9640 1 -- 172.21.15.76:0/3500194097 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8c8151390 con 0x7fb8c8159840 2024-04-25T19:18:25.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.142+0000 7fb8beffd640 1 -- 172.21.15.76:0/3500194097 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0014830 con 0x7fb8c8159840 2024-04-25T19:18:25.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.142+0000 7fb8ccef9640 1 -- 172.21.15.76:0/3500194097 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8c8151d40 con 0x7fb8c8159840 2024-04-25T19:18:25.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.143+0000 7fb8beffd640 1 -- 172.21.15.76:0/3500194097 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb8c001c070 con 0x7fb8c8159840 2024-04-25T19:18:25.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.143+0000 7fb8ccef9640 1 --2- 172.21.15.76:0/3500194097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8c8151f10 0x7fb8c815c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:25.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.144+0000 7fb8ccef9640 1 -- 172.21.15.76:0/3500194097 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c8152a80 con 0x7fb8c8151f10 2024-04-25T19:18:25.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.144+0000 7fb8c659c640 1 --2- 172.21.15.76:0/3500194097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8c8151f10 0x7fb8c815c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:25.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.144+0000 7fb8beffd640 1 --2- 172.21.15.76:0/3500194097 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb8a80744e0 0x7fb8a80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:25.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.144+0000 7fb8beffd640 1 -- 172.21.15.76:0/3500194097 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fb8c0095020 con 0x7fb8c8159840 2024-04-25T19:18:25.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.145+0000 7fb8c659c640 1 -- 172.21.15.76:0/3500194097 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb8c80cb9e0 con 0x7fb8c8151f10 2024-04-25T19:18:25.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.145+0000 7fb8bffff640 1 --2- 172.21.15.76:0/3500194097 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb8a80744e0 0x7fb8a80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:25.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.145+0000 7fb8c659c640 1 --2- 172.21.15.76:0/3500194097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8c8151f10 0x7fb8c815c7d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800e590 tx=0x7fb8b800ea50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:25.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.145+0000 7fb8beffd640 1 -- 172.21.15.76:0/3500194097 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8017070 con 0x7fb8c8151f10 2024-04-25T19:18:25.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.145+0000 7fb8bffff640 1 --2- 172.21.15.76:0/3500194097 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb8a80744e0 0x7fb8a80769f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb8b00090b0 tx=0x7fb8b002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:25.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.146+0000 7fb8beffd640 1 -- 172.21.15.76:0/3500194097 <== mon.1 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb8b8013680 con 0x7fb8c8151f10 2024-04-25T19:18:25.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.146+0000 7fb8beffd640 1 -- 172.21.15.76:0/3500194097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8c8151f10 msgr2=0x7fb8c815c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:25.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.146+0000 7fb8beffd640 1 --2- 172.21.15.76:0/3500194097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8c8151f10 0x7fb8c815c7d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800e590 tx=0x7fb8b800ea50 comp rx=0 tx=0).stop 2024-04-25T19:18:25.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.146+0000 7fb8beffd640 1 -- 172.21.15.76:0/3500194097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8c8151f10 msgr2=0x7fb8c815c7d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:18:25.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.146+0000 7fb8beffd640 1 --2- 172.21.15.76:0/3500194097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8c8151f10 0x7fb8c815c7d0 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800e590 tx=0x7fb8b800ea50 comp rx=0 tx=0).stop 2024-04-25T19:18:25.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.156+0000 7fb8ccef9640 1 --2- 172.21.15.76:0/3500194097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8c80ce5b0 0x7fb8c81f0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:25.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.156+0000 7fb8ccef9640 1 -- 172.21.15.76:0/3500194097 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c8151ba0 con 0x7fb8c80ce5b0 2024-04-25T19:18:25.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.157+0000 7fb8c659c640 1 --2- 172.21.15.76:0/3500194097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8c80ce5b0 0x7fb8c81f0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:25.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.157+0000 7fb8c659c640 1 -- 172.21.15.76:0/3500194097 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb8c81601a0 con 0x7fb8c80ce5b0 2024-04-25T19:18:25.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.158+0000 7fb8c659c640 1 --2- 172.21.15.76:0/3500194097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8c80ce5b0 0x7fb8c81f0ac0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800e590 tx=0x7fb8b8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:25.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.158+0000 7fb8beffd640 1 -- 172.21.15.76:0/3500194097 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8017070 con 0x7fb8c80ce5b0 2024-04-25T19:18:25.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.158+0000 7fb8beffd640 1 -- 172.21.15.76:0/3500194097 <== mon.1 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5064 (secure 0 0 0) 0x7fb8b8006250 con 0x7fb8c80ce5b0 2024-04-25T19:18:25.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.158+0000 7fb8beffd640 1 -- 172.21.15.76:0/3500194097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8c80ce5b0 msgr2=0x7fb8c81f0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:25.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.158+0000 7fb8beffd640 1 --2- 172.21.15.76:0/3500194097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8c80ce5b0 0x7fb8c81f0ac0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800e590 tx=0x7fb8b8004730 comp rx=0 tx=0).stop 2024-04-25T19:18:25.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.158+0000 7fb8beffd640 1 -- 172.21.15.76:0/3500194097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8c80ce5b0 msgr2=0x7fb8c81f0ac0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:18:25.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.158+0000 7fb8beffd640 1 --2- 172.21.15.76:0/3500194097 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8c80ce5b0 0x7fb8c81f0ac0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.160 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:18:25.160 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:18:25.160 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:18:25.161 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "peon", 2024-04-25T19:18:25.161 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 24, 2024-04-25T19:18:25.161 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 56650, 2024-04-25T19:18:25.161 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:18:25.161 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:18:25.161 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-04-25T19:18:25.161 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:18:25.161 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:25.161 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 2, 2024-04-25T19:18:25.161 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:18:25.161 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:18:25.161 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:18:25.161 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:25.161 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:25.161 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:25.162 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:25.162 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:25.162 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:25.162 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:25.162 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:25.162 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:25.162 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:25.162 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:25.162 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:25.162 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:18:25.162 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:18:25.162 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:25.162 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:25.162 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:25.163 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:25.163 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:25.163 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:25.163 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:25.163 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:25.163 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:25.163 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:25.163 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:25.163 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:25.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:25.163 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:18:25.163 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:18:25.163 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:18:25.164 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:18:25.164 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:18:25.164 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:18:25.164 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:18:25.164 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:18:25.164 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:18:25.164 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:18:25.164 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:18:25.164 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:18:25.164 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:18:25.164 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:18:25.164 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:18:25.164 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:18:25.164 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:18:25.164 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:25.165 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:25.165 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:25.165 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:25.165 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:25.165 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:25.165 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:25.165 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:25.165 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:25.165 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:25.165 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:25.165 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:25.165 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:18:25.165 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:25.165 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:18:25.165 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.166 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:18:25.166 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:18:25.166 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:25.166 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:25.166 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.166 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:25.166 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:18:25.166 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:25.166 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:25.166 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.166 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:25.166 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:18:25.166 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:25.166 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:25.167 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:25.167 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:25.167 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:18:25.167 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:18:25.167 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:25.167 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:25.167 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:25.167 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:25.167 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.167 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:18:25.167 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:18:25.167 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:25.167 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:25.167 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.167 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:25.168 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:18:25.168 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:25.168 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:25.168 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.168 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:25.168 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:18:25.168 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:25.168 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:25.168 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:25.168 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:25.168 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:18:25.168 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:18:25.168 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:25.168 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:25.168 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:25.168 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:25.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.169 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:18:25.169 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:18:25.169 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:25.169 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:25.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.169 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:25.169 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:18:25.169 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:25.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:25.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.169 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:25.169 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:18:25.169 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:25.169 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:25.170 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:25.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:25.170 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:18:25.170 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:18:25.170 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:25.170 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:25.170 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:25.170 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:25.170 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:25.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:25.170 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:18:25.170 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:18:25.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.170 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:25.171 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:25.171 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:18:25.171 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:25.171 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:25.171 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": [ 2024-04-25T19:18:25.171 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.171 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:25.171 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:25.171 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 3 2024-04-25T19:18:25.171 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:25.171 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:25.171 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": [ 2024-04-25T19:18:25.171 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.171 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:25.172 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:25.172 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 3 2024-04-25T19:18:25.172 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:25.172 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:25.172 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:18:25.172 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.172 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:25.172 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:25.172 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 3 2024-04-25T19:18:25.172 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:25.172 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:25.172 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": [ 2024-04-25T19:18:25.172 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.172 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:25.172 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:25.172 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 2 2024-04-25T19:18:25.173 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:25.173 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:25.173 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:25.173 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:18:25.173 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:18:25.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.159+0000 7fb8ccef9640 1 -- 172.21.15.76:0/3500194097 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb8a80744e0 msgr2=0x7fb8a80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:25.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.159+0000 7fb8ccef9640 1 --2- 172.21.15.76:0/3500194097 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb8a80744e0 0x7fb8a80769f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb8b00090b0 tx=0x7fb8b002e000 comp rx=0 tx=0).stop 2024-04-25T19:18:25.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.159+0000 7fb8ccef9640 1 -- 172.21.15.76:0/3500194097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8c8159840 msgr2=0x7fb8c815c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:25.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.159+0000 7fb8ccef9640 1 --2- 172.21.15.76:0/3500194097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8c8159840 0x7fb8c815c290 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000bdf0 tx=0x7fb8c000bee0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.159+0000 7fb8c6d9d640 1 -- 172.21.15.76:0/3500194097 reap_dead start 2024-04-25T19:18:25.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.159+0000 7fb8c6d9d640 1 -- 172.21.15.76:0/3500194097 reap_dead start 2024-04-25T19:18:25.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.159+0000 7fb8ccef9640 1 -- 172.21.15.76:0/3500194097 shutdown_connections 2024-04-25T19:18:25.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.159+0000 7fb8ccef9640 1 -- 172.21.15.76:0/3500194097 >> 172.21.15.76:0/3500194097 conn(0x7fb8c80cecc0 msgr2=0x7fb8c80c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:25.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.159+0000 7fb8ccef9640 1 -- 172.21.15.76:0/3500194097 shutdown_connections 2024-04-25T19:18:25.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.159+0000 7fb8ccef9640 1 -- 172.21.15.76:0/3500194097 wait complete. 2024-04-25T19:18:25.181 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T19:18:25.181 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T19:18:25.181 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T19:18:25.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.344+0000 7f73cfc61640 1 Processor -- start 2024-04-25T19:18:25.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.345+0000 7f73cfc61640 1 -- start start 2024-04-25T19:18:25.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.346+0000 7f73cfc61640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f73c80c3480 0x7f73c80c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:25.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.346+0000 7f73cfc61640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73c80c9e20 0x7f73c8153210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:25.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.346+0000 7f73cfc61640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f73c806fc50 0x7f73c8070060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:25.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.346+0000 7f73cfc61640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f73c80c8e60 con 0x7f73c806fc50 2024-04-25T19:18:25.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.346+0000 7f73cfc61640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f73c80c56c0 con 0x7f73c80c3480 2024-04-25T19:18:25.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.346+0000 7f73cfc61640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f73c80c3c10 con 0x7f73c80c9e20 2024-04-25T19:18:25.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.347+0000 7f73ce1fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f73c806fc50 0x7f73c8070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:25.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.347+0000 7f73ce1fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f73c806fc50 0x7f73c8070060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34380/0 (socket says 172.21.15.76:34380) 2024-04-25T19:18:25.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.347+0000 7f73ce1fe640 1 -- 172.21.15.76:0/1617258839 learned_addr learned my addr 172.21.15.76:0/1617258839 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:25.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.347+0000 7f73cd9fd640 1 --2- 172.21.15.76:0/1617258839 >> v2:172.21.15.105:3300/0 conn(0x7f73c80c3480 0x7f73c80c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:25.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.347+0000 7f73cd1fc640 1 --2- 172.21.15.76:0/1617258839 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73c80c9e20 0x7f73c8153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:25.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.348+0000 7f73ce1fe640 1 -- 172.21.15.76:0/1617258839 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73c80c9e20 msgr2=0x7f73c8153210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:25.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.348+0000 7f73ce1fe640 1 --2- 172.21.15.76:0/1617258839 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73c80c9e20 0x7f73c8153210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.348+0000 7f73ce1fe640 1 -- 172.21.15.76:0/1617258839 >> v2:172.21.15.105:3300/0 conn(0x7f73c80c3480 msgr2=0x7f73c80c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:25.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.348+0000 7f73ce1fe640 1 --2- 172.21.15.76:0/1617258839 >> v2:172.21.15.105:3300/0 conn(0x7f73c80c3480 0x7f73c80c5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.348+0000 7f73ce1fe640 1 -- 172.21.15.76:0/1617258839 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c80c3950 con 0x7f73c806fc50 2024-04-25T19:18:25.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.348+0000 7f73cd9fd640 1 --2- 172.21.15.76:0/1617258839 >> v2:172.21.15.105:3300/0 conn(0x7f73c80c3480 0x7f73c80c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:18:25.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.349+0000 7f73ce1fe640 1 --2- 172.21.15.76:0/1617258839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73c806fc50 0x7f73c8070060 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f73c8072610 tx=0x7f73c002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8437a704b3915f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:25.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.349+0000 7f73cc9fb640 1 -- 172.21.15.76:0/1617258839 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0036070 con 0x7f73c806fc50 2024-04-25T19:18:25.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.349+0000 7f73cc9fb640 1 -- 172.21.15.76:0/1617258839 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73c0031600 con 0x7f73c806fc50 2024-04-25T19:18:25.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.349+0000 7f73cc9fb640 1 -- 172.21.15.76:0/1617258839 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c00319d0 con 0x7f73c806fc50 2024-04-25T19:18:25.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.350+0000 7f73cfc61640 1 -- 172.21.15.76:0/1617258839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73c806fc50 msgr2=0x7f73c8070060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:25.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.350+0000 7f73cfc61640 1 --2- 172.21.15.76:0/1617258839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73c806fc50 0x7f73c8070060 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f73c8072610 tx=0x7f73c002a6c0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.351+0000 7f73cfc61640 1 -- 172.21.15.76:0/1617258839 shutdown_connections 2024-04-25T19:18:25.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.351+0000 7f73cfc61640 1 --2- 172.21.15.76:0/1617258839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73c806fc50 0x7f73c8070060 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.351+0000 7f73cfc61640 1 --2- 172.21.15.76:0/1617258839 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73c80c9e20 0x7f73c8153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.351+0000 7f73cfc61640 1 --2- 172.21.15.76:0/1617258839 >> v2:172.21.15.105:3300/0 conn(0x7f73c80c3480 0x7f73c80c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.351+0000 7f73cfc61640 1 -- 172.21.15.76:0/1617258839 >> 172.21.15.76:0/1617258839 conn(0x7f73c806c5c0 msgr2=0x7f73c806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:25.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.351+0000 7f73cfc61640 1 -- 172.21.15.76:0/1617258839 shutdown_connections 2024-04-25T19:18:25.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.351+0000 7f73cfc61640 1 -- 172.21.15.76:0/1617258839 wait complete. 2024-04-25T19:18:25.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.353+0000 7f73cfc61640 1 Processor -- start 2024-04-25T19:18:25.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.353+0000 7f73cfc61640 1 -- start start 2024-04-25T19:18:25.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.355+0000 7f73cfc61640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73c806fc50 0x7f73c81565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:25.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.355+0000 7f73cfc61640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73c80c3480 0x7f73c814f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:25.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.355+0000 7f73cfc61640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73c80c9e20 0x7f73c81550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:25.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.355+0000 7f73cfc61640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f73c80caad0 con 0x7f73c806fc50 2024-04-25T19:18:25.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.355+0000 7f73cfc61640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f73c80ca7f0 con 0x7f73c80c3480 2024-04-25T19:18:25.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.355+0000 7f73cfc61640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f73c8153750 con 0x7f73c80c9e20 2024-04-25T19:18:25.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.355+0000 7f73cd9fd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73c806fc50 0x7f73c81565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:25.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.355+0000 7f73ce1fe640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73c80c9e20 0x7f73c81550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:25.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.355+0000 7f73cd9fd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73c806fc50 0x7f73c81565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34396/0 (socket says 172.21.15.76:34396) 2024-04-25T19:18:25.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.355+0000 7f73cd9fd640 1 -- 172.21.15.76:0/1822235242 learned_addr learned my addr 172.21.15.76:0/1822235242 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:25.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.355+0000 7f73cd1fc640 1 --2- 172.21.15.76:0/1822235242 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73c80c3480 0x7f73c814f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:25.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.355+0000 7f73cd1fc640 1 -- 172.21.15.76:0/1822235242 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73c80c9e20 msgr2=0x7f73c81550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:25.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.355+0000 7f73cd1fc640 1 --2- 172.21.15.76:0/1822235242 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73c80c9e20 0x7f73c81550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.355+0000 7f73cd1fc640 1 -- 172.21.15.76:0/1822235242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73c806fc50 msgr2=0x7f73c81565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:25.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.355+0000 7f73cd1fc640 1 --2- 172.21.15.76:0/1822235242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73c806fc50 0x7f73c81565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.355+0000 7f73cd1fc640 1 -- 172.21.15.76:0/1822235242 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c814efe0 con 0x7f73c80c3480 2024-04-25T19:18:25.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.356+0000 7f73cd1fc640 1 --2- 172.21.15.76:0/1822235242 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73c80c3480 0x7f73c814f9a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f73b8009e90 tx=0x7f73b800ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:25.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.356+0000 7f73be7fc640 1 -- 172.21.15.76:0/1822235242 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b8020070 con 0x7f73c80c3480 2024-04-25T19:18:25.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.356+0000 7f73be7fc640 1 -- 172.21.15.76:0/1822235242 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73b800c8c0 con 0x7f73c80c3480 2024-04-25T19:18:25.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.356+0000 7f73be7fc640 1 -- 172.21.15.76:0/1822235242 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b800cc90 con 0x7f73c80c3480 2024-04-25T19:18:25.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.356+0000 7f73cfc61640 1 -- 172.21.15.76:0/1822235242 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73c80c76b0 con 0x7f73c80c3480 2024-04-25T19:18:25.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.356+0000 7f73cfc61640 1 -- 172.21.15.76:0/1822235242 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73c8155dd0 con 0x7f73c80c3480 2024-04-25T19:18:25.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.357+0000 7f73cfc61640 1 --2- 172.21.15.76:0/1822235242 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73c8155fd0 0x7f73c8149df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:25.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.357+0000 7f73cfc61640 1 -- 172.21.15.76:0/1822235242 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f73c8153e00 con 0x7f73c8155fd0 2024-04-25T19:18:25.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.358+0000 7f73cd9fd640 1 --2- 172.21.15.76:0/1822235242 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73c8155fd0 0x7f73c8149df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:25.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.358+0000 7f73cd9fd640 1 -- 172.21.15.76:0/1822235242 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f73c806d6a0 con 0x7f73c8155fd0 2024-04-25T19:18:25.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.359+0000 7f73be7fc640 1 -- 172.21.15.76:0/1822235242 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f73b8023070 con 0x7f73c80c3480 2024-04-25T19:18:25.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.359+0000 7f73cd9fd640 1 --2- 172.21.15.76:0/1822235242 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73c8155fd0 0x7f73c8149df0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f73c40119d0 tx=0x7f73c4011e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:25.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.360+0000 7f73be7fc640 1 --2- 172.21.15.76:0/1822235242 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f73ac074500 0x7f73ac076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:25.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.360+0000 7f73be7fc640 1 -- 172.21.15.76:0/1822235242 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f73b8098d50 con 0x7f73c80c3480 2024-04-25T19:18:25.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.360+0000 7f73be7fc640 1 -- 172.21.15.76:0/1822235242 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c400fb10 con 0x7f73c8155fd0 2024-04-25T19:18:25.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.360+0000 7f73ce1fe640 1 --2- 172.21.15.76:0/1822235242 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f73ac074500 0x7f73ac076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:25.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.360+0000 7f73be7fc640 1 -- 172.21.15.76:0/1822235242 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f73c400fcb0 con 0x7f73c8155fd0 2024-04-25T19:18:25.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.361+0000 7f73be7fc640 1 -- 172.21.15.76:0/1822235242 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73c8155fd0 msgr2=0x7f73c8149df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:25.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.361+0000 7f73be7fc640 1 --2- 172.21.15.76:0/1822235242 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73c8155fd0 0x7f73c8149df0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f73c40119d0 tx=0x7f73c4011e90 comp rx=0 tx=0).stop 2024-04-25T19:18:25.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.361+0000 7f73be7fc640 1 -- 172.21.15.76:0/1822235242 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73c8155fd0 msgr2=0x7f73c8149df0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:18:25.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.361+0000 7f73be7fc640 1 --2- 172.21.15.76:0/1822235242 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73c8155fd0 0x7f73c8149df0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.362+0000 7f73ce1fe640 1 --2- 172.21.15.76:0/1822235242 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f73ac074500 0x7f73ac076a10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f73c0033000 tx=0x7f73c0009000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:25.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.370+0000 7f73cfc61640 1 --2- 172.21.15.76:0/1822235242 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73c81555e0 0x7f73c814a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:25.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.370+0000 7f73cfc61640 1 -- 172.21.15.76:0/1822235242 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f73c806a390 con 0x7f73c81555e0 2024-04-25T19:18:25.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.371+0000 7f73cd9fd640 1 --2- 172.21.15.76:0/1822235242 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73c81555e0 0x7f73c814a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:25.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.371+0000 7f73cd9fd640 1 -- 172.21.15.76:0/1822235242 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f73c814a870 con 0x7f73c81555e0 2024-04-25T19:18:25.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.372+0000 7f73cd9fd640 1 --2- 172.21.15.76:0/1822235242 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73c81555e0 0x7f73c814a330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f73c40119d0 tx=0x7f73c40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:25.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.372+0000 7f73be7fc640 1 -- 172.21.15.76:0/1822235242 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c4022980 con 0x7f73c81555e0 2024-04-25T19:18:25.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.372+0000 7f73be7fc640 1 -- 172.21.15.76:0/1822235242 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4895 (secure 0 0 0) 0x7f73c400fe60 con 0x7f73c81555e0 2024-04-25T19:18:25.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.372+0000 7f73be7fc640 1 -- 172.21.15.76:0/1822235242 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73c81555e0 msgr2=0x7f73c814a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:25.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.372+0000 7f73be7fc640 1 --2- 172.21.15.76:0/1822235242 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73c81555e0 0x7f73c814a330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f73c40119d0 tx=0x7f73c40046c0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.372+0000 7f73be7fc640 1 -- 172.21.15.76:0/1822235242 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73c81555e0 msgr2=0x7f73c814a330 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:18:25.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.372+0000 7f73be7fc640 1 --2- 172.21.15.76:0/1822235242 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f73c81555e0 0x7f73c814a330 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f73c40119d0 tx=0x7f73c40046c0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.375 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:18:25.375 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:18:25.375 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:18:25.375 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "peon", 2024-04-25T19:18:25.375 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 24, 2024-04-25T19:18:25.375 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 109150, 2024-04-25T19:18:25.375 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:18:25.375 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:18:25.375 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-04-25T19:18:25.375 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:18:25.375 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:25.375 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 3, 2024-04-25T19:18:25.375 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:18:25.375 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:18:25.375 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:18:25.376 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:25.376 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:25.376 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:25.376 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:25.376 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:25.376 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:25.376 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:25.376 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:25.376 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:25.376 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:25.376 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:25.376 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:25.376 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:18:25.376 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:18:25.376 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:25.377 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:25.377 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:25.377 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:25.377 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:25.377 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:25.377 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:25.377 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:25.377 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:25.377 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:25.377 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:25.377 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:25.377 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:25.377 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:18:25.377 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:18:25.378 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:18:25.378 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:18:25.378 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:18:25.378 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:18:25.378 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:18:25.378 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:18:25.378 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:18:25.378 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:18:25.378 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:18:25.378 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:18:25.378 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:18:25.378 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:18:25.378 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:18:25.379 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:18:25.379 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:18:25.379 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:25.379 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:25.379 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:25.379 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:25.379 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:25.379 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:25.379 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:25.379 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:25.379 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:25.379 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:25.379 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:25.379 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:25.380 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:18:25.380 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:25.380 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:18:25.380 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.380 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:18:25.380 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:18:25.380 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:25.380 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:25.380 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.380 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:25.380 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:18:25.380 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:25.380 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:25.381 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.381 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:25.381 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:18:25.381 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:25.381 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:25.381 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:25.381 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:25.381 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:18:25.381 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:18:25.381 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:25.381 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:25.381 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:25.382 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:25.382 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.382 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:18:25.382 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:18:25.382 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:25.382 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:25.382 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.382 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:25.382 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:18:25.382 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:25.382 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:25.382 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.383 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:25.383 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:18:25.383 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:25.383 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:25.383 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:25.383 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:25.383 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:18:25.383 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:18:25.383 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:25.383 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:25.383 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:25.383 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:25.383 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.383 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:18:25.384 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:18:25.384 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:25.384 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:25.384 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.384 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:25.384 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:18:25.384 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:25.384 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:25.384 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.384 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:25.384 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:18:25.384 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:25.384 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:25.384 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:25.385 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:25.385 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:18:25.385 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:18:25.385 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:25.385 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:25.385 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:25.385 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:25.385 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:25.385 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:25.385 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:18:25.385 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:18:25.385 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.385 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:25.386 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:25.386 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:18:25.386 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:25.386 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:25.386 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": [ 2024-04-25T19:18:25.386 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.386 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:25.386 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:25.386 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 3 2024-04-25T19:18:25.386 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:25.386 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:25.386 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": [ 2024-04-25T19:18:25.387 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.387 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:25.387 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:25.387 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 5 2024-04-25T19:18:25.387 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:25.387 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:25.387 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:18:25.387 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:25.387 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:25.387 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:25.387 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 4 2024-04-25T19:18:25.387 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:25.387 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:25.387 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:25.387 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:18:25.388 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:18:25.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.373+0000 7f73cfc61640 1 -- 172.21.15.76:0/1822235242 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f73ac074500 msgr2=0x7f73ac076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:25.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.373+0000 7f73cfc61640 1 --2- 172.21.15.76:0/1822235242 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f73ac074500 0x7f73ac076a10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f73c0033000 tx=0x7f73c0009000 comp rx=0 tx=0).stop 2024-04-25T19:18:25.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.373+0000 7f73cfc61640 1 -- 172.21.15.76:0/1822235242 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73c80c3480 msgr2=0x7f73c814f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:25.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.373+0000 7f73cfc61640 1 --2- 172.21.15.76:0/1822235242 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f73c80c3480 0x7f73c814f9a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f73b8009e90 tx=0x7f73b800ec70 comp rx=0 tx=0).stop 2024-04-25T19:18:25.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.373+0000 7f73ce1fe640 1 -- 172.21.15.76:0/1822235242 reap_dead start 2024-04-25T19:18:25.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.373+0000 7f73ce1fe640 1 -- 172.21.15.76:0/1822235242 reap_dead start 2024-04-25T19:18:25.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.373+0000 7f73cfc61640 1 -- 172.21.15.76:0/1822235242 shutdown_connections 2024-04-25T19:18:25.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.373+0000 7f73cfc61640 1 -- 172.21.15.76:0/1822235242 >> 172.21.15.76:0/1822235242 conn(0x7f73c806c5c0 msgr2=0x7f73c8158130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:25.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.373+0000 7f73cfc61640 1 -- 172.21.15.76:0/1822235242 shutdown_connections 2024-04-25T19:18:25.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.373+0000 7f73cfc61640 1 -- 172.21.15.76:0/1822235242 wait complete. 2024-04-25T19:18:25.393 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T19:18:25.393 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-25T19:18:25.393 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-25T19:18:25.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.559+0000 7f173acd8640 1 Processor -- start 2024-04-25T19:18:25.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.559+0000 7f173acd8640 1 -- start start 2024-04-25T19:18:25.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.561+0000 7f173acd8640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f173415a9f0 0x7f17340c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:25.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.561+0000 7f173acd8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f173415df30 0x7f1734074130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:25.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.561+0000 7f173acd8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f173415ae00 con 0x7f1734154ee0 2024-04-25T19:18:25.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.561+0000 7f173acd8640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f1734068bf0 con 0x7f173415a9f0 2024-04-25T19:18:25.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.561+0000 7f173acd8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f17340c8e60 con 0x7f173415df30 2024-04-25T19:18:25.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.561+0000 7f1739275640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f173415df30 0x7f1734074130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:25.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.561+0000 7f1739275640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f173415df30 0x7f1734074130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:60352/0 (socket says 172.21.15.76:60352) 2024-04-25T19:18:25.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.561+0000 7f1739275640 1 -- 172.21.15.76:0/284072619 learned_addr learned my addr 172.21.15.76:0/284072619 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:25.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.561+0000 7f172bfff640 1 --2- 172.21.15.76:0/284072619 >> v2:172.21.15.105:3300/0 conn(0x7f173415a9f0 0x7f17340c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:25.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.561+0000 7f1739275640 1 -- 172.21.15.76:0/284072619 >> v2:172.21.15.105:3300/0 conn(0x7f173415a9f0 msgr2=0x7f17340c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:25.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.561+0000 7f1739275640 1 --2- 172.21.15.76:0/284072619 >> v2:172.21.15.105:3300/0 conn(0x7f173415a9f0 0x7f17340c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.561+0000 7f1739275640 1 -- 172.21.15.76:0/284072619 >> v1:172.21.15.105:6789/0 conn(0x7f1734154ee0 legacy=0x7f17340684e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:25.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.561+0000 7f1739275640 1 -- 172.21.15.76:0/284072619 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173415e6c0 con 0x7f173415df30 2024-04-25T19:18:25.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.561+0000 7f172bfff640 1 --2- 172.21.15.76:0/284072619 >> v2:172.21.15.105:3300/0 conn(0x7f173415a9f0 0x7f17340c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:18:25.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.563+0000 7f1739275640 1 --2- 172.21.15.76:0/284072619 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f173415df30 0x7f1734074130 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f172c009e50 tx=0x7f172c02a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=899c08631f676e50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:25.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.563+0000 7f172b7fe640 1 -- 172.21.15.76:0/284072619 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c036070 con 0x7f173415df30 2024-04-25T19:18:25.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.563+0000 7f172b7fe640 1 -- 172.21.15.76:0/284072619 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f172c031600 con 0x7f173415df30 2024-04-25T19:18:25.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.563+0000 7f172b7fe640 1 -- 172.21.15.76:0/284072619 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c0319d0 con 0x7f173415df30 2024-04-25T19:18:25.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.564+0000 7f173acd8640 1 -- 172.21.15.76:0/284072619 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f173415df30 msgr2=0x7f1734074130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:25.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.564+0000 7f173acd8640 1 --2- 172.21.15.76:0/284072619 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f173415df30 0x7f1734074130 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f172c009e50 tx=0x7f172c02a7a0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.564+0000 7f173acd8640 1 -- 172.21.15.76:0/284072619 shutdown_connections 2024-04-25T19:18:25.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.564+0000 7f173acd8640 1 --2- 172.21.15.76:0/284072619 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f173415df30 0x7f1734074130 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.564+0000 7f173acd8640 1 --2- 172.21.15.76:0/284072619 >> v2:172.21.15.105:3300/0 conn(0x7f173415a9f0 0x7f17340c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.564+0000 7f173acd8640 1 -- 172.21.15.76:0/284072619 >> 172.21.15.76:0/284072619 conn(0x7f173406c5c0 msgr2=0x7f173406c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:25.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.564+0000 7f173acd8640 1 -- 172.21.15.76:0/284072619 shutdown_connections 2024-04-25T19:18:25.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.564+0000 7f173acd8640 1 -- 172.21.15.76:0/284072619 wait complete. 2024-04-25T19:18:25.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.566+0000 7f173acd8640 1 Processor -- start 2024-04-25T19:18:25.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.566+0000 7f173acd8640 1 -- start start 2024-04-25T19:18:25.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.568+0000 7f173acd8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1734154ee0 0x7f173415c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:25.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.568+0000 7f173acd8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f173415a9f0 0x7f17341f11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:25.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.568+0000 7f173acd8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f173415df30 0x7f173414c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:25.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.568+0000 7f173acd8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f17340c90b0 con 0x7f173415a9f0 2024-04-25T19:18:25.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.568+0000 7f173acd8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f173415ec80 con 0x7f173415df30 2024-04-25T19:18:25.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.568+0000 7f173acd8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f17340c9390 con 0x7f1734154ee0 2024-04-25T19:18:25.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.568+0000 7f172bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f173415a9f0 0x7f17341f11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.568+0000 7f172bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f173415a9f0 0x7f17341f11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34398/0 (socket says 172.21.15.76:34398) 2024-04-25T19:18:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.568+0000 7f172bfff640 1 -- 172.21.15.76:0/2684845685 learned_addr learned my addr 172.21.15.76:0/2684845685 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.568+0000 7f1738a74640 1 --2- 172.21.15.76:0/2684845685 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1734154ee0 0x7f173415c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.568+0000 7f1739275640 1 --2- 172.21.15.76:0/2684845685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f173415df30 0x7f173414c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.569+0000 7f172bfff640 1 -- 172.21.15.76:0/2684845685 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1734154ee0 msgr2=0x7f173415c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.569+0000 7f172bfff640 1 --2- 172.21.15.76:0/2684845685 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1734154ee0 0x7f173415c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.569+0000 7f172bfff640 1 -- 172.21.15.76:0/2684845685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f173415df30 msgr2=0x7f173414c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.569+0000 7f172bfff640 1 --2- 172.21.15.76:0/2684845685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f173415df30 0x7f173414c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.569+0000 7f172bfff640 1 -- 172.21.15.76:0/2684845685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1734158650 con 0x7f173415a9f0 2024-04-25T19:18:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.569+0000 7f172bfff640 1 --2- 172.21.15.76:0/2684845685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f173415a9f0 0x7f17341f11a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f172400e990 tx=0x7f172400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.569+0000 7f17297fa640 1 -- 172.21.15.76:0/2684845685 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1724017070 con 0x7f173415a9f0 2024-04-25T19:18:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.569+0000 7f17297fa640 1 -- 172.21.15.76:0/2684845685 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f172400c9e0 con 0x7f173415a9f0 2024-04-25T19:18:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.569+0000 7f17297fa640 1 -- 172.21.15.76:0/2684845685 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172400cdb0 con 0x7f173415a9f0 2024-04-25T19:18:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.570+0000 7f173acd8640 1 -- 172.21.15.76:0/2684845685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17341f16e0 con 0x7f173415a9f0 2024-04-25T19:18:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.570+0000 7f173acd8640 1 -- 172.21.15.76:0/2684845685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f173415d670 con 0x7f173415a9f0 2024-04-25T19:18:25.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.572+0000 7f17297fa640 1 -- 172.21.15.76:0/2684845685 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f172401a070 con 0x7f173415a9f0 2024-04-25T19:18:25.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.572+0000 7f17297fa640 1 --2- 172.21.15.76:0/2684845685 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f17200788f0 0x7f172007ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:25.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.572+0000 7f173acd8640 1 -- 172.21.15.76:0/2684845685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f173406d6a0 con 0x7f173415a9f0 2024-04-25T19:18:25.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.573+0000 7f17297fa640 1 -- 172.21.15.76:0/2684845685 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f1724095070 con 0x7f173415a9f0 2024-04-25T19:18:25.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.573+0000 7f1738a74640 1 --2- 172.21.15.76:0/2684845685 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f17200788f0 0x7f172007ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:25.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.574+0000 7f1738a74640 1 --2- 172.21.15.76:0/2684845685 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f17200788f0 0x7f172007ae00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1730000f60 tx=0x7f1730009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:25.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.579+0000 7f17297fa640 1 -- 172.21.15.76:0/2684845685 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f172405f1c0 con 0x7f173415a9f0 2024-04-25T19:18:25.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.945+0000 7f173acd8640 1 -- 172.21.15.76:0/2684845685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f1734071a90 con 0x7f173415a9f0 2024-04-25T19:18:25.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.946+0000 7f17297fa640 1 -- 172.21.15.76:0/2684845685 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f17240634c0 con 0x7f173415a9f0 2024-04-25T19:18:25.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.949+0000 7f173acd8640 1 -- 172.21.15.76:0/2684845685 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f17200788f0 msgr2=0x7f172007ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:25.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.949+0000 7f173acd8640 1 --2- 172.21.15.76:0/2684845685 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f17200788f0 0x7f172007ae00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1730000f60 tx=0x7f1730009040 comp rx=0 tx=0).stop 2024-04-25T19:18:25.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.949+0000 7f173acd8640 1 -- 172.21.15.76:0/2684845685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f173415a9f0 msgr2=0x7f17341f11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:25.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.949+0000 7f173acd8640 1 --2- 172.21.15.76:0/2684845685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f173415a9f0 0x7f17341f11a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f172400e990 tx=0x7f172400ee50 comp rx=0 tx=0).stop 2024-04-25T19:18:25.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.949+0000 7f173acd8640 1 -- 172.21.15.76:0/2684845685 shutdown_connections 2024-04-25T19:18:25.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.949+0000 7f173acd8640 1 --2- 172.21.15.76:0/2684845685 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f17200788f0 0x7f172007ae00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.949+0000 7f173acd8640 1 --2- 172.21.15.76:0/2684845685 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f173415df30 0x7f173414c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.949+0000 7f173acd8640 1 --2- 172.21.15.76:0/2684845685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f173415a9f0 0x7f17341f11a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.949+0000 7f173acd8640 1 --2- 172.21.15.76:0/2684845685 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f1734154ee0 0x7f173415c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:25.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.949+0000 7f173acd8640 1 -- 172.21.15.76:0/2684845685 >> 172.21.15.76:0/2684845685 conn(0x7f173406c5c0 msgr2=0x7f1734073bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:25.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.949+0000 7f173acd8640 1 -- 172.21.15.76:0/2684845685 shutdown_connections 2024-04-25T19:18:25.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:25.949+0000 7f173acd8640 1 -- 172.21.15.76:0/2684845685 wait complete. 2024-04-25T19:18:25.971 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-04-25T19:18:26.379 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:18:26.382 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:18:35.972 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T19:18:35.973 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:18:36.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.123+0000 7f13985ce640 1 Processor -- start 2024-04-25T19:18:36.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.123+0000 7f13985ce640 1 -- start start 2024-04-25T19:18:36.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.124+0000 7f13985ce640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f13900d0810 0x7f139006f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:36.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.125+0000 7f13985ce640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f139015ce80 con 0x7f13900cea70 2024-04-25T19:18:36.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.125+0000 7f139636a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f13900d0810 0x7f139006f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:36.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.125+0000 7f13985ce640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f139015d160 con 0x7f139006e870 2024-04-25T19:18:36.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.125+0000 7f139636a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f13900d0810 0x7f139006f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37020/0 (socket says 172.21.15.76:37020) 2024-04-25T19:18:36.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.125+0000 7f13985ce640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f139015d440 con 0x7f13900d0810 2024-04-25T19:18:36.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.125+0000 7f139636a640 1 -- 172.21.15.76:0/1173149063 learned_addr learned my addr 172.21.15.76:0/1173149063 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:36.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.125+0000 7f139636a640 1 -- 172.21.15.76:0/1173149063 >> v1:172.21.15.105:6789/0 conn(0x7f139006e870 legacy=0x7f13901674f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T19:18:36.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.125+0000 7f139636a640 1 -- 172.21.15.76:0/1173149063 >> v1:172.21.15.76:6789/0 conn(0x7f13900cea70 legacy=0x7f139006e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:36.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.125+0000 7f139636a640 1 -- 172.21.15.76:0/1173149063 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f139015d720 con 0x7f13900d0810 2024-04-25T19:18:36.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.126+0000 7f139636a640 1 --2- 172.21.15.76:0/1173149063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13900d0810 0x7f139006f000 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f138c009870 tx=0x7f138c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781161b19a05d536 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:36.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.126+0000 7f1395368640 1 -- 172.21.15.76:0/1173149063 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c037070 con 0x7f13900d0810 2024-04-25T19:18:36.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.126+0000 7f1395368640 1 -- 172.21.15.76:0/1173149063 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f138c032440 con 0x7f13900d0810 2024-04-25T19:18:36.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.126+0000 7f1395368640 1 -- 172.21.15.76:0/1173149063 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c032810 con 0x7f13900d0810 2024-04-25T19:18:36.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.127+0000 7f13985ce640 1 -- 172.21.15.76:0/1173149063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13900d0810 msgr2=0x7f139006f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:36.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.127+0000 7f13985ce640 1 --2- 172.21.15.76:0/1173149063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13900d0810 0x7f139006f000 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f138c009870 tx=0x7f138c02a2a0 comp rx=0 tx=0).stop 2024-04-25T19:18:36.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.127+0000 7f13985ce640 1 -- 172.21.15.76:0/1173149063 shutdown_connections 2024-04-25T19:18:36.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.127+0000 7f13985ce640 1 --2- 172.21.15.76:0/1173149063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13900d0810 0x7f139006f000 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:36.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.127+0000 7f13985ce640 1 -- 172.21.15.76:0/1173149063 >> 172.21.15.76:0/1173149063 conn(0x7f13900b3c00 msgr2=0x7f13900b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:36.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.127+0000 7f13985ce640 1 -- 172.21.15.76:0/1173149063 shutdown_connections 2024-04-25T19:18:36.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.127+0000 7f13985ce640 1 -- 172.21.15.76:0/1173149063 wait complete. 2024-04-25T19:18:36.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.130+0000 7f13985ce640 1 Processor -- start 2024-04-25T19:18:36.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.130+0000 7f13985ce640 1 -- start start 2024-04-25T19:18:36.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.131+0000 7f13985ce640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f139006e870 0x7f13901f6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:36.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.131+0000 7f13985ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13900cea70 0x7f139014da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:36.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.131+0000 7f13985ce640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13900d0810 0x7f139015b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:36.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.131+0000 7f13985ce640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f139015cb70 con 0x7f13900d0810 2024-04-25T19:18:36.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.131+0000 7f13985ce640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f139006dc40 con 0x7f13900cea70 2024-04-25T19:18:36.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.131+0000 7f13985ce640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f139006f540 con 0x7f139006e870 2024-04-25T19:18:36.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.131+0000 7f1396b6b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13900d0810 0x7f139015b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:36.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.131+0000 7f1396b6b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13900d0810 0x7f139015b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37026/0 (socket says 172.21.15.76:37026) 2024-04-25T19:18:36.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.131+0000 7f1396b6b640 1 -- 172.21.15.76:0/3144131600 learned_addr learned my addr 172.21.15.76:0/3144131600 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:36.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.132+0000 7f1396b6b640 1 -- 172.21.15.76:0/3144131600 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f139006e870 msgr2=0x7f13901f6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:36.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.132+0000 7f1396b6b640 1 --2- 172.21.15.76:0/3144131600 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f139006e870 0x7f13901f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:36.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.132+0000 7f1396b6b640 1 -- 172.21.15.76:0/3144131600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13900cea70 msgr2=0x7f139014da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:36.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.132+0000 7f1395b69640 1 --2- 172.21.15.76:0/3144131600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13900cea70 0x7f139014da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:36.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.132+0000 7f1396b6b640 1 --2- 172.21.15.76:0/3144131600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13900cea70 0x7f139014da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:36.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.132+0000 7f1396b6b640 1 -- 172.21.15.76:0/3144131600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13901f6c70 con 0x7f13900d0810 2024-04-25T19:18:36.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.132+0000 7f1396b6b640 1 --2- 172.21.15.76:0/3144131600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13900d0810 0x7f139015b580 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f138800b9c0 tx=0x7f138800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:36.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.132+0000 7f1386ffd640 1 -- 172.21.15.76:0/3144131600 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1388019070 con 0x7f13900d0810 2024-04-25T19:18:36.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.132+0000 7f13985ce640 1 -- 172.21.15.76:0/3144131600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1390158210 con 0x7f13900d0810 2024-04-25T19:18:36.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.132+0000 7f1386ffd640 1 -- 172.21.15.76:0/3144131600 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f138800cba0 con 0x7f13900d0810 2024-04-25T19:18:36.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.132+0000 7f1386ffd640 1 -- 172.21.15.76:0/3144131600 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1388014440 con 0x7f13900d0810 2024-04-25T19:18:36.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.132+0000 7f13985ce640 1 -- 172.21.15.76:0/3144131600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1390157970 con 0x7f13900d0810 2024-04-25T19:18:36.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.134+0000 7f13985ce640 1 -- 172.21.15.76:0/3144131600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f139006a390 con 0x7f13900d0810 2024-04-25T19:18:36.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.135+0000 7f1386ffd640 1 -- 172.21.15.76:0/3144131600 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f138801c070 con 0x7f13900d0810 2024-04-25T19:18:36.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.135+0000 7f1386ffd640 1 --2- 172.21.15.76:0/3144131600 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f13740744b0 0x7f13740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:36.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.135+0000 7f1386ffd640 1 -- 172.21.15.76:0/3144131600 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f1388095030 con 0x7f13900d0810 2024-04-25T19:18:36.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.135+0000 7f139636a640 1 --2- 172.21.15.76:0/3144131600 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f13740744b0 0x7f13740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:36.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.136+0000 7f139636a640 1 --2- 172.21.15.76:0/3144131600 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f13740744b0 0x7f13740769c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f138c034000 tx=0x7f138c02c000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:36.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.141+0000 7f1386ffd640 1 -- 172.21.15.76:0/3144131600 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f138805f750 con 0x7f13900d0810 2024-04-25T19:18:36.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.522+0000 7f13985ce640 1 -- 172.21.15.76:0/3144131600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f13901f6390 con 0x7f13900d0810 2024-04-25T19:18:36.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.523+0000 7f1386ffd640 1 -- 172.21.15.76:0/3144131600 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7f138805f050 con 0x7f13900d0810 2024-04-25T19:18:36.526 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":24,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:18:36.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.527+0000 7f13985ce640 1 -- 172.21.15.76:0/3144131600 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f13740744b0 msgr2=0x7f13740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:36.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.527+0000 7f13985ce640 1 --2- 172.21.15.76:0/3144131600 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f13740744b0 0x7f13740769c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f138c034000 tx=0x7f138c02c000 comp rx=0 tx=0).stop 2024-04-25T19:18:36.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.527+0000 7f13985ce640 1 -- 172.21.15.76:0/3144131600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13900d0810 msgr2=0x7f139015b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:36.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.527+0000 7f13985ce640 1 --2- 172.21.15.76:0/3144131600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13900d0810 0x7f139015b580 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f138800b9c0 tx=0x7f138800be80 comp rx=0 tx=0).stop 2024-04-25T19:18:36.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.527+0000 7f13985ce640 1 -- 172.21.15.76:0/3144131600 shutdown_connections 2024-04-25T19:18:36.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.527+0000 7f13985ce640 1 --2- 172.21.15.76:0/3144131600 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f13740744b0 0x7f13740769c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:36.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.527+0000 7f13985ce640 1 --2- 172.21.15.76:0/3144131600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13900d0810 0x7f139015b580 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:36.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.527+0000 7f13985ce640 1 --2- 172.21.15.76:0/3144131600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13900cea70 0x7f139014da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:36.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.527+0000 7f13985ce640 1 --2- 172.21.15.76:0/3144131600 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f139006e870 0x7f13901f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:36.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.527+0000 7f13985ce640 1 -- 172.21.15.76:0/3144131600 >> 172.21.15.76:0/3144131600 conn(0x7f13900b3c00 msgr2=0x7f13900ca550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:36.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.527+0000 7f13985ce640 1 -- 172.21.15.76:0/3144131600 shutdown_connections 2024-04-25T19:18:36.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.527+0000 7f13985ce640 1 -- 172.21.15.76:0/3144131600 wait complete. 2024-04-25T19:18:36.546 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T19:18:36.546 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-04-25T19:18:36.546 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T19:18:36.546 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T19:18:36.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.714+0000 7fd3bf3a8640 1 Processor -- start 2024-04-25T19:18:36.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.714+0000 7fd3bf3a8640 1 -- start start 2024-04-25T19:18:36.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.716+0000 7fd3bf3a8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd3b80c60c0 0x7fd3b814ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:36.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.716+0000 7fd3bf3a8640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3b8159760 con 0x7fd3b815a350 2024-04-25T19:18:36.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.716+0000 7fd3bf3a8640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3b80c6ae0 con 0x7fd3b8159f40 2024-04-25T19:18:36.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.716+0000 7fd3bf3a8640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd3b80c83d0 con 0x7fd3b80c60c0 2024-04-25T19:18:36.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.716+0000 7fd3bd144640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd3b80c60c0 0x7fd3b814ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:36.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.716+0000 7fd3bd144640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd3b80c60c0 0x7fd3b814ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37032/0 (socket says 172.21.15.76:37032) 2024-04-25T19:18:36.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.716+0000 7fd3bd144640 1 -- 172.21.15.76:0/4163103501 learned_addr learned my addr 172.21.15.76:0/4163103501 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:36.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.717+0000 7fd3affff640 1 -- 172.21.15.76:0/4163103501 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4225247452 0 0) 0x7fd3b8159760 con 0x7fd3b815a350 2024-04-25T19:18:36.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.717+0000 7fd3bd144640 1 -- 172.21.15.76:0/4163103501 >> v1:172.21.15.105:6789/0 conn(0x7fd3b8159f40 legacy=0x7fd3b80c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:36.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.717+0000 7fd3bd144640 1 -- 172.21.15.76:0/4163103501 >> v1:172.21.15.76:6789/0 conn(0x7fd3b815a350 legacy=0x7fd3b81640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:36.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.717+0000 7fd3bd144640 1 -- 172.21.15.76:0/4163103501 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b80c6dc0 con 0x7fd3b80c60c0 2024-04-25T19:18:36.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.718+0000 7fd3bd144640 1 --2- 172.21.15.76:0/4163103501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3b80c60c0 0x7fd3b814ffc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800bb10 tx=0x7fd3a802c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1755b2ad9961ea6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:36.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.718+0000 7fd3affff640 1 -- 172.21.15.76:0/4163103501 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8036020 con 0x7fd3b80c60c0 2024-04-25T19:18:36.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.718+0000 7fd3affff640 1 -- 172.21.15.76:0/4163103501 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3a8032640 con 0x7fd3b80c60c0 2024-04-25T19:18:36.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.718+0000 7fd3affff640 1 -- 172.21.15.76:0/4163103501 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8032a10 con 0x7fd3b80c60c0 2024-04-25T19:18:36.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.719+0000 7fd3bf3a8640 1 -- 172.21.15.76:0/4163103501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3b80c60c0 msgr2=0x7fd3b814ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:36.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.719+0000 7fd3bf3a8640 1 --2- 172.21.15.76:0/4163103501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3b80c60c0 0x7fd3b814ffc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800bb10 tx=0x7fd3a802c940 comp rx=0 tx=0).stop 2024-04-25T19:18:36.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.719+0000 7fd3bf3a8640 1 -- 172.21.15.76:0/4163103501 shutdown_connections 2024-04-25T19:18:36.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.719+0000 7fd3bf3a8640 1 --2- 172.21.15.76:0/4163103501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3b80c60c0 0x7fd3b814ffc0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:36.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.719+0000 7fd3bf3a8640 1 -- 172.21.15.76:0/4163103501 >> 172.21.15.76:0/4163103501 conn(0x7fd3b8071a30 msgr2=0x7fd3b8071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:36.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.719+0000 7fd3bf3a8640 1 -- 172.21.15.76:0/4163103501 shutdown_connections 2024-04-25T19:18:36.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.719+0000 7fd3bf3a8640 1 -- 172.21.15.76:0/4163103501 wait complete. 2024-04-25T19:18:36.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.721+0000 7fd3bf3a8640 1 Processor -- start 2024-04-25T19:18:36.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.721+0000 7fd3bf3a8640 1 -- start start 2024-04-25T19:18:36.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.722+0000 7fd3bf3a8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3b80c60c0 0x7fd3b81ef4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:36.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.722+0000 7fd3bf3a8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3b8159f40 0x7fd3b8151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:36.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.722+0000 7fd3bf3a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3b815a350 0x7fd3b80bc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:36.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.722+0000 7fd3bf3a8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b815ac50 con 0x7fd3b8159f40 2024-04-25T19:18:36.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.722+0000 7fd3bf3a8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b8159a20 con 0x7fd3b815a350 2024-04-25T19:18:36.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.722+0000 7fd3bf3a8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fd3b80c6600 con 0x7fd3b80c60c0 2024-04-25T19:18:36.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.723+0000 7fd3bc943640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3b8159f40 0x7fd3b8151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:36.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.723+0000 7fd3bc943640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3b8159f40 0x7fd3b8151450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37046/0 (socket says 172.21.15.76:37046) 2024-04-25T19:18:36.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.723+0000 7fd3bc943640 1 -- 172.21.15.76:0/1917095793 learned_addr learned my addr 172.21.15.76:0/1917095793 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:36.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.723+0000 7fd3bd144640 1 --2- 172.21.15.76:0/1917095793 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3b80c60c0 0x7fd3b81ef4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:36.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.723+0000 7fd3bd945640 1 --2- 172.21.15.76:0/1917095793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3b815a350 0x7fd3b80bc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:36.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.723+0000 7fd3bc943640 1 -- 172.21.15.76:0/1917095793 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3b80c60c0 msgr2=0x7fd3b81ef4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:36.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.723+0000 7fd3bc943640 1 --2- 172.21.15.76:0/1917095793 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fd3b80c60c0 0x7fd3b81ef4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:36.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.723+0000 7fd3bc943640 1 -- 172.21.15.76:0/1917095793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3b815a350 msgr2=0x7fd3b80bc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:36.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.723+0000 7fd3bc943640 1 --2- 172.21.15.76:0/1917095793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd3b815a350 0x7fd3b80bc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:36.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.723+0000 7fd3bc943640 1 -- 172.21.15.76:0/1917095793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b80baf20 con 0x7fd3b8159f40 2024-04-25T19:18:36.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.723+0000 7fd3bc943640 1 --2- 172.21.15.76:0/1917095793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3b8159f40 0x7fd3b8151450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd3b400c3c0 tx=0x7fd3b400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:36.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.723+0000 7fd3adffb640 1 -- 172.21.15.76:0/1917095793 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4017070 con 0x7fd3b8159f40 2024-04-25T19:18:36.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.723+0000 7fd3adffb640 1 -- 172.21.15.76:0/1917095793 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b40052f0 con 0x7fd3b8159f40 2024-04-25T19:18:36.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.723+0000 7fd3adffb640 1 -- 172.21.15.76:0/1917095793 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4003830 con 0x7fd3b8159f40 2024-04-25T19:18:36.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.723+0000 7fd3bf3a8640 1 -- 172.21.15.76:0/1917095793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b80bc260 con 0x7fd3b8159f40 2024-04-25T19:18:36.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.724+0000 7fd3bf3a8640 1 -- 172.21.15.76:0/1917095793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b80bb4a0 con 0x7fd3b8159f40 2024-04-25T19:18:36.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.724+0000 7fd3bf3a8640 1 --2- 172.21.15.76:0/1917095793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3b80bceb0 0x7fd3b80bb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:36.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.724+0000 7fd3bf3a8640 1 -- 172.21.15.76:0/1917095793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b81efd80 con 0x7fd3b80bceb0 2024-04-25T19:18:36.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.725+0000 7fd3bd144640 1 --2- 172.21.15.76:0/1917095793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3b80bceb0 0x7fd3b80bb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:36.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.725+0000 7fd3adffb640 1 -- 172.21.15.76:0/1917095793 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd3b401a070 con 0x7fd3b8159f40 2024-04-25T19:18:36.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.725+0000 7fd3bd144640 1 -- 172.21.15.76:0/1917095793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd3b806a390 con 0x7fd3b80bceb0 2024-04-25T19:18:36.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.725+0000 7fd3bd144640 1 --2- 172.21.15.76:0/1917095793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3b80bceb0 0x7fd3b80bb6a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd3b80c69d0 tx=0x7fd3a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:36.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.725+0000 7fd3adffb640 1 --2- 172.21.15.76:0/1917095793 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fd39c0744b0 0x7fd39c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:36.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.726+0000 7fd3adffb640 1 -- 172.21.15.76:0/1917095793 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fd3b40943e0 con 0x7fd3b8159f40 2024-04-25T19:18:36.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.726+0000 7fd3bd945640 1 --2- 172.21.15.76:0/1917095793 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fd39c0744b0 0x7fd39c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:36.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.726+0000 7fd3adffb640 1 -- 172.21.15.76:0/1917095793 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8036040 con 0x7fd3b80bceb0 2024-04-25T19:18:36.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.726+0000 7fd3adffb640 1 -- 172.21.15.76:0/1917095793 <== mon.0 v2:172.21.15.76:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd3a8032640 con 0x7fd3b80bceb0 2024-04-25T19:18:36.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.726+0000 7fd3adffb640 1 -- 172.21.15.76:0/1917095793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3b80bceb0 msgr2=0x7fd3b80bb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:36.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.726+0000 7fd3adffb640 1 --2- 172.21.15.76:0/1917095793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3b80bceb0 0x7fd3b80bb6a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd3b80c69d0 tx=0x7fd3a80046c0 comp rx=0 tx=0).stop 2024-04-25T19:18:36.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.726+0000 7fd3adffb640 1 -- 172.21.15.76:0/1917095793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3b80bceb0 msgr2=0x7fd3b80bb6a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:18:36.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.726+0000 7fd3adffb640 1 --2- 172.21.15.76:0/1917095793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3b80bceb0 0x7fd3b80bb6a0 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd3b80c69d0 tx=0x7fd3a80046c0 comp rx=0 tx=0).stop 2024-04-25T19:18:36.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.726+0000 7fd3bd945640 1 --2- 172.21.15.76:0/1917095793 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fd39c0744b0 0x7fd39c0769c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd3b00012e0 tx=0x7fd3b0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:36.731 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:18:36.731 INFO:tasks.workunit.client.0.smithi076.stdout: Children see throughput for 1 initial writers = 702148.62 kB/sec 2024-04-25T19:18:36.731 INFO:tasks.workunit.client.0.smithi076.stdout: Parent sees throughput for 1 initial writers = 702138.01 kB/sec 2024-04-25T19:18:36.731 INFO:tasks.workunit.client.0.smithi076.stdout: Min throughput per process = 702148.62 kB/sec 2024-04-25T19:18:36.731 INFO:tasks.workunit.client.0.smithi076.stdout: Max throughput per process = 702148.62 kB/sec 2024-04-25T19:18:36.731 INFO:tasks.workunit.client.0.smithi076.stdout: Avg throughput per process = 702148.62 kB/sec 2024-04-25T19:18:36.731 INFO:tasks.workunit.client.0.smithi076.stdout: Min xfer = 10485760.00 kB 2024-04-25T19:18:36.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.732+0000 7fd3bf3a8640 1 --2- 172.21.15.76:0/1917095793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3b80bbbe0 0x7fd3b81f1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:36.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.732+0000 7fd3bf3a8640 1 -- 172.21.15.76:0/1917095793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b8074870 con 0x7fd3b80bbbe0 2024-04-25T19:18:36.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.732+0000 7fd3bd144640 1 --2- 172.21.15.76:0/1917095793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3b80bbbe0 0x7fd3b81f1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:36.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.733+0000 7fd3bd144640 1 -- 172.21.15.76:0/1917095793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd3b806dba0 con 0x7fd3b80bbbe0 2024-04-25T19:18:36.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.733+0000 7fd3bd144640 1 --2- 172.21.15.76:0/1917095793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3b80bbbe0 0x7fd3b81f1590 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd3a802c980 tx=0x7fd3a8033000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:36.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.733+0000 7fd3adffb640 1 -- 172.21.15.76:0/1917095793 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8051070 con 0x7fd3b80bbbe0 2024-04-25T19:18:36.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.733+0000 7fd3adffb640 1 -- 172.21.15.76:0/1917095793 <== mon.0 v2:172.21.15.76:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4557 (secure 0 0 0) 0x7fd3a804b670 con 0x7fd3b80bbbe0 2024-04-25T19:18:36.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.733+0000 7fd3adffb640 1 -- 172.21.15.76:0/1917095793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3b80bbbe0 msgr2=0x7fd3b81f1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:36.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.733+0000 7fd3adffb640 1 --2- 172.21.15.76:0/1917095793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3b80bbbe0 0x7fd3b81f1590 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd3a802c980 tx=0x7fd3a8033000 comp rx=0 tx=0).stop 2024-04-25T19:18:36.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.733+0000 7fd3adffb640 1 -- 172.21.15.76:0/1917095793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3b80bbbe0 msgr2=0x7fd3b81f1590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:18:36.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.733+0000 7fd3adffb640 1 --2- 172.21.15.76:0/1917095793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3b80bbbe0 0x7fd3b81f1590 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:36.736 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:18:36.736 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:18:36.736 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:18:36.736 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "leader", 2024-04-25T19:18:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 24, 2024-04-25T19:18:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 15845, 2024-04-25T19:18:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:18:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:18:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-04-25T19:18:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:18:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 14, 2024-04-25T19:18:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:18:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:18:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:18:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:18:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:18:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:18:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:18:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:18:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:18:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:18:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:18:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:18:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:18:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:18:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:18:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:18:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:18:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:18:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:18:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:18:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:18:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:18:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:18:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:18:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:18:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:18:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:18:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:18:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:18:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:18:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:18:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:18:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:18:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:18:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:18:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:18:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:18:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:18:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:18:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:18:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:36.746 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:18:36.746 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:18:36.746 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:36.746 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:36.746 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:36.746 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:36.746 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:36.746 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:36.746 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:18:36.746 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:18:36.746 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:36.746 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:36.746 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:36.747 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:18:36.747 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:36.747 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:36.747 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:18:36.747 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:36.747 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:36.747 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:36.747 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 2 2024-04-25T19:18:36.747 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:36.747 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:36.747 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:36.747 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:18:36.747 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:18:36.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.734+0000 7fd3bf3a8640 1 -- 172.21.15.76:0/1917095793 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fd39c0744b0 msgr2=0x7fd39c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:36.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.734+0000 7fd3bf3a8640 1 --2- 172.21.15.76:0/1917095793 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fd39c0744b0 0x7fd39c0769c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd3b00012e0 tx=0x7fd3b0009040 comp rx=0 tx=0).stop 2024-04-25T19:18:36.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.734+0000 7fd3bf3a8640 1 -- 172.21.15.76:0/1917095793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3b8159f40 msgr2=0x7fd3b8151450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:36.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.734+0000 7fd3bf3a8640 1 --2- 172.21.15.76:0/1917095793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3b8159f40 0x7fd3b8151450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd3b400c3c0 tx=0x7fd3b400c880 comp rx=0 tx=0).stop 2024-04-25T19:18:36.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.734+0000 7fd3bd945640 1 -- 172.21.15.76:0/1917095793 reap_dead start 2024-04-25T19:18:36.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.734+0000 7fd3bd945640 1 -- 172.21.15.76:0/1917095793 reap_dead start 2024-04-25T19:18:36.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.734+0000 7fd3bf3a8640 1 -- 172.21.15.76:0/1917095793 shutdown_connections 2024-04-25T19:18:36.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.734+0000 7fd3bf3a8640 1 -- 172.21.15.76:0/1917095793 >> 172.21.15.76:0/1917095793 conn(0x7fd3b8071a30 msgr2=0x7fd3b806d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:36.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.734+0000 7fd3bf3a8640 1 -- 172.21.15.76:0/1917095793 shutdown_connections 2024-04-25T19:18:36.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.734+0000 7fd3bf3a8640 1 -- 172.21.15.76:0/1917095793 wait complete. 2024-04-25T19:18:36.750 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T19:18:36.750 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T19:18:36.750 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T19:18:36.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.857+0000 7f903ae51640 1 Processor -- start 2024-04-25T19:18:36.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.857+0000 7f903ae51640 1 -- start start 2024-04-25T19:18:36.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.858+0000 7f903ae51640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f90340d0810 0x7f903406f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:36.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.858+0000 7f903ae51640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f903406e870 0x7f90340cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:36.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.858+0000 7f903ae51640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f903415ce80 con 0x7f90340cea70 2024-04-25T19:18:36.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.858+0000 7f903ae51640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f903415d160 con 0x7f90340d0810 2024-04-25T19:18:36.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.858+0000 7f903ae51640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f903415d440 con 0x7f903406e870 2024-04-25T19:18:36.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.859+0000 7f9038bed640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f90340d0810 0x7f903406f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:36.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.859+0000 7f90393ee640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f903406e870 0x7f90340cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:36.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.859+0000 7f9038bed640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f90340d0810 0x7f903406f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:41298/0 (socket says 172.21.15.76:41298) 2024-04-25T19:18:36.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.859+0000 7f90393ee640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f903406e870 0x7f90340cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:48172/0 (socket says 172.21.15.76:48172) 2024-04-25T19:18:36.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.859+0000 7f9038bed640 1 -- 172.21.15.76:0/3299008787 learned_addr learned my addr 172.21.15.76:0/3299008787 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:36.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.859+0000 7f90393ee640 1 -- 172.21.15.76:0/3299008787 >> v2:172.21.15.105:3300/0 conn(0x7f90340d0810 msgr2=0x7f903406f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:36.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.859+0000 7f90393ee640 1 --2- 172.21.15.76:0/3299008787 >> v2:172.21.15.105:3300/0 conn(0x7f90340d0810 0x7f903406f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:36.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.859+0000 7f90393ee640 1 -- 172.21.15.76:0/3299008787 >> v1:172.21.15.105:6789/0 conn(0x7f90340cea70 legacy=0x7f903406e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:36.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.859+0000 7f90393ee640 1 -- 172.21.15.76:0/3299008787 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903415d720 con 0x7f903406e870 2024-04-25T19:18:36.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.859+0000 7f9038bed640 1 --2- 172.21.15.76:0/3299008787 >> v2:172.21.15.105:3300/0 conn(0x7f90340d0810 0x7f903406f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:18:36.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.860+0000 7f90393ee640 1 --2- 172.21.15.76:0/3299008787 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f903406e870 0x7f90340cee80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f902c009d90 tx=0x7f902c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0bb10f5afc33986 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:36.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.860+0000 7f90337fe640 1 -- 172.21.15.76:0/3299008787 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c007740 con 0x7f903406e870 2024-04-25T19:18:36.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.860+0000 7f90337fe640 1 -- 172.21.15.76:0/3299008787 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f902c0078e0 con 0x7f903406e870 2024-04-25T19:18:36.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.860+0000 7f90337fe640 1 -- 172.21.15.76:0/3299008787 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c007cb0 con 0x7f903406e870 2024-04-25T19:18:36.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.861+0000 7f903ae51640 1 -- 172.21.15.76:0/3299008787 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f903406e870 msgr2=0x7f90340cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:36.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.861+0000 7f903ae51640 1 --2- 172.21.15.76:0/3299008787 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f903406e870 0x7f90340cee80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f902c009d90 tx=0x7f902c02a660 comp rx=0 tx=0).stop 2024-04-25T19:18:36.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.861+0000 7f903ae51640 1 -- 172.21.15.76:0/3299008787 shutdown_connections 2024-04-25T19:18:36.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.861+0000 7f903ae51640 1 --2- 172.21.15.76:0/3299008787 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f903406e870 0x7f90340cee80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:36.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.861+0000 7f903ae51640 1 --2- 172.21.15.76:0/3299008787 >> v2:172.21.15.105:3300/0 conn(0x7f90340d0810 0x7f903406f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:36.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.861+0000 7f903ae51640 1 -- 172.21.15.76:0/3299008787 >> 172.21.15.76:0/3299008787 conn(0x7f90340b3c00 msgr2=0x7f90340b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:36.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.861+0000 7f903ae51640 1 -- 172.21.15.76:0/3299008787 shutdown_connections 2024-04-25T19:18:36.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.861+0000 7f903ae51640 1 -- 172.21.15.76:0/3299008787 wait complete. 2024-04-25T19:18:36.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.862+0000 7f903ae51640 1 Processor -- start 2024-04-25T19:18:36.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.863+0000 7f903ae51640 1 -- start start 2024-04-25T19:18:36.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.864+0000 7f903ae51640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f903406e870 0x7f90341f6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:36.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.864+0000 7f903ae51640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f90340cea70 0x7f9034157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:36.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.864+0000 7f903ae51640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90340d0810 0x7f90341506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:36.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.864+0000 7f903ae51640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f903415cb70 con 0x7f903406e870 2024-04-25T19:18:36.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.864+0000 7f903ae51640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f903406dc40 con 0x7f90340d0810 2024-04-25T19:18:36.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.864+0000 7f903ae51640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f903406f540 con 0x7f90340cea70 2024-04-25T19:18:36.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.864+0000 7f9038bed640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f903406e870 0x7f90341f6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:36.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.864+0000 7f9038bed640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f903406e870 0x7f90341f6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37074/0 (socket says 172.21.15.76:37074) 2024-04-25T19:18:36.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.864+0000 7f9038bed640 1 -- 172.21.15.76:0/292365929 learned_addr learned my addr 172.21.15.76:0/292365929 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:36.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.864+0000 7f9033fff640 1 --2- 172.21.15.76:0/292365929 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f90340cea70 0x7f9034157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:36.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.864+0000 7f90393ee640 1 --2- 172.21.15.76:0/292365929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90340d0810 0x7f90341506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:36.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.864+0000 7f9038bed640 1 -- 172.21.15.76:0/292365929 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f90340cea70 msgr2=0x7f9034157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:36.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.864+0000 7f9038bed640 1 --2- 172.21.15.76:0/292365929 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f90340cea70 0x7f9034157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:36.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.864+0000 7f9038bed640 1 -- 172.21.15.76:0/292365929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90340d0810 msgr2=0x7f90341506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:36.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.864+0000 7f9038bed640 1 --2- 172.21.15.76:0/292365929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90340d0810 0x7f90341506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:36.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.864+0000 7f9038bed640 1 -- 172.21.15.76:0/292365929 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90341565f0 con 0x7f903406e870 2024-04-25T19:18:36.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.865+0000 7f9038bed640 1 --2- 172.21.15.76:0/292365929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f903406e870 0x7f90341f6280 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f902800ea10 tx=0x7f902800eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:36.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.865+0000 7f90317fa640 1 -- 172.21.15.76:0/292365929 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9028017070 con 0x7f903406e870 2024-04-25T19:18:36.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.865+0000 7f90317fa640 1 -- 172.21.15.76:0/292365929 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f902800ca00 con 0x7f903406e870 2024-04-25T19:18:36.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.865+0000 7f903ae51640 1 -- 172.21.15.76:0/292365929 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9034151720 con 0x7f903406e870 2024-04-25T19:18:36.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.865+0000 7f90317fa640 1 -- 172.21.15.76:0/292365929 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902800cdb0 con 0x7f903406e870 2024-04-25T19:18:36.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.865+0000 7f903ae51640 1 -- 172.21.15.76:0/292365929 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9034151ce0 con 0x7f903406e870 2024-04-25T19:18:36.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.866+0000 7f903ae51640 1 --2- 172.21.15.76:0/292365929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f903415a240 0x7f9034150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:36.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.866+0000 7f903ae51640 1 -- 172.21.15.76:0/292365929 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9034158700 con 0x7f903415a240 2024-04-25T19:18:36.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.866+0000 7f9033fff640 1 --2- 172.21.15.76:0/292365929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f903415a240 0x7f9034150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:36.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.866+0000 7f9033fff640 1 -- 172.21.15.76:0/292365929 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f903406a390 con 0x7f903415a240 2024-04-25T19:18:36.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.866+0000 7f9033fff640 1 --2- 172.21.15.76:0/292365929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f903415a240 0x7f9034150be0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f90341f0940 tx=0x7f902000f900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:36.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.866+0000 7f90317fa640 1 -- 172.21.15.76:0/292365929 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f902801a070 con 0x7f903406e870 2024-04-25T19:18:36.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.867+0000 7f90317fa640 1 --2- 172.21.15.76:0/292365929 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f90180744e0 0x7f90180769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:36.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.867+0000 7f90317fa640 1 -- 172.21.15.76:0/292365929 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f9028095310 con 0x7f903406e870 2024-04-25T19:18:36.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.867+0000 7f90393ee640 1 --2- 172.21.15.76:0/292365929 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f90180744e0 0x7f90180769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:36.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.867+0000 7f90317fa640 1 -- 172.21.15.76:0/292365929 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020019070 con 0x7f903415a240 2024-04-25T19:18:36.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.867+0000 7f90317fa640 1 -- 172.21.15.76:0/292365929 <== mon.1 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9020005320 con 0x7f903415a240 2024-04-25T19:18:36.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.867+0000 7f90317fa640 1 -- 172.21.15.76:0/292365929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f903415a240 msgr2=0x7f9034150be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:36.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.867+0000 7f90317fa640 1 --2- 172.21.15.76:0/292365929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f903415a240 0x7f9034150be0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f90341f0940 tx=0x7f902000f900 comp rx=0 tx=0).stop 2024-04-25T19:18:36.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.867+0000 7f90317fa640 1 -- 172.21.15.76:0/292365929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f903415a240 msgr2=0x7f9034150be0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:18:36.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.867+0000 7f90317fa640 1 --2- 172.21.15.76:0/292365929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f903415a240 0x7f9034150be0 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f90341f0940 tx=0x7f902000f900 comp rx=0 tx=0).stop 2024-04-25T19:18:36.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.867+0000 7f90393ee640 1 --2- 172.21.15.76:0/292365929 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f90180744e0 0x7f90180769f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f902c034000 tx=0x7f902c02f000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:36.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.873+0000 7f903ae51640 1 --2- 172.21.15.76:0/292365929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90341f67c0 0x7f903415b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:36.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.873+0000 7f903ae51640 1 -- 172.21.15.76:0/292365929 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f90341563a0 con 0x7f90341f67c0 2024-04-25T19:18:36.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.874+0000 7f9033fff640 1 --2- 172.21.15.76:0/292365929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90341f67c0 0x7f903415b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:36.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.874+0000 7f9033fff640 1 -- 172.21.15.76:0/292365929 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f90341f6bd0 con 0x7f90341f67c0 2024-04-25T19:18:36.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.874+0000 7f9033fff640 1 --2- 172.21.15.76:0/292365929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90341f67c0 0x7f903415b970 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9020016000 tx=0x7f9020000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:36.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.874+0000 7f90317fa640 1 -- 172.21.15.76:0/292365929 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020019030 con 0x7f90341f67c0 2024-04-25T19:18:36.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.875+0000 7f90317fa640 1 -- 172.21.15.76:0/292365929 <== mon.1 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5065 (secure 0 0 0) 0x7f9020006b00 con 0x7f90341f67c0 2024-04-25T19:18:36.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.875+0000 7f90317fa640 1 -- 172.21.15.76:0/292365929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90341f67c0 msgr2=0x7f903415b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:36.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.875+0000 7f90317fa640 1 --2- 172.21.15.76:0/292365929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90341f67c0 0x7f903415b970 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9020016000 tx=0x7f9020000ed0 comp rx=0 tx=0).stop 2024-04-25T19:18:36.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.875+0000 7f90317fa640 1 -- 172.21.15.76:0/292365929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90341f67c0 msgr2=0x7f903415b970 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:18:36.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.875+0000 7f90317fa640 1 --2- 172.21.15.76:0/292365929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f90341f67c0 0x7f903415b970 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:36.878 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:18:36.878 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:18:36.878 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:18:36.878 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "peon", 2024-04-25T19:18:36.878 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 24, 2024-04-25T19:18:36.878 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 68367, 2024-04-25T19:18:36.878 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:18:36.878 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:18:36.878 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-04-25T19:18:36.878 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:18:36.878 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:36.879 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 14, 2024-04-25T19:18:36.879 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:18:36.879 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:18:36.879 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:18:36.879 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:36.879 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:36.879 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:36.879 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:36.879 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:36.879 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:36.879 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:36.879 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:36.879 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:36.879 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:36.880 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:36.880 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:36.880 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:18:36.880 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:18:36.880 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:36.880 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:36.880 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:36.880 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:36.880 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:36.880 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:36.880 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:36.880 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:36.880 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:36.880 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:36.880 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:36.880 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:36.881 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:36.881 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:18:36.881 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:18:36.881 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:18:36.881 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:18:36.881 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:18:36.881 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:18:36.881 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:18:36.881 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:18:36.881 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:18:36.881 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:18:36.881 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:18:36.881 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:18:36.881 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:18:36.881 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:18:36.882 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:18:36.882 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:18:36.882 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:18:36.882 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:36.882 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:36.882 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:36.882 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:36.882 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:36.882 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:36.882 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:36.882 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:36.882 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:36.882 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:36.882 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:36.882 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:36.883 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:18:36.883 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:36.883 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:18:36.883 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:36.883 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:18:36.883 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:18:36.883 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:36.883 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:36.883 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:36.883 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:36.883 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:18:36.883 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:36.883 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:36.883 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:36.883 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:36.884 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:18:36.884 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:36.884 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:36.884 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:36.884 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:36.884 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:18:36.884 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:18:36.884 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:36.884 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:36.884 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:36.884 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:36.884 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:36.884 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:18:36.884 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:18:36.884 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:36.884 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:36.885 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:36.885 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:36.885 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:18:36.885 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:36.885 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:36.885 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:36.885 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:36.885 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:18:36.885 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:36.885 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:36.885 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:36.885 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:36.885 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:18:36.885 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:18:36.885 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:36.886 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:36.886 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:36.886 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:36.886 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:36.886 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:18:36.886 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:18:36.886 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:36.886 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:36.886 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:36.886 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:36.886 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:18:36.886 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:36.886 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:36.886 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:36.886 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:36.887 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:18:36.887 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:36.887 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:36.887 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:36.887 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:36.887 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:18:36.887 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:18:36.887 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:36.887 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:36.887 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:36.887 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:36.887 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:36.887 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:36.887 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:18:36.888 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:18:36.888 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:36.888 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:36.888 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:36.888 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:18:36.888 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:36.888 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:36.888 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": [ 2024-04-25T19:18:36.888 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:36.888 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:36.888 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:36.888 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 3 2024-04-25T19:18:36.888 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:36.888 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:36.888 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": [ 2024-04-25T19:18:36.888 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:36.889 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:36.889 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:36.889 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 3 2024-04-25T19:18:36.889 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:36.889 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:36.889 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:18:36.889 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:36.889 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:36.889 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:36.889 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 3 2024-04-25T19:18:36.889 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:36.889 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:36.889 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": [ 2024-04-25T19:18:36.889 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:36.889 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:36.890 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:36.890 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 2 2024-04-25T19:18:36.890 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:36.890 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:36.890 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:36.890 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:18:36.890 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:18:36.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.875+0000 7f903ae51640 1 -- 172.21.15.76:0/292365929 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f90180744e0 msgr2=0x7f90180769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:36.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.875+0000 7f903ae51640 1 --2- 172.21.15.76:0/292365929 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f90180744e0 0x7f90180769f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f902c034000 tx=0x7f902c02f000 comp rx=0 tx=0).stop 2024-04-25T19:18:36.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.875+0000 7f903ae51640 1 -- 172.21.15.76:0/292365929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f903406e870 msgr2=0x7f90341f6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:36.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.875+0000 7f903ae51640 1 --2- 172.21.15.76:0/292365929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f903406e870 0x7f90341f6280 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f902800ea10 tx=0x7f902800eed0 comp rx=0 tx=0).stop 2024-04-25T19:18:36.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.875+0000 7f90393ee640 1 -- 172.21.15.76:0/292365929 reap_dead start 2024-04-25T19:18:36.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.875+0000 7f903ae51640 1 -- 172.21.15.76:0/292365929 shutdown_connections 2024-04-25T19:18:36.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.875+0000 7f903ae51640 1 --2- 172.21.15.76:0/292365929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f903406e870 0x7f90341f6280 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:36.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.875+0000 7f903ae51640 1 -- 172.21.15.76:0/292365929 >> 172.21.15.76:0/292365929 conn(0x7f90340b3c00 msgr2=0x7f90340ca380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:36.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.875+0000 7f903ae51640 1 -- 172.21.15.76:0/292365929 shutdown_connections 2024-04-25T19:18:36.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.875+0000 7f903ae51640 1 -- 172.21.15.76:0/292365929 wait complete. 2024-04-25T19:18:36.892 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T19:18:36.892 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T19:18:36.892 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T19:18:37.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.999+0000 7f2be2213640 1 Processor -- start 2024-04-25T19:18:37.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:36.999+0000 7f2be2213640 1 -- start start 2024-04-25T19:18:37.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.000+0000 7f2be2213640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f2bdc15cd80 0x7f2bdc15d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.000+0000 7f2be2213640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bdc0c8020 con 0x7f2bdc0c7750 2024-04-25T19:18:37.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.000+0000 7f2be2213640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bdc1598a0 con 0x7f2bdc0c7b60 2024-04-25T19:18:37.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.000+0000 7f2be2213640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f2bdc159b40 con 0x7f2bdc15cd80 2024-04-25T19:18:37.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.001+0000 7f2bdb7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2bdc0c7750 0x7f2bdc153590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57940/0 (socket says 172.21.15.76:57940) 2024-04-25T19:18:37.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.001+0000 7f2bdb7fe640 1 -- 172.21.15.76:0/1507520678 learned_addr learned my addr 172.21.15.76:0/1507520678 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:18:37.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.001+0000 7f2bdbfff640 1 --2- 172.21.15.76:0/1507520678 >> v2:172.21.15.105:3300/0 conn(0x7f2bdc15cd80 0x7f2bdc15d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.001+0000 7f2bda7fc640 1 -- 172.21.15.76:0/1507520678 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 833152781 0 0) 0x7f2bdc0c8020 con 0x7f2bdc0c7750 2024-04-25T19:18:37.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.001+0000 7f2bda7fc640 1 -- 172.21.15.76:0/1507520678 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bc4003520 con 0x7f2bdc0c7750 2024-04-25T19:18:37.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.001+0000 7f2bdbfff640 1 -- 172.21.15.76:0/1507520678 >> v1:172.21.15.105:6789/0 conn(0x7f2bdc0c7b60 legacy=0x7f2bdc0cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:37.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.001+0000 7f2bdbfff640 1 -- 172.21.15.76:0/1507520678 >> v1:172.21.15.76:6789/0 conn(0x7f2bdc0c7750 legacy=0x7f2bdc153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:37.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.001+0000 7f2bdbfff640 1 -- 172.21.15.76:0/1507520678 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bdc159e20 con 0x7f2bdc15cd80 2024-04-25T19:18:37.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.002+0000 7f2bdbfff640 1 --2- 172.21.15.76:0/1507520678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2bdc15cd80 0x7f2bdc15d190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009d90 tx=0x7f2bd402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26ec24dc97202876 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:37.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.002+0000 7f2bda7fc640 1 -- 172.21.15.76:0/1507520678 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4007740 con 0x7f2bdc15cd80 2024-04-25T19:18:37.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.002+0000 7f2bda7fc640 1 -- 172.21.15.76:0/1507520678 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd40078e0 con 0x7f2bdc15cd80 2024-04-25T19:18:37.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.002+0000 7f2bda7fc640 1 -- 172.21.15.76:0/1507520678 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4007cb0 con 0x7f2bdc15cd80 2024-04-25T19:18:37.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.003+0000 7f2be2213640 1 -- 172.21.15.76:0/1507520678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2bdc15cd80 msgr2=0x7f2bdc15d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:37.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.003+0000 7f2be2213640 1 --2- 172.21.15.76:0/1507520678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2bdc15cd80 0x7f2bdc15d190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009d90 tx=0x7f2bd402a6f0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.003+0000 7f2be2213640 1 -- 172.21.15.76:0/1507520678 shutdown_connections 2024-04-25T19:18:37.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.003+0000 7f2be2213640 1 --2- 172.21.15.76:0/1507520678 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2bdc15cd80 0x7f2bdc15d190 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.003+0000 7f2be2213640 1 -- 172.21.15.76:0/1507520678 >> 172.21.15.76:0/1507520678 conn(0x7f2bdc070800 msgr2=0x7f2bdc070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:37.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.003+0000 7f2be2213640 1 -- 172.21.15.76:0/1507520678 shutdown_connections 2024-04-25T19:18:37.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.003+0000 7f2be2213640 1 -- 172.21.15.76:0/1507520678 wait complete. 2024-04-25T19:18:37.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.004+0000 7f2be2213640 1 Processor -- start 2024-04-25T19:18:37.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.005+0000 7f2be2213640 1 -- start start 2024-04-25T19:18:37.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.006+0000 7f2be2213640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2bdc0c7750 0x7f2bdc155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.006+0000 7f2be2213640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2bdc15cd80 0x7f2bdc156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.006+0000 7f2be2213640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bdc14b8b0 0x7f2bdc150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.006+0000 7f2be2213640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc15d6d0 con 0x7f2bdc14b8b0 2024-04-25T19:18:37.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.006+0000 7f2be2213640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc15a3e0 con 0x7f2bdc0c7750 2024-04-25T19:18:37.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.006+0000 7f2be2213640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f2bdc0c8520 con 0x7f2bdc15cd80 2024-04-25T19:18:37.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.006+0000 7f2bdbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bdc14b8b0 0x7f2bdc150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.006+0000 7f2bdbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bdc14b8b0 0x7f2bdc150e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37088/0 (socket says 172.21.15.76:37088) 2024-04-25T19:18:37.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.006+0000 7f2bdbfff640 1 -- 172.21.15.76:0/701427615 learned_addr learned my addr 172.21.15.76:0/701427615 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:37.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.006+0000 7f2bdaffd640 1 --2- 172.21.15.76:0/701427615 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2bdc15cd80 0x7f2bdc156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.006+0000 7f2bdb7fe640 1 --2- 172.21.15.76:0/701427615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2bdc0c7750 0x7f2bdc155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.006+0000 7f2bdbfff640 1 -- 172.21.15.76:0/701427615 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2bdc15cd80 msgr2=0x7f2bdc156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:37.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.006+0000 7f2bdbfff640 1 --2- 172.21.15.76:0/701427615 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2bdc15cd80 0x7f2bdc156960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.006+0000 7f2bdbfff640 1 -- 172.21.15.76:0/701427615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2bdc0c7750 msgr2=0x7f2bdc155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:37.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.006+0000 7f2bdbfff640 1 --2- 172.21.15.76:0/701427615 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2bdc0c7750 0x7f2bdc155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.006+0000 7f2bdbfff640 1 -- 172.21.15.76:0/701427615 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bdc1558c0 con 0x7f2bdc14b8b0 2024-04-25T19:18:37.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.006+0000 7f2bdbfff640 1 --2- 172.21.15.76:0/701427615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bdc14b8b0 0x7f2bdc150e00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009eb0 tx=0x7f2bd4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:37.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.006+0000 7f2bc3fff640 1 -- 172.21.15.76:0/701427615 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd403f070 con 0x7f2bdc14b8b0 2024-04-25T19:18:37.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.007+0000 7f2bc3fff640 1 -- 172.21.15.76:0/701427615 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd4031bc0 con 0x7f2bdc14b8b0 2024-04-25T19:18:37.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.007+0000 7f2be2213640 1 -- 172.21.15.76:0/701427615 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bdc1542c0 con 0x7f2bdc14b8b0 2024-04-25T19:18:37.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.007+0000 7f2bc3fff640 1 -- 172.21.15.76:0/701427615 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4009450 con 0x7f2bdc14b8b0 2024-04-25T19:18:37.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.007+0000 7f2be2213640 1 -- 172.21.15.76:0/701427615 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bdc154880 con 0x7f2bdc14b8b0 2024-04-25T19:18:37.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.008+0000 7f2be2213640 1 --2- 172.21.15.76:0/701427615 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2bdc14bcc0 0x7f2bdc151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.008+0000 7f2be2213640 1 -- 172.21.15.76:0/701427615 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f2bdc06a390 con 0x7f2bdc14bcc0 2024-04-25T19:18:37.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.008+0000 7f2bdb7fe640 1 --2- 172.21.15.76:0/701427615 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2bdc14bcc0 0x7f2bdc151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.008+0000 7f2bc3fff640 1 -- 172.21.15.76:0/701427615 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2bd403b410 con 0x7f2bdc14b8b0 2024-04-25T19:18:37.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.008+0000 7f2bdb7fe640 1 -- 172.21.15.76:0/701427615 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2bdc155030 con 0x7f2bdc14bcc0 2024-04-25T19:18:37.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.008+0000 7f2bdb7fe640 1 --2- 172.21.15.76:0/701427615 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2bdc14bcc0 0x7f2bdc151340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0011e0 tx=0x7f2bcc00e8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:37.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.008+0000 7f2bc3fff640 1 --2- 172.21.15.76:0/701427615 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f2bc80744b0 0x7f2bc80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.008+0000 7f2bc3fff640 1 -- 172.21.15.76:0/701427615 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f2bd40b5d50 con 0x7f2bdc14b8b0 2024-04-25T19:18:37.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.008+0000 7f2bdaffd640 1 --2- 172.21.15.76:0/701427615 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f2bc80744b0 0x7f2bc80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.009+0000 7f2bc3fff640 1 -- 172.21.15.76:0/701427615 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc019070 con 0x7f2bdc14bcc0 2024-04-25T19:18:37.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.009+0000 7f2bc3fff640 1 -- 172.21.15.76:0/701427615 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2bcc005ac0 con 0x7f2bdc14bcc0 2024-04-25T19:18:37.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.009+0000 7f2bc3fff640 1 -- 172.21.15.76:0/701427615 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2bdc14bcc0 msgr2=0x7f2bdc151340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:37.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.009+0000 7f2bc3fff640 1 --2- 172.21.15.76:0/701427615 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2bdc14bcc0 0x7f2bdc151340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0011e0 tx=0x7f2bcc00e8f0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.009+0000 7f2bc3fff640 1 -- 172.21.15.76:0/701427615 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2bdc14bcc0 msgr2=0x7f2bdc151340 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:18:37.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.009+0000 7f2bc3fff640 1 --2- 172.21.15.76:0/701427615 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2bdc14bcc0 0x7f2bdc151340 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.009+0000 7f2bdaffd640 1 --2- 172.21.15.76:0/701427615 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f2bc80744b0 0x7f2bc80769c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0007fd0 tx=0x7f2bd0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:37.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.015+0000 7f2be2213640 1 --2- 172.21.15.76:0/701427615 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2bdc154a50 0x7f2bdc14e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.015+0000 7f2be2213640 1 -- 172.21.15.76:0/701427615 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f2bdc15d8e0 con 0x7f2bdc154a50 2024-04-25T19:18:37.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.015+0000 7f2bdb7fe640 1 --2- 172.21.15.76:0/701427615 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2bdc154a50 0x7f2bdc14e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.016+0000 7f2bdb7fe640 1 -- 172.21.15.76:0/701427615 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2bdc076020 con 0x7f2bdc154a50 2024-04-25T19:18:37.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.016+0000 7f2bdb7fe640 1 --2- 172.21.15.76:0/701427615 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2bdc154a50 0x7f2bdc14e5c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0011e0 tx=0x7f2bcc001150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:37.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.016+0000 7f2bc3fff640 1 -- 172.21.15.76:0/701427615 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc019050 con 0x7f2bdc154a50 2024-04-25T19:18:37.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.016+0000 7f2bc3fff640 1 -- 172.21.15.76:0/701427615 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4896 (secure 0 0 0) 0x7f2bcc01b820 con 0x7f2bdc154a50 2024-04-25T19:18:37.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.016+0000 7f2bc3fff640 1 -- 172.21.15.76:0/701427615 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2bdc154a50 msgr2=0x7f2bdc14e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:37.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.016+0000 7f2bc3fff640 1 --2- 172.21.15.76:0/701427615 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2bdc154a50 0x7f2bdc14e5c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0011e0 tx=0x7f2bcc001150 comp rx=0 tx=0).stop 2024-04-25T19:18:37.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.016+0000 7f2bc3fff640 1 -- 172.21.15.76:0/701427615 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2bdc154a50 msgr2=0x7f2bdc14e5c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:18:37.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.016+0000 7f2bc3fff640 1 --2- 172.21.15.76:0/701427615 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2bdc154a50 0x7f2bdc14e5c0 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0011e0 tx=0x7f2bcc001150 comp rx=0 tx=0).stop 2024-04-25T19:18:37.019 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:18:37.019 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:18:37.019 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:18:37.019 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "peon", 2024-04-25T19:18:37.019 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 24, 2024-04-25T19:18:37.019 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 120795, 2024-04-25T19:18:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:18:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:18:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-04-25T19:18:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:18:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 14, 2024-04-25T19:18:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:18:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:18:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:18:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:18:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:18:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:37.022 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:37.022 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:37.022 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:37.022 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:37.022 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:37.022 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:37.022 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:37.022 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:18:37.022 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:18:37.022 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:18:37.022 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:18:37.022 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:18:37.022 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:18:37.022 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:18:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:18:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:18:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:18:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:18:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:18:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:18:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:18:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:18:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:18:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:18:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:18:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:18:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:18:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:18:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:18:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:18:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:18:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:18:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:18:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:18:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:18:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:18:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:18:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:18:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:18:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:18:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:18:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:18:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:18:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:18:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:18:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:18:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:18:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": [ 2024-04-25T19:18:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:37.030 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 3 2024-04-25T19:18:37.030 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:37.030 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:37.030 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": [ 2024-04-25T19:18:37.030 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:37.030 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:37.030 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:37.030 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 5 2024-04-25T19:18:37.030 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:37.030 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:37.030 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:18:37.030 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:37.030 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:37.030 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:37.030 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 4 2024-04-25T19:18:37.031 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:37.031 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:37.031 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:37.031 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:18:37.031 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:18:37.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.016+0000 7f2be2213640 1 -- 172.21.15.76:0/701427615 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f2bc80744b0 msgr2=0x7f2bc80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:37.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.016+0000 7f2be2213640 1 --2- 172.21.15.76:0/701427615 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f2bc80744b0 0x7f2bc80769c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0007fd0 tx=0x7f2bd0009040 comp rx=0 tx=0).stop 2024-04-25T19:18:37.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.016+0000 7f2be2213640 1 -- 172.21.15.76:0/701427615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bdc14b8b0 msgr2=0x7f2bdc150e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:37.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.016+0000 7f2be2213640 1 --2- 172.21.15.76:0/701427615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2bdc14b8b0 0x7f2bdc150e00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009eb0 tx=0x7f2bd4004a40 comp rx=0 tx=0).stop 2024-04-25T19:18:37.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.016+0000 7f2bdbfff640 1 -- 172.21.15.76:0/701427615 reap_dead start 2024-04-25T19:18:37.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.016+0000 7f2bdbfff640 1 -- 172.21.15.76:0/701427615 reap_dead start 2024-04-25T19:18:37.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.017+0000 7f2be2213640 1 -- 172.21.15.76:0/701427615 shutdown_connections 2024-04-25T19:18:37.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.017+0000 7f2be2213640 1 -- 172.21.15.76:0/701427615 >> 172.21.15.76:0/701427615 conn(0x7f2bdc070800 msgr2=0x7f2bdc070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:37.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.017+0000 7f2be2213640 1 -- 172.21.15.76:0/701427615 shutdown_connections 2024-04-25T19:18:37.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.017+0000 7f2be2213640 1 -- 172.21.15.76:0/701427615 wait complete. 2024-04-25T19:18:37.033 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T19:18:37.033 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-04-25T19:18:37.033 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-04-25T19:18:37.033 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-04-25T19:18:37.033 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-04-25T19:18:37.033 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T19:18:37.034 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:18:37.046 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T19:18:37.046 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-04-25T19:18:37.047 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:18:37.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.151+0000 7efd3bbf8640 1 Processor -- start 2024-04-25T19:18:37.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.151+0000 7efd3bbf8640 1 -- start start 2024-04-25T19:18:37.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.152+0000 7efd3bbf8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efd34158060 0x7efd340c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.152+0000 7efd3bbf8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7efd340cab60 0x7efd3415af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.152+0000 7efd3bbf8640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd340c56a0 con 0x7efd340ca750 2024-04-25T19:18:37.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.152+0000 7efd3bbf8640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7efd340c5960 con 0x7efd340cab60 2024-04-25T19:18:37.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.152+0000 7efd3bbf8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7efd340c5c40 con 0x7efd34158060 2024-04-25T19:18:37.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.153+0000 7efd3a195640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7efd340cab60 0x7efd3415af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.153+0000 7efd3a195640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7efd340cab60 0x7efd3415af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37168/0 (socket says 172.21.15.76:37168) 2024-04-25T19:18:37.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.153+0000 7efd3a195640 1 -- 172.21.15.76:0/3177119258 learned_addr learned my addr 172.21.15.76:0/3177119258 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:37.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.153+0000 7efd39193640 1 --1- 172.21.15.76:0/3177119258 >> v1:172.21.15.76:6789/0 conn(0x7efd340ca750 0x7efd340cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57946/0 (socket says 172.21.15.76:57946) 2024-04-25T19:18:37.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.153+0000 7efd39994640 1 --2- 172.21.15.76:0/3177119258 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efd34158060 0x7efd340c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.153+0000 7efd3a195640 1 -- 172.21.15.76:0/3177119258 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efd34158060 msgr2=0x7efd340c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:37.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.153+0000 7efd3a195640 1 --2- 172.21.15.76:0/3177119258 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efd34158060 0x7efd340c4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.153+0000 7efd3a195640 1 -- 172.21.15.76:0/3177119258 >> v1:172.21.15.76:6789/0 conn(0x7efd340ca750 legacy=0x7efd340cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:37.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.153+0000 7efd3a195640 1 -- 172.21.15.76:0/3177119258 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd340c5f20 con 0x7efd340cab60 2024-04-25T19:18:37.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.153+0000 7efd38992640 1 -- 172.21.15.76:0/3177119258 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2321234829 0 0) 0x7efd340c56a0 con 0x7efd340ca750 2024-04-25T19:18:37.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.154+0000 7efd3a195640 1 --2- 172.21.15.76:0/3177119258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd340cab60 0x7efd3415af80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009870 tx=0x7efd2c02a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b9bf7614ad81b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:37.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.154+0000 7efd38992640 1 -- 172.21.15.76:0/3177119258 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c037070 con 0x7efd340cab60 2024-04-25T19:18:37.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.154+0000 7efd38992640 1 -- 172.21.15.76:0/3177119258 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd2c032640 con 0x7efd340cab60 2024-04-25T19:18:37.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.154+0000 7efd38992640 1 -- 172.21.15.76:0/3177119258 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c032a10 con 0x7efd340cab60 2024-04-25T19:18:37.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.154+0000 7efd3bbf8640 1 -- 172.21.15.76:0/3177119258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd340cab60 msgr2=0x7efd3415af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:37.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.154+0000 7efd3bbf8640 1 --2- 172.21.15.76:0/3177119258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd340cab60 0x7efd3415af80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009870 tx=0x7efd2c02a320 comp rx=0 tx=0).stop 2024-04-25T19:18:37.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.154+0000 7efd3bbf8640 1 -- 172.21.15.76:0/3177119258 shutdown_connections 2024-04-25T19:18:37.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.154+0000 7efd3bbf8640 1 --2- 172.21.15.76:0/3177119258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd340cab60 0x7efd3415af80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.155+0000 7efd3bbf8640 1 --2- 172.21.15.76:0/3177119258 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efd34158060 0x7efd340c4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.155+0000 7efd3bbf8640 1 -- 172.21.15.76:0/3177119258 >> 172.21.15.76:0/3177119258 conn(0x7efd34068b10 msgr2=0x7efd34072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:37.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.155+0000 7efd3bbf8640 1 -- 172.21.15.76:0/3177119258 shutdown_connections 2024-04-25T19:18:37.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.155+0000 7efd3bbf8640 1 -- 172.21.15.76:0/3177119258 wait complete. 2024-04-25T19:18:37.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.156+0000 7efd3bbf8640 1 Processor -- start 2024-04-25T19:18:37.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.156+0000 7efd3bbf8640 1 -- start start 2024-04-25T19:18:37.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.157+0000 7efd3bbf8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efd340ca750 0x7efd340c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.157+0000 7efd3bbf8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd340cab60 0x7efd3406d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.157+0000 7efd3bbf8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd34158060 0x7efd34150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.157+0000 7efd3bbf8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efd340c5320 con 0x7efd34158060 2024-04-25T19:18:37.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.157+0000 7efd3bbf8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efd340c6200 con 0x7efd340cab60 2024-04-25T19:18:37.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.158+0000 7efd3bbf8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7efd340ccef0 con 0x7efd340ca750 2024-04-25T19:18:37.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.158+0000 7efd3a195640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd34158060 0x7efd34150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.158+0000 7efd3a195640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd34158060 0x7efd34150ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37170/0 (socket says 172.21.15.76:37170) 2024-04-25T19:18:37.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.158+0000 7efd3a195640 1 -- 172.21.15.76:0/604465874 learned_addr learned my addr 172.21.15.76:0/604465874 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:37.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.158+0000 7efd39193640 1 -- 172.21.15.76:0/604465874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd340cab60 msgr2=0x7efd3406d880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3300/0 2024-04-25T19:18:37.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.158+0000 7efd39193640 1 --2- 172.21.15.76:0/604465874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd340cab60 0x7efd3406d880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:18:37.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.158+0000 7efd39994640 1 --2- 172.21.15.76:0/604465874 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efd340ca750 0x7efd340c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.158+0000 7efd3a195640 1 -- 172.21.15.76:0/604465874 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efd340ca750 msgr2=0x7efd340c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:37.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.158+0000 7efd3a195640 1 --2- 172.21.15.76:0/604465874 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efd340ca750 0x7efd340c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.158+0000 7efd3a195640 1 -- 172.21.15.76:0/604465874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd340cab60 msgr2=0x7efd3406d880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:18:37.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.158+0000 7efd3a195640 1 --2- 172.21.15.76:0/604465874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd340cab60 0x7efd3406d880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.158+0000 7efd3a195640 1 -- 172.21.15.76:0/604465874 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3406ddc0 con 0x7efd34158060 2024-04-25T19:18:37.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.158+0000 7efd3a195640 1 --2- 172.21.15.76:0/604465874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd34158060 0x7efd34150ea0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efd2c02a360 tx=0x7efd2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:37.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.158+0000 7efd2a7fc640 1 -- 172.21.15.76:0/604465874 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c041020 con 0x7efd34158060 2024-04-25T19:18:37.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.158+0000 7efd2a7fc640 1 -- 172.21.15.76:0/604465874 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd2c030770 con 0x7efd34158060 2024-04-25T19:18:37.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.158+0000 7efd3bbf8640 1 -- 172.21.15.76:0/604465874 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd341f5e30 con 0x7efd34158060 2024-04-25T19:18:37.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.158+0000 7efd2a7fc640 1 -- 172.21.15.76:0/604465874 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2c037040 con 0x7efd34158060 2024-04-25T19:18:37.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.158+0000 7efd3bbf8640 1 -- 172.21.15.76:0/604465874 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd341f63f0 con 0x7efd34158060 2024-04-25T19:18:37.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.159+0000 7efd3bbf8640 1 -- 172.21.15.76:0/604465874 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd34076480 con 0x7efd34158060 2024-04-25T19:18:37.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.160+0000 7efd2a7fc640 1 -- 172.21.15.76:0/604465874 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efd2c044020 con 0x7efd34158060 2024-04-25T19:18:37.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.160+0000 7efd2a7fc640 1 --2- 172.21.15.76:0/604465874 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7efd18074490 0x7efd180769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:37.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.161+0000 7efd2a7fc640 1 -- 172.21.15.76:0/604465874 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7efd2c0b59c0 con 0x7efd34158060 2024-04-25T19:18:37.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.161+0000 7efd39994640 1 --2- 172.21.15.76:0/604465874 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7efd18074490 0x7efd180769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:37.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.161+0000 7efd39994640 1 --2- 172.21.15.76:0/604465874 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7efd18074490 0x7efd180769a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efd300099c0 tx=0x7efd30003990 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:37.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.165+0000 7efd2a7fc640 1 -- 172.21.15.76:0/604465874 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efd34076480 con 0x7efd34158060 2024-04-25T19:18:37.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.418+0000 7efd3bbf8640 1 -- 172.21.15.76:0/604465874 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7efd34157870 con 0x7efd34158060 2024-04-25T19:18:37.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.418+0000 7efd2a7fc640 1 -- 172.21.15.76:0/604465874 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7efd34157870 con 0x7efd34158060 2024-04-25T19:18:37.421 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":24,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:18:37.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.420+0000 7efd3bbf8640 1 -- 172.21.15.76:0/604465874 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7efd18074490 msgr2=0x7efd180769a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:37.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.420+0000 7efd3bbf8640 1 --2- 172.21.15.76:0/604465874 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7efd18074490 0x7efd180769a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efd300099c0 tx=0x7efd30003990 comp rx=0 tx=0).stop 2024-04-25T19:18:37.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.420+0000 7efd3bbf8640 1 -- 172.21.15.76:0/604465874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd34158060 msgr2=0x7efd34150ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:37.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.420+0000 7efd3bbf8640 1 --2- 172.21.15.76:0/604465874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd34158060 0x7efd34150ea0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efd2c02a360 tx=0x7efd2c0046c0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.421+0000 7efd3bbf8640 1 -- 172.21.15.76:0/604465874 shutdown_connections 2024-04-25T19:18:37.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.421+0000 7efd3bbf8640 1 --2- 172.21.15.76:0/604465874 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7efd18074490 0x7efd180769a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.421+0000 7efd3bbf8640 1 --2- 172.21.15.76:0/604465874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd34158060 0x7efd34150ea0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.421+0000 7efd3bbf8640 1 --2- 172.21.15.76:0/604465874 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efd340cab60 0x7efd3406d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.421+0000 7efd3bbf8640 1 --2- 172.21.15.76:0/604465874 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efd340ca750 0x7efd340c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:37.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.421+0000 7efd3bbf8640 1 -- 172.21.15.76:0/604465874 >> 172.21.15.76:0/604465874 conn(0x7efd34068b10 msgr2=0x7efd34075d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:37.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.421+0000 7efd3bbf8640 1 -- 172.21.15.76:0/604465874 shutdown_connections 2024-04-25T19:18:37.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:37.421+0000 7efd3bbf8640 1 -- 172.21.15.76:0/604465874 wait complete. 2024-04-25T19:18:40.435 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:18:40.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.549+0000 7f23ca51f640 1 Processor -- start 2024-04-25T19:18:40.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.549+0000 7f23ca51f640 1 -- start start 2024-04-25T19:18:40.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.550+0000 7f23ca51f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23c40cb880 0x7f23c415d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:40.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.550+0000 7f23ca51f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f23c4072ba0 0x7f23c4072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:40.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.550+0000 7f23ca51f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23c40cbd50 con 0x7f23c40cad30 2024-04-25T19:18:40.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.550+0000 7f23ca51f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f23c4161340 con 0x7f23c4072ba0 2024-04-25T19:18:40.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.550+0000 7f23ca51f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f23c41607a0 con 0x7f23c40cb880 2024-04-25T19:18:40.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.550+0000 7f23c8abc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f23c4072ba0 0x7f23c4072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:40.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.550+0000 7f23c37fe640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f23c40cad30 legacy=0x7f23c4160c30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-04-25T19:18:40.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.551+0000 7f23c8abc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f23c4072ba0 0x7f23c4072fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34804/0 (socket says 172.21.15.76:34804) 2024-04-25T19:18:40.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.551+0000 7f23c8abc640 1 -- 172.21.15.76:0/2780366257 learned_addr learned my addr 172.21.15.76:0/2780366257 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:40.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.551+0000 7f23c3fff640 1 --2- 172.21.15.76:0/2780366257 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23c40cb880 0x7f23c415d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:40.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.551+0000 7f23c8abc640 1 -- 172.21.15.76:0/2780366257 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23c40cb880 msgr2=0x7f23c415d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:40.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.551+0000 7f23c8abc640 1 --2- 172.21.15.76:0/2780366257 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23c40cb880 0x7f23c415d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:40.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.551+0000 7f23c8abc640 1 -- 172.21.15.76:0/2780366257 >> v1:172.21.15.105:6789/0 conn(0x7f23c40cad30 legacy=0x7f23c4160c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:18:40.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.551+0000 7f23c8abc640 1 -- 172.21.15.76:0/2780366257 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c40cc010 con 0x7f23c4072ba0 2024-04-25T19:18:40.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.551+0000 7f23c8abc640 1 --2- 172.21.15.76:0/2780366257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23c4072ba0 0x7f23c4072fb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009e50 tx=0x7f23bc02a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c80d6640af580a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:40.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.552+0000 7f23c2ffd640 1 -- 172.21.15.76:0/2780366257 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc036070 con 0x7f23c4072ba0 2024-04-25T19:18:40.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.552+0000 7f23c2ffd640 1 -- 172.21.15.76:0/2780366257 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23bc031600 con 0x7f23c4072ba0 2024-04-25T19:18:40.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.552+0000 7f23c2ffd640 1 -- 172.21.15.76:0/2780366257 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc0319d0 con 0x7f23c4072ba0 2024-04-25T19:18:40.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.552+0000 7f23ca51f640 1 -- 172.21.15.76:0/2780366257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23c4072ba0 msgr2=0x7f23c4072fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:40.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.552+0000 7f23ca51f640 1 --2- 172.21.15.76:0/2780366257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23c4072ba0 0x7f23c4072fb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009e50 tx=0x7f23bc02a750 comp rx=0 tx=0).stop 2024-04-25T19:18:40.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.552+0000 7f23ca51f640 1 -- 172.21.15.76:0/2780366257 shutdown_connections 2024-04-25T19:18:40.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.552+0000 7f23ca51f640 1 --2- 172.21.15.76:0/2780366257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23c4072ba0 0x7f23c4072fb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:40.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.552+0000 7f23ca51f640 1 --2- 172.21.15.76:0/2780366257 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23c40cb880 0x7f23c415d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:40.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.552+0000 7f23ca51f640 1 -- 172.21.15.76:0/2780366257 >> 172.21.15.76:0/2780366257 conn(0x7f23c4070b00 msgr2=0x7f23c4070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:40.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.553+0000 7f23ca51f640 1 -- 172.21.15.76:0/2780366257 shutdown_connections 2024-04-25T19:18:40.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.553+0000 7f23ca51f640 1 -- 172.21.15.76:0/2780366257 wait complete. 2024-04-25T19:18:40.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.554+0000 7f23ca51f640 1 Processor -- start 2024-04-25T19:18:40.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.554+0000 7f23ca51f640 1 -- start start 2024-04-25T19:18:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.556+0000 7f23ca51f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23c4072ba0 0x7f23c41f2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.556+0000 7f23ca51f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23c40cad30 0x7f23c406d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.556+0000 7f23ca51f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23c40cb880 0x7f23c4159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.556+0000 7f23ca51f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f23c406e7f0 con 0x7f23c4072ba0 2024-04-25T19:18:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.556+0000 7f23ca51f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f23c4160430 con 0x7f23c40cb880 2024-04-25T19:18:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.556+0000 7f23ca51f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f23c40cc5d0 con 0x7f23c40cad30 2024-04-25T19:18:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.556+0000 7f23c3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23c4072ba0 0x7f23c41f2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.556+0000 7f23c3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23c4072ba0 0x7f23c41f2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34814/0 (socket says 172.21.15.76:34814) 2024-04-25T19:18:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.556+0000 7f23c3fff640 1 -- 172.21.15.76:0/4249400964 learned_addr learned my addr 172.21.15.76:0/4249400964 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.556+0000 7f23c8abc640 1 -- 172.21.15.76:0/4249400964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23c40cb880 msgr2=0x7f23c4159d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3300/0 2024-04-25T19:18:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.556+0000 7f23c8abc640 1 --2- 172.21.15.76:0/4249400964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23c40cb880 0x7f23c4159d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:18:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.556+0000 7f23c37fe640 1 --2- 172.21.15.76:0/4249400964 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23c40cad30 0x7f23c406d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:40.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.556+0000 7f23c3fff640 1 -- 172.21.15.76:0/4249400964 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23c40cad30 msgr2=0x7f23c406d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:40.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.556+0000 7f23c3fff640 1 --2- 172.21.15.76:0/4249400964 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23c40cad30 0x7f23c406d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:40.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.556+0000 7f23c3fff640 1 -- 172.21.15.76:0/4249400964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23c40cb880 msgr2=0x7f23c4159d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:18:40.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.556+0000 7f23c3fff640 1 --2- 172.21.15.76:0/4249400964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23c40cb880 0x7f23c4159d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:40.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.556+0000 7f23c3fff640 1 -- 172.21.15.76:0/4249400964 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c40c9f40 con 0x7f23c4072ba0 2024-04-25T19:18:40.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.556+0000 7f23c3fff640 1 --2- 172.21.15.76:0/4249400964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23c4072ba0 0x7f23c41f2c80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f23b400c820 tx=0x7f23b400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:40.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.556+0000 7f23c0ff9640 1 -- 172.21.15.76:0/4249400964 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b4015070 con 0x7f23c4072ba0 2024-04-25T19:18:40.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.556+0000 7f23c0ff9640 1 -- 172.21.15.76:0/4249400964 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23b4004020 con 0x7f23c4072ba0 2024-04-25T19:18:40.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.556+0000 7f23ca51f640 1 -- 172.21.15.76:0/4249400964 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c41f6190 con 0x7f23c4072ba0 2024-04-25T19:18:40.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.556+0000 7f23c0ff9640 1 -- 172.21.15.76:0/4249400964 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b40043f0 con 0x7f23c4072ba0 2024-04-25T19:18:40.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.556+0000 7f23ca51f640 1 -- 172.21.15.76:0/4249400964 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c41f6750 con 0x7f23c4072ba0 2024-04-25T19:18:40.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.557+0000 7f23ca51f640 1 -- 172.21.15.76:0/4249400964 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c406a390 con 0x7f23c4072ba0 2024-04-25T19:18:40.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.558+0000 7f23c0ff9640 1 -- 172.21.15.76:0/4249400964 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f23b40077d0 con 0x7f23c4072ba0 2024-04-25T19:18:40.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.558+0000 7f23c0ff9640 1 --2- 172.21.15.76:0/4249400964 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f23b0074460 0x7f23b0076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:40.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.559+0000 7f23c0ff9640 1 -- 172.21.15.76:0/4249400964 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f23b4095010 con 0x7f23c4072ba0 2024-04-25T19:18:40.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.559+0000 7f23c37fe640 1 --2- 172.21.15.76:0/4249400964 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f23b0074460 0x7f23b0076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:40.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.559+0000 7f23c37fe640 1 --2- 172.21.15.76:0/4249400964 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f23b0074460 0x7f23b0076970 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f23b8006fd0 tx=0x7f23b8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:40.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.563+0000 7f23c0ff9640 1 -- 172.21.15.76:0/4249400964 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f23b405f720 con 0x7f23c4072ba0 2024-04-25T19:18:40.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.807+0000 7f23ca51f640 1 -- 172.21.15.76:0/4249400964 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f23c406ea90 con 0x7f23c4072ba0 2024-04-25T19:18:40.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.807+0000 7f23c0ff9640 1 -- 172.21.15.76:0/4249400964 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7f23b405f020 con 0x7f23c4072ba0 2024-04-25T19:18:40.810 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":24,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:18:40.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.809+0000 7f23ca51f640 1 -- 172.21.15.76:0/4249400964 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f23b0074460 msgr2=0x7f23b0076970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:40.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.809+0000 7f23ca51f640 1 --2- 172.21.15.76:0/4249400964 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f23b0074460 0x7f23b0076970 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f23b8006fd0 tx=0x7f23b8008040 comp rx=0 tx=0).stop 2024-04-25T19:18:40.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.809+0000 7f23ca51f640 1 -- 172.21.15.76:0/4249400964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23c4072ba0 msgr2=0x7f23c41f2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:40.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.809+0000 7f23ca51f640 1 --2- 172.21.15.76:0/4249400964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23c4072ba0 0x7f23c41f2c80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f23b400c820 tx=0x7f23b400cce0 comp rx=0 tx=0).stop 2024-04-25T19:18:40.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.809+0000 7f23ca51f640 1 -- 172.21.15.76:0/4249400964 shutdown_connections 2024-04-25T19:18:40.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.809+0000 7f23ca51f640 1 --2- 172.21.15.76:0/4249400964 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f23b0074460 0x7f23b0076970 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:40.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.809+0000 7f23ca51f640 1 --2- 172.21.15.76:0/4249400964 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f23c40cb880 0x7f23c4159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:40.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.809+0000 7f23ca51f640 1 --2- 172.21.15.76:0/4249400964 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f23c40cad30 0x7f23c406d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:40.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.809+0000 7f23ca51f640 1 --2- 172.21.15.76:0/4249400964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23c4072ba0 0x7f23c41f2c80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:40.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.809+0000 7f23ca51f640 1 -- 172.21.15.76:0/4249400964 >> 172.21.15.76:0/4249400964 conn(0x7f23c4070b00 msgr2=0x7f23c4070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:40.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.809+0000 7f23ca51f640 1 -- 172.21.15.76:0/4249400964 shutdown_connections 2024-04-25T19:18:40.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:40.809+0000 7f23ca51f640 1 -- 172.21.15.76:0/4249400964 wait complete. 2024-04-25T19:18:43.823 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:18:43.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.981+0000 7f0c1bbce640 1 Processor -- start 2024-04-25T19:18:43.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.981+0000 7f0c1bbce640 1 -- start start 2024-04-25T19:18:43.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.983+0000 7f0c1bbce640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0c14156f60 0x7f0c140cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:43.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.983+0000 7f0c1bbce640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0c140c7ea0 0x7f0c140cf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:43.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.983+0000 7f0c1bbce640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c14073c30 con 0x7f0c140c95c0 2024-04-25T19:18:43.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.983+0000 7f0c1bbce640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f0c1415d730 con 0x7f0c140c7ea0 2024-04-25T19:18:43.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.983+0000 7f0c1bbce640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f0c140746b0 con 0x7f0c14156f60 2024-04-25T19:18:43.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.983+0000 7f0c1a16b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0c140c7ea0 0x7f0c140cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:43.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.983+0000 7f0c1996a640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f0c14156f60 msgr2=0x7f0c140cefe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-04-25T19:18:43.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.983+0000 7f0c19169640 1 -- >> v1:172.21.15.105:6789/0 conn(0x7f0c140c95c0 legacy=0x7f0c140c7790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-04-25T19:18:43.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.983+0000 7f0c1a16b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0c140c7ea0 0x7f0c140cf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34828/0 (socket says 172.21.15.76:34828) 2024-04-25T19:18:43.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.983+0000 7f0c1a16b640 1 -- 172.21.15.76:0/375629393 learned_addr learned my addr 172.21.15.76:0/375629393 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:43.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.983+0000 7f0c1996a640 1 --2- 172.21.15.76:0/375629393 >> v2:172.21.15.105:3300/0 conn(0x7f0c14156f60 0x7f0c140cefe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:18:43.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.983+0000 7f0c1a16b640 1 -- 172.21.15.76:0/375629393 >> v2:172.21.15.105:3300/0 conn(0x7f0c14156f60 msgr2=0x7f0c140cefe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:18:43.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.983+0000 7f0c1a16b640 1 --2- 172.21.15.76:0/375629393 >> v2:172.21.15.105:3300/0 conn(0x7f0c14156f60 0x7f0c140cefe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:43.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.983+0000 7f0c1a16b640 1 -- 172.21.15.76:0/375629393 >> v1:172.21.15.105:6789/0 conn(0x7f0c140c95c0 legacy=0x7f0c140c7790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:18:43.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.983+0000 7f0c1a16b640 1 -- 172.21.15.76:0/375629393 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c14073940 con 0x7f0c140c7ea0 2024-04-25T19:18:43.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.984+0000 7f0c1a16b640 1 --2- 172.21.15.76:0/375629393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c140c7ea0 0x7f0c140cf520 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0c1406c720 tx=0x7f0c0c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17cf0e085608c208 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:43.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.984+0000 7f0c18968640 1 -- 172.21.15.76:0/375629393 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c036070 con 0x7f0c140c7ea0 2024-04-25T19:18:43.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.985+0000 7f0c18968640 1 -- 172.21.15.76:0/375629393 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c0c031600 con 0x7f0c140c7ea0 2024-04-25T19:18:43.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.985+0000 7f0c18968640 1 -- 172.21.15.76:0/375629393 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0319d0 con 0x7f0c140c7ea0 2024-04-25T19:18:43.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.987+0000 7f0c1bbce640 1 -- 172.21.15.76:0/375629393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c140c7ea0 msgr2=0x7f0c140cf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:43.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.987+0000 7f0c1bbce640 1 --2- 172.21.15.76:0/375629393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c140c7ea0 0x7f0c140cf520 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0c1406c720 tx=0x7f0c0c02a710 comp rx=0 tx=0).stop 2024-04-25T19:18:43.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.988+0000 7f0c1bbce640 1 -- 172.21.15.76:0/375629393 shutdown_connections 2024-04-25T19:18:43.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.988+0000 7f0c1bbce640 1 --2- 172.21.15.76:0/375629393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c140c7ea0 0x7f0c140cf520 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:43.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.988+0000 7f0c1bbce640 1 --2- 172.21.15.76:0/375629393 >> v2:172.21.15.105:3300/0 conn(0x7f0c14156f60 0x7f0c140cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:43.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.988+0000 7f0c1bbce640 1 -- 172.21.15.76:0/375629393 >> 172.21.15.76:0/375629393 conn(0x7f0c14071a30 msgr2=0x7f0c14071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:43.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.988+0000 7f0c1bbce640 1 -- 172.21.15.76:0/375629393 shutdown_connections 2024-04-25T19:18:43.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.988+0000 7f0c1bbce640 1 -- 172.21.15.76:0/375629393 wait complete. 2024-04-25T19:18:43.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.991+0000 7f0c1bbce640 1 Processor -- start 2024-04-25T19:18:43.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.991+0000 7f0c1bbce640 1 -- start start 2024-04-25T19:18:43.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.993+0000 7f0c1bbce640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0c140c7ea0 0x7f0c1414c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:43.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.993+0000 7f0c1bbce640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c140c95c0 0x7f0c1414f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:43.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.993+0000 7f0c1bbce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c14156f60 0x7f0c140ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:43.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.993+0000 7f0c1bbce640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0c140740e0 con 0x7f0c140c95c0 2024-04-25T19:18:43.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.993+0000 7f0c1bbce640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1415da10 con 0x7f0c14156f60 2024-04-25T19:18:43.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.993+0000 7f0c1bbce640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f0c140c9ea0 con 0x7f0c140c7ea0 2024-04-25T19:18:43.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.993+0000 7f0c1a16b640 1 -- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c14156f60 msgr2=0x7f0c140ca8e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3300/0 2024-04-25T19:18:43.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.993+0000 7f0c1a16b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c14156f60 0x7f0c140ca8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:18:43.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.993+0000 7f0c19169640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c140c95c0 0x7f0c1414f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:43.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.993+0000 7f0c19169640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c140c95c0 0x7f0c1414f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34842/0 (socket says 172.21.15.76:34842) 2024-04-25T19:18:43.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.993+0000 7f0c19169640 1 -- 172.21.15.76:0/3652484373 learned_addr learned my addr 172.21.15.76:0/3652484373 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:43.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.994+0000 7f0c1996a640 1 --2- 172.21.15.76:0/3652484373 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0c140c7ea0 0x7f0c1414c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:43.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.994+0000 7f0c19169640 1 -- 172.21.15.76:0/3652484373 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0c140c7ea0 msgr2=0x7f0c1414c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:43.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.994+0000 7f0c19169640 1 --2- 172.21.15.76:0/3652484373 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0c140c7ea0 0x7f0c1414c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:43.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.994+0000 7f0c19169640 1 -- 172.21.15.76:0/3652484373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c14156f60 msgr2=0x7f0c140ca8e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:18:43.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.994+0000 7f0c19169640 1 --2- 172.21.15.76:0/3652484373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c14156f60 0x7f0c140ca8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:43.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.994+0000 7f0c19169640 1 -- 172.21.15.76:0/3652484373 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1414fc20 con 0x7f0c140c95c0 2024-04-25T19:18:43.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.994+0000 7f0c19169640 1 --2- 172.21.15.76:0/3652484373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c140c95c0 0x7f0c1414f6e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400b480 tx=0x7f0c0400b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:43.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.995+0000 7f0c0a7fc640 1 -- 172.21.15.76:0/3652484373 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c04003cc0 con 0x7f0c140c95c0 2024-04-25T19:18:43.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.995+0000 7f0c0a7fc640 1 -- 172.21.15.76:0/3652484373 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c04015070 con 0x7f0c140c95c0 2024-04-25T19:18:43.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.995+0000 7f0c0a7fc640 1 -- 172.21.15.76:0/3652484373 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c04003e60 con 0x7f0c140c95c0 2024-04-25T19:18:43.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.995+0000 7f0c1bbce640 1 -- 172.21.15.76:0/3652484373 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c140cb3c0 con 0x7f0c140c95c0 2024-04-25T19:18:43.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.995+0000 7f0c1bbce640 1 -- 172.21.15.76:0/3652484373 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c1414cd20 con 0x7f0c140c95c0 2024-04-25T19:18:43.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.997+0000 7f0bf7fff640 1 -- 172.21.15.76:0/3652484373 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c141560e0 con 0x7f0c140c95c0 2024-04-25T19:18:43.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.997+0000 7f0c0a7fc640 1 -- 172.21.15.76:0/3652484373 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0c04005290 con 0x7f0c140c95c0 2024-04-25T19:18:44.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.998+0000 7f0c0a7fc640 1 --2- 172.21.15.76:0/3652484373 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0bf8074490 0x7f0bf80769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:44.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.998+0000 7f0c0a7fc640 1 -- 172.21.15.76:0/3652484373 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f0c04094ee0 con 0x7f0c140c95c0 2024-04-25T19:18:44.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.998+0000 7f0c1996a640 1 --2- 172.21.15.76:0/3652484373 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0bf8074490 0x7f0bf80769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:44.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:43.999+0000 7f0c1996a640 1 --2- 172.21.15.76:0/3652484373 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0bf8074490 0x7f0bf80769a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0c100093d0 tx=0x7f0c10009360 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:44.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:44.003+0000 7f0c0a7fc640 1 -- 172.21.15.76:0/3652484373 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0c141560e0 con 0x7f0c140c95c0 2024-04-25T19:18:44.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:44.400+0000 7f0bf7fff640 1 -- 172.21.15.76:0/3652484373 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0c1406a390 con 0x7f0c140c95c0 2024-04-25T19:18:44.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:44.401+0000 7f0c0a7fc640 1 -- 172.21.15.76:0/3652484373 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7f0c0405f710 con 0x7f0c140c95c0 2024-04-25T19:18:44.402 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":24,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:18:44.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:44.405+0000 7f0bf7fff640 1 -- 172.21.15.76:0/3652484373 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0bf8074490 msgr2=0x7f0bf80769a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:44.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:44.405+0000 7f0bf7fff640 1 --2- 172.21.15.76:0/3652484373 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0bf8074490 0x7f0bf80769a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0c100093d0 tx=0x7f0c10009360 comp rx=0 tx=0).stop 2024-04-25T19:18:44.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:44.405+0000 7f0bf7fff640 1 -- 172.21.15.76:0/3652484373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c140c95c0 msgr2=0x7f0c1414f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:44.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:44.405+0000 7f0bf7fff640 1 --2- 172.21.15.76:0/3652484373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c140c95c0 0x7f0c1414f6e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400b480 tx=0x7f0c0400b940 comp rx=0 tx=0).stop 2024-04-25T19:18:44.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:44.405+0000 7f0bf7fff640 1 -- 172.21.15.76:0/3652484373 shutdown_connections 2024-04-25T19:18:44.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:44.405+0000 7f0bf7fff640 1 --2- 172.21.15.76:0/3652484373 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0bf8074490 0x7f0bf80769a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:44.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:44.405+0000 7f0bf7fff640 1 --2- 172.21.15.76:0/3652484373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c14156f60 0x7f0c140ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:44.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:44.405+0000 7f0bf7fff640 1 --2- 172.21.15.76:0/3652484373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c140c95c0 0x7f0c1414f6e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:44.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:44.405+0000 7f0bf7fff640 1 --2- 172.21.15.76:0/3652484373 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0c140c7ea0 0x7f0c1414c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:44.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:44.405+0000 7f0bf7fff640 1 -- 172.21.15.76:0/3652484373 >> 172.21.15.76:0/3652484373 conn(0x7f0c14071a30 msgr2=0x7f0c140c8330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:44.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:44.405+0000 7f0bf7fff640 1 -- 172.21.15.76:0/3652484373 shutdown_connections 2024-04-25T19:18:44.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:44.405+0000 7f0bf7fff640 1 -- 172.21.15.76:0/3652484373 wait complete. 2024-04-25T19:18:47.427 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:18:47.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.595+0000 7f52df2d8640 1 Processor -- start 2024-04-25T19:18:47.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.596+0000 7f52df2d8640 1 -- start start 2024-04-25T19:18:47.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.598+0000 7f52df2d8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52d8071520 0x7f52d8151350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:47.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.598+0000 7f52df2d8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f52d815ac70 0x7f52d815b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:47.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.598+0000 7f52df2d8640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52d8151890 con 0x7f52d8073880 2024-04-25T19:18:47.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.598+0000 7f52df2d8640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f52d8071ad0 con 0x7f52d815ac70 2024-04-25T19:18:47.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.598+0000 7f52df2d8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f52d8071d70 con 0x7f52d8071520 2024-04-25T19:18:47.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.598+0000 7f52dd875640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f52d815ac70 0x7f52d815b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:47.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.598+0000 7f52dc873640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f52d8073880 0x7f52d80c83d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:34008/0 (socket says 172.21.15.76:34008) 2024-04-25T19:18:47.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.598+0000 7f52dd875640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f52d815ac70 0x7f52d815b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34846/0 (socket says 172.21.15.76:34846) 2024-04-25T19:18:47.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.598+0000 7f52dc873640 1 -- 172.21.15.76:0/2499217323 learned_addr learned my addr 172.21.15.76:0/2499217323 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:18:47.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.598+0000 7f52dd875640 1 -- 172.21.15.76:0/2499217323 >> v2:172.21.15.76:3300/0 conn(0x7f52d815ac70 msgr2=0x7f52d815b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T19:18:47.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.598+0000 7f52dd875640 1 -- 172.21.15.76:0/2499217323 >> v2:172.21.15.76:3300/0 conn(0x7f52d815ac70 msgr2=0x7f52d815b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:18:47.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.598+0000 7f52dd875640 1 --2- 172.21.15.76:0/2499217323 >> v2:172.21.15.76:3300/0 conn(0x7f52d815ac70 0x7f52d815b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:18:47.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.598+0000 7f52dd074640 1 --2- 172.21.15.76:0/2499217323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52d8071520 0x7f52d8151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:47.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.598+0000 7f52dd875640 1 --2- 172.21.15.76:0/2499217323 >> v2:172.21.15.76:3300/0 conn(0x7f52d815ac70 0x7f52d815b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:18:47.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.599+0000 7f52dd074640 1 -- 172.21.15.76:0/2499217323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52d8071520 msgr2=0x7f52d8151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T19:18:47.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.599+0000 7f52dd074640 1 -- 172.21.15.76:0/2499217323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52d8071520 msgr2=0x7f52d8151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:18:47.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.599+0000 7f52dd074640 1 --2- 172.21.15.76:0/2499217323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52d8071520 0x7f52d8151350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:18:47.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.599+0000 7f52dd074640 1 --2- 172.21.15.76:0/2499217323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52d8071520 0x7f52d8151350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:18:47.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.799+0000 7f52dd875640 1 --2- 172.21.15.76:0/2499217323 >> v2:172.21.15.76:3300/0 conn(0x7f52d815ac70 0x7f52d815b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:47.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.799+0000 7f52dd875640 1 -- 172.21.15.76:0/2499217323 >> v2:172.21.15.76:3300/0 conn(0x7f52d815ac70 msgr2=0x7f52d815b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T19:18:47.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.799+0000 7f52dd875640 1 -- 172.21.15.76:0/2499217323 >> v2:172.21.15.76:3300/0 conn(0x7f52d815ac70 msgr2=0x7f52d815b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:18:47.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.799+0000 7f52dd875640 1 --2- 172.21.15.76:0/2499217323 >> v2:172.21.15.76:3300/0 conn(0x7f52d815ac70 0x7f52d815b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:18:47.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.799+0000 7f52dd875640 1 --2- 172.21.15.76:0/2499217323 >> v2:172.21.15.76:3300/0 conn(0x7f52d815ac70 0x7f52d815b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T19:18:47.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.800+0000 7f52dd074640 1 --2- 172.21.15.76:0/2499217323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52d8071520 0x7f52d8151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:47.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.800+0000 7f52dd074640 1 -- 172.21.15.76:0/2499217323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52d8071520 msgr2=0x7f52d8151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T19:18:47.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.800+0000 7f52dd074640 1 -- 172.21.15.76:0/2499217323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52d8071520 msgr2=0x7f52d8151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:18:47.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.800+0000 7f52dd074640 1 --2- 172.21.15.76:0/2499217323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52d8071520 0x7f52d8151350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:18:47.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:47.800+0000 7f52dd074640 1 --2- 172.21.15.76:0/2499217323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52d8071520 0x7f52d8151350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T19:18:48.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:48.200+0000 7f52dd875640 1 --2- 172.21.15.76:0/2499217323 >> v2:172.21.15.76:3300/0 conn(0x7f52d815ac70 0x7f52d815b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:48.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:48.200+0000 7f52dd875640 1 -- 172.21.15.76:0/2499217323 >> v2:172.21.15.76:3300/0 conn(0x7f52d815ac70 msgr2=0x7f52d815b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T19:18:48.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:48.200+0000 7f52dd875640 1 -- 172.21.15.76:0/2499217323 >> v2:172.21.15.76:3300/0 conn(0x7f52d815ac70 msgr2=0x7f52d815b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:18:48.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:48.200+0000 7f52dd875640 1 --2- 172.21.15.76:0/2499217323 >> v2:172.21.15.76:3300/0 conn(0x7f52d815ac70 0x7f52d815b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:18:48.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:48.200+0000 7f52dd875640 1 --2- 172.21.15.76:0/2499217323 >> v2:172.21.15.76:3300/0 conn(0x7f52d815ac70 0x7f52d815b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T19:18:48.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:48.201+0000 7f52dd074640 1 --2- 172.21.15.76:0/2499217323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52d8071520 0x7f52d8151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:48.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:48.201+0000 7f52dd074640 1 -- 172.21.15.76:0/2499217323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52d8071520 msgr2=0x7f52d8151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T19:18:48.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:48.201+0000 7f52dd074640 1 -- 172.21.15.76:0/2499217323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52d8071520 msgr2=0x7f52d8151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:18:48.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:48.201+0000 7f52dd074640 1 --2- 172.21.15.76:0/2499217323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52d8071520 0x7f52d8151350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:18:48.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:48.201+0000 7f52dd074640 1 --2- 172.21.15.76:0/2499217323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52d8071520 0x7f52d8151350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T19:18:49.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:49.001+0000 7f52dd875640 1 --2- 172.21.15.76:0/2499217323 >> v2:172.21.15.76:3300/0 conn(0x7f52d815ac70 0x7f52d815b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:49.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:49.001+0000 7f52dd875640 1 -- 172.21.15.76:0/2499217323 >> v2:172.21.15.76:3300/0 conn(0x7f52d815ac70 msgr2=0x7f52d815b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T19:18:49.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:49.001+0000 7f52dd875640 1 -- 172.21.15.76:0/2499217323 >> v2:172.21.15.76:3300/0 conn(0x7f52d815ac70 msgr2=0x7f52d815b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:18:49.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:49.001+0000 7f52dd875640 1 --2- 172.21.15.76:0/2499217323 >> v2:172.21.15.76:3300/0 conn(0x7f52d815ac70 0x7f52d815b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:18:49.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:49.001+0000 7f52dd875640 1 --2- 172.21.15.76:0/2499217323 >> v2:172.21.15.76:3300/0 conn(0x7f52d815ac70 0x7f52d815b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T19:18:49.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:49.002+0000 7f52dd074640 1 --2- 172.21.15.76:0/2499217323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52d8071520 0x7f52d8151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:49.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:49.002+0000 7f52dd074640 1 -- 172.21.15.76:0/2499217323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52d8071520 msgr2=0x7f52d8151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T19:18:49.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:49.002+0000 7f52dd074640 1 -- 172.21.15.76:0/2499217323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52d8071520 msgr2=0x7f52d8151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:18:49.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:49.002+0000 7f52dd074640 1 --2- 172.21.15.76:0/2499217323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52d8071520 0x7f52d8151350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:18:49.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:49.002+0000 7f52dd074640 1 --2- 172.21.15.76:0/2499217323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52d8071520 0x7f52d8151350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T19:18:50.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:50.598+0000 7f52ceffd640 1 -- 172.21.15.76:0/2499217323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52d8071520 msgr2=0x7f52d8151350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:18:50.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:50.598+0000 7f52ceffd640 1 --2- 172.21.15.76:0/2499217323 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52d8071520 0x7f52d8151350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:50.598+0000 7f52ceffd640 1 -- 172.21.15.76:0/2499217323 >> v2:172.21.15.76:3300/0 conn(0x7f52d815ac70 msgr2=0x7f52d815b080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:18:50.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:50.598+0000 7f52ceffd640 1 --2- 172.21.15.76:0/2499217323 >> v2:172.21.15.76:3300/0 conn(0x7f52d815ac70 0x7f52d815b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:50.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:50.598+0000 7f52ceffd640 1 -- 172.21.15.76:0/2499217323 >> v1:172.21.15.76:6789/0 conn(0x7f52d8073880 legacy=0x7f52d80c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:50.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:50.598+0000 7f52ceffd640 1 --2- 172.21.15.76:0/2499217323 >> v2:172.21.15.105:3300/0 conn(0x7f52bc007430 0x7f52bc009920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:50.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:50.598+0000 7f52ceffd640 1 -- 172.21.15.76:0/2499217323 --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52d8074150 con 0x7f52bc0036a0 2024-04-25T19:18:50.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:50.598+0000 7f52ceffd640 1 -- 172.21.15.76:0/2499217323 --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f52d8071ad0 con 0x7f52bc007430 2024-04-25T19:18:50.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:50.598+0000 7f52dd074640 1 -- 172.21.15.76:0/2499217323 >> v1:172.21.15.105:6789/0 conn(0x7f52bc0036a0 legacy=0x7f52bc005b70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-04-25T19:18:50.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:50.598+0000 7f52dc873640 1 -- 172.21.15.76:0/2499217323 >> v2:172.21.15.105:3300/0 conn(0x7f52bc007430 msgr2=0x7f52bc009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-04-25T19:18:50.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:50.598+0000 7f52dc873640 1 --2- 172.21.15.76:0/2499217323 >> v2:172.21.15.105:3300/0 conn(0x7f52bc007430 0x7f52bc009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:18:50.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:50.799+0000 7f52dc873640 1 -- 172.21.15.76:0/2499217323 >> v2:172.21.15.105:3300/0 conn(0x7f52bc007430 msgr2=0x7f52bc009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-04-25T19:18:50.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:50.799+0000 7f52dd074640 1 -- 172.21.15.76:0/2499217323 >> v1:172.21.15.105:6789/0 conn(0x7f52bc0036a0 legacy=0x7f52bc005b70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-04-25T19:18:50.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:50.799+0000 7f52dc873640 1 --2- 172.21.15.76:0/2499217323 >> v2:172.21.15.105:3300/0 conn(0x7f52bc007430 0x7f52bc009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T19:18:51.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:51.200+0000 7f52dc873640 1 -- 172.21.15.76:0/2499217323 >> v2:172.21.15.105:3300/0 conn(0x7f52bc007430 msgr2=0x7f52bc009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-04-25T19:18:51.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:51.200+0000 7f52dd074640 1 -- 172.21.15.76:0/2499217323 >> v1:172.21.15.105:6789/0 conn(0x7f52bc0036a0 legacy=0x7f52bc005b70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-04-25T19:18:51.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:51.200+0000 7f52dc873640 1 --2- 172.21.15.76:0/2499217323 >> v2:172.21.15.105:3300/0 conn(0x7f52bc007430 0x7f52bc009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T19:18:52.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:52.001+0000 7f52dc873640 1 -- 172.21.15.76:0/2499217323 >> v2:172.21.15.105:3300/0 conn(0x7f52bc007430 msgr2=0x7f52bc009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-04-25T19:18:52.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:52.001+0000 7f52dd074640 1 -- 172.21.15.76:0/2499217323 >> v1:172.21.15.105:6789/0 conn(0x7f52bc0036a0 legacy=0x7f52bc005b70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.105:6789/0 2024-04-25T19:18:52.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:52.001+0000 7f52dc873640 1 --2- 172.21.15.76:0/2499217323 >> v2:172.21.15.105:3300/0 conn(0x7f52bc007430 0x7f52bc009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T19:18:53.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.598+0000 7f52ceffd640 1 -- 172.21.15.76:0/2499217323 >> v2:172.21.15.105:3300/0 conn(0x7f52bc007430 msgr2=0x7f52bc009920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:18:53.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.598+0000 7f52ceffd640 1 --2- 172.21.15.76:0/2499217323 >> v2:172.21.15.105:3300/0 conn(0x7f52bc007430 0x7f52bc009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:53.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.598+0000 7f52ceffd640 1 -- 172.21.15.76:0/2499217323 >> v1:172.21.15.105:6789/0 conn(0x7f52bc0036a0 legacy=0x7f52bc005b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:18:53.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.598+0000 7f52dd875640 1 -- 172.21.15.76:0/2499217323 reap_dead start 2024-04-25T19:18:53.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.598+0000 7f52ceffd640 1 --2- 172.21.15.76:0/2499217323 >> v2:172.21.15.105:3300/0 conn(0x7f52bc009fa0 0x7f52bc009920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:53.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.598+0000 7f52ceffd640 1 --2- 172.21.15.76:0/2499217323 >> v2:172.21.15.76:3300/0 conn(0x7f52bc00a3b0 0x7f52bc00a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:53.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.598+0000 7f52ceffd640 1 -- 172.21.15.76:0/2499217323 --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52bc00b0e0 con 0x7f52bc005ac0 2024-04-25T19:18:53.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.598+0000 7f52ceffd640 1 -- 172.21.15.76:0/2499217323 --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f52d8071ad0 con 0x7f52bc00a3b0 2024-04-25T19:18:53.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.598+0000 7f52ceffd640 1 -- 172.21.15.76:0/2499217323 --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f52d8071d70 con 0x7f52bc009fa0 2024-04-25T19:18:53.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.599+0000 7f52dc873640 1 --2- 172.21.15.76:0/2499217323 >> v2:172.21.15.76:3300/0 conn(0x7f52bc00a3b0 0x7f52bc00a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:53.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.599+0000 7f52dd074640 1 -- 172.21.15.76:0/2499217323 >> v2:172.21.15.105:3300/0 conn(0x7f52bc009fa0 msgr2=0x7f52bc009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-04-25T19:18:53.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.599+0000 7f52dd074640 1 --2- 172.21.15.76:0/2499217323 >> v2:172.21.15.105:3300/0 conn(0x7f52bc009fa0 0x7f52bc009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:18:53.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.599+0000 7f52dc873640 1 -- 172.21.15.76:0/2499217323 >> v2:172.21.15.105:3300/0 conn(0x7f52bc009fa0 msgr2=0x7f52bc009920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:18:53.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.599+0000 7f52dc873640 1 --2- 172.21.15.76:0/2499217323 >> v2:172.21.15.105:3300/0 conn(0x7f52bc009fa0 0x7f52bc009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:53.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.599+0000 7f52dc873640 1 -- 172.21.15.76:0/2499217323 >> v1:172.21.15.76:6789/0 conn(0x7f52bc005ac0 legacy=0x7f52bc0062a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:53.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.599+0000 7f52dc873640 1 -- 172.21.15.76:0/2499217323 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d8074150 con 0x7f52bc00a3b0 2024-04-25T19:18:53.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.600+0000 7f52dc873640 1 --2- 172.21.15.76:0/2499217323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52bc00a3b0 0x7f52bc00a890 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f52d806d620 tx=0x7f52c802a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5c59311457928f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:53.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.600+0000 7f52cffff640 1 -- 172.21.15.76:0/2499217323 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c8004aa0 con 0x7f52bc00a3b0 2024-04-25T19:18:53.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.600+0000 7f52cffff640 1 -- 172.21.15.76:0/2499217323 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c802e070 con 0x7f52bc00a3b0 2024-04-25T19:18:53.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.600+0000 7f52cffff640 1 -- 172.21.15.76:0/2499217323 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c8004e60 con 0x7f52bc00a3b0 2024-04-25T19:18:53.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.601+0000 7f52df2d8640 1 -- 172.21.15.76:0/2499217323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52bc00a3b0 msgr2=0x7f52bc00a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:53.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.601+0000 7f52df2d8640 1 --2- 172.21.15.76:0/2499217323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52bc00a3b0 0x7f52bc00a890 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f52d806d620 tx=0x7f52c802a440 comp rx=0 tx=0).stop 2024-04-25T19:18:53.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.601+0000 7f52df2d8640 1 -- 172.21.15.76:0/2499217323 shutdown_connections 2024-04-25T19:18:53.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.601+0000 7f52df2d8640 1 --2- 172.21.15.76:0/2499217323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52bc00a3b0 0x7f52bc00a890 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:53.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.601+0000 7f52df2d8640 1 --2- 172.21.15.76:0/2499217323 >> v2:172.21.15.105:3300/0 conn(0x7f52bc009fa0 0x7f52bc009920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:53.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.601+0000 7f52df2d8640 1 -- 172.21.15.76:0/2499217323 >> 172.21.15.76:0/2499217323 conn(0x7f52d80cecc0 msgr2=0x7f52d80cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:53.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.602+0000 7f52df2d8640 1 -- 172.21.15.76:0/2499217323 shutdown_connections 2024-04-25T19:18:53.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.602+0000 7f52df2d8640 1 -- 172.21.15.76:0/2499217323 wait complete. 2024-04-25T19:18:53.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.604+0000 7f52df2d8640 1 Processor -- start 2024-04-25T19:18:53.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.604+0000 7f52df2d8640 1 -- start start 2024-04-25T19:18:53.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.606+0000 7f52df2d8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52bc005ac0 0x7f52d815a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:53.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.606+0000 7f52df2d8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52bc009fa0 0x7f52d81f0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:53.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.606+0000 7f52dd074640 1 -- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52bc005ac0 msgr2=0x7f52d815a290 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3300/0 2024-04-25T19:18:53.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.606+0000 7f52dd074640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52bc005ac0 0x7f52d815a290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:18:53.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.606+0000 7f52df2d8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52bc00a3b0 0x7f52d806c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:53.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.606+0000 7f52df2d8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f52d80749f0 con 0x7f52bc009fa0 2024-04-25T19:18:53.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.606+0000 7f52df2d8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f52d8074710 con 0x7f52bc005ac0 2024-04-25T19:18:53.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.606+0000 7f52df2d8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f52d8154fc0 con 0x7f52bc00a3b0 2024-04-25T19:18:53.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.606+0000 7f52dc873640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52bc009fa0 0x7f52d81f0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:53.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.606+0000 7f52dc873640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52bc009fa0 0x7f52d81f0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52752/0 (socket says 172.21.15.76:52752) 2024-04-25T19:18:53.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.606+0000 7f52dc873640 1 -- 172.21.15.76:0/2248826613 learned_addr learned my addr 172.21.15.76:0/2248826613 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:53.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.606+0000 7f52dd875640 1 --2- 172.21.15.76:0/2248826613 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52bc00a3b0 0x7f52d806c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:53.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.607+0000 7f52dc873640 1 -- 172.21.15.76:0/2248826613 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52bc00a3b0 msgr2=0x7f52d806c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:53.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.607+0000 7f52dc873640 1 --2- 172.21.15.76:0/2248826613 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52bc00a3b0 0x7f52d806c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:53.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.607+0000 7f52dc873640 1 -- 172.21.15.76:0/2248826613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52bc005ac0 msgr2=0x7f52d815a290 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:18:53.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.607+0000 7f52dc873640 1 --2- 172.21.15.76:0/2248826613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52bc005ac0 0x7f52d815a290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:53.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.607+0000 7f52dc873640 1 -- 172.21.15.76:0/2248826613 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d81f0fd0 con 0x7f52bc009fa0 2024-04-25T19:18:53.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.607+0000 7f52dc873640 1 --2- 172.21.15.76:0/2248826613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52bc009fa0 0x7f52d81f0a90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f52c8000fb0 tx=0x7f52c80014f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:53.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.607+0000 7f52cdffb640 1 -- 172.21.15.76:0/2248826613 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c8007270 con 0x7f52bc009fa0 2024-04-25T19:18:53.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.607+0000 7f52cdffb640 1 -- 172.21.15.76:0/2248826613 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c8007410 con 0x7f52bc009fa0 2024-04-25T19:18:53.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.607+0000 7f52df2d8640 1 -- 172.21.15.76:0/2248826613 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d815ec70 con 0x7f52bc009fa0 2024-04-25T19:18:53.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.607+0000 7f52cdffb640 1 -- 172.21.15.76:0/2248826613 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c80077c0 con 0x7f52bc009fa0 2024-04-25T19:18:53.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.607+0000 7f52df2d8640 1 -- 172.21.15.76:0/2248826613 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d8158d30 con 0x7f52bc009fa0 2024-04-25T19:18:53.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.609+0000 7f52df2d8640 1 -- 172.21.15.76:0/2248826613 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52d8159530 con 0x7f52bc009fa0 2024-04-25T19:18:53.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.609+0000 7f52cdffb640 1 -- 172.21.15.76:0/2248826613 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f52c8030040 con 0x7f52bc009fa0 2024-04-25T19:18:53.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.610+0000 7f52cdffb640 1 --2- 172.21.15.76:0/2248826613 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f52bc0761e0 0x7f52bc0786f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:53.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.610+0000 7f52cdffb640 1 -- 172.21.15.76:0/2248826613 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f52c80b4730 con 0x7f52bc009fa0 2024-04-25T19:18:53.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.610+0000 7f52dd074640 1 --2- 172.21.15.76:0/2248826613 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f52bc0761e0 0x7f52bc0786f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:53.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.611+0000 7f52dd074640 1 --2- 172.21.15.76:0/2248826613 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f52bc0761e0 0x7f52bc0786f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f52bc007400 tx=0x7f52d4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:53.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:53.615+0000 7f52cdffb640 1 -- 172.21.15.76:0/2248826613 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f52d8159530 con 0x7f52bc009fa0 2024-04-25T19:18:54.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.029+0000 7f52df2d8640 1 -- 172.21.15.76:0/2248826613 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f52d806a390 con 0x7f52bc009fa0 2024-04-25T19:18:54.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.030+0000 7f52cdffb640 1 -- 172.21.15.76:0/2248826613 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f52d8151890 con 0x7f52bc009fa0 2024-04-25T19:18:54.031 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":28,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:18:54.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.032+0000 7f52df2d8640 1 -- 172.21.15.76:0/2248826613 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f52bc0761e0 msgr2=0x7f52bc0786f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:54.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.032+0000 7f52df2d8640 1 --2- 172.21.15.76:0/2248826613 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f52bc0761e0 0x7f52bc0786f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f52bc007400 tx=0x7f52d4008040 comp rx=0 tx=0).stop 2024-04-25T19:18:54.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.032+0000 7f52df2d8640 1 -- 172.21.15.76:0/2248826613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52bc009fa0 msgr2=0x7f52d81f0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:54.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.032+0000 7f52df2d8640 1 --2- 172.21.15.76:0/2248826613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52bc009fa0 0x7f52d81f0a90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f52c8000fb0 tx=0x7f52c80014f0 comp rx=0 tx=0).stop 2024-04-25T19:18:54.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.032+0000 7f52df2d8640 1 -- 172.21.15.76:0/2248826613 shutdown_connections 2024-04-25T19:18:54.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.032+0000 7f52df2d8640 1 --2- 172.21.15.76:0/2248826613 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f52bc0761e0 0x7f52bc0786f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:54.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.032+0000 7f52df2d8640 1 --2- 172.21.15.76:0/2248826613 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f52bc00a3b0 0x7f52d806c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:54.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.032+0000 7f52df2d8640 1 --2- 172.21.15.76:0/2248826613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52bc009fa0 0x7f52d81f0a90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:54.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.032+0000 7f52df2d8640 1 --2- 172.21.15.76:0/2248826613 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f52bc005ac0 0x7f52d815a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:54.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.033+0000 7f52df2d8640 1 -- 172.21.15.76:0/2248826613 >> 172.21.15.76:0/2248826613 conn(0x7f52d80cecc0 msgr2=0x7f52d814e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:54.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.033+0000 7f52df2d8640 1 -- 172.21.15.76:0/2248826613 shutdown_connections 2024-04-25T19:18:54.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.033+0000 7f52df2d8640 1 -- 172.21.15.76:0/2248826613 wait complete. 2024-04-25T19:18:54.056 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-04-25T19:18:54.056 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-04-25T19:18:54.056 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T19:18:54.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.227+0000 7f006b7d6640 1 Processor -- start 2024-04-25T19:18:54.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.227+0000 7f006b7d6640 1 -- start start 2024-04-25T19:18:54.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.229+0000 7f006b7d6640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0064154ee0 0x7f006415a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:54.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.229+0000 7f006b7d6640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f00640c98b0 0x7f006415df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:54.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.230+0000 7f006b7d6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f00640684e0 0x7f0064074130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:54.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.230+0000 7f006b7d6640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f00640c8e60 con 0x7f00640684e0 2024-04-25T19:18:54.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.230+0000 7f006b7d6640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f006415e470 con 0x7f00640c98b0 2024-04-25T19:18:54.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.230+0000 7f006b7d6640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f00640c9cc0 con 0x7f0064154ee0 2024-04-25T19:18:54.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.230+0000 7f0068d71640 1 -- >> v2:172.21.15.105:3300/0 conn(0x7f00640c98b0 msgr2=0x7f006415df30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.105:3300/0 2024-04-25T19:18:54.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.230+0000 7f0068d71640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f00640c98b0 0x7f006415df30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:18:54.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.230+0000 7f0069d73640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f00640684e0 0x7f0064074130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:54.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.230+0000 7f0069572640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0064154ee0 0x7f006415a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:54.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.230+0000 7f0069d73640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f00640684e0 0x7f0064074130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52766/0 (socket says 172.21.15.76:52766) 2024-04-25T19:18:54.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.230+0000 7f0069572640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0064154ee0 0x7f006415a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:38502/0 (socket says 172.21.15.76:38502) 2024-04-25T19:18:54.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.230+0000 7f0069d73640 1 -- 172.21.15.76:0/490437462 learned_addr learned my addr 172.21.15.76:0/490437462 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:54.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.231+0000 7f0069d73640 1 -- 172.21.15.76:0/490437462 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0064154ee0 msgr2=0x7f006415a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:54.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.231+0000 7f0069d73640 1 --2- 172.21.15.76:0/490437462 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0064154ee0 0x7f006415a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:54.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.231+0000 7f0069d73640 1 -- 172.21.15.76:0/490437462 >> v2:172.21.15.105:3300/0 conn(0x7f00640c98b0 msgr2=0x7f006415df30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:18:54.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.231+0000 7f0069d73640 1 --2- 172.21.15.76:0/490437462 >> v2:172.21.15.105:3300/0 conn(0x7f00640c98b0 0x7f006415df30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:54.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.231+0000 7f0069d73640 1 -- 172.21.15.76:0/490437462 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006415e6c0 con 0x7f00640684e0 2024-04-25T19:18:54.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.232+0000 7f0069d73640 1 --2- 172.21.15.76:0/490437462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00640684e0 0x7f0064074130 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0064074bc0 tx=0x7f005c02a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac046e727dcf423c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:54.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.232+0000 7f005bfff640 1 -- 172.21.15.76:0/490437462 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c036070 con 0x7f00640684e0 2024-04-25T19:18:54.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.232+0000 7f005bfff640 1 -- 172.21.15.76:0/490437462 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f005c031600 con 0x7f00640684e0 2024-04-25T19:18:54.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.232+0000 7f005bfff640 1 -- 172.21.15.76:0/490437462 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c0319d0 con 0x7f00640684e0 2024-04-25T19:18:54.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.233+0000 7f006b7d6640 1 -- 172.21.15.76:0/490437462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00640684e0 msgr2=0x7f0064074130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:54.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.233+0000 7f006b7d6640 1 --2- 172.21.15.76:0/490437462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00640684e0 0x7f0064074130 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0064074bc0 tx=0x7f005c02a740 comp rx=0 tx=0).stop 2024-04-25T19:18:54.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.233+0000 7f006b7d6640 1 -- 172.21.15.76:0/490437462 shutdown_connections 2024-04-25T19:18:54.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.233+0000 7f006b7d6640 1 --2- 172.21.15.76:0/490437462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00640684e0 0x7f0064074130 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:54.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.233+0000 7f006b7d6640 1 --2- 172.21.15.76:0/490437462 >> v2:172.21.15.105:3300/0 conn(0x7f00640c98b0 0x7f006415df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:54.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.233+0000 7f006b7d6640 1 --2- 172.21.15.76:0/490437462 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0064154ee0 0x7f006415a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:54.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.233+0000 7f006b7d6640 1 -- 172.21.15.76:0/490437462 >> 172.21.15.76:0/490437462 conn(0x7f006406c5c0 msgr2=0x7f006406c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:54.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.233+0000 7f006b7d6640 1 -- 172.21.15.76:0/490437462 shutdown_connections 2024-04-25T19:18:54.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.233+0000 7f006b7d6640 1 -- 172.21.15.76:0/490437462 wait complete. 2024-04-25T19:18:54.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.236+0000 7f006b7d6640 1 Processor -- start 2024-04-25T19:18:54.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.236+0000 7f006b7d6640 1 -- start start 2024-04-25T19:18:54.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.238+0000 7f006b7d6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00640684e0 0x7f006415d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:54.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.238+0000 7f006b7d6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00640c98b0 0x7f00640c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:54.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.238+0000 7f006b7d6640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0064154ee0 0x7f00641598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:54.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.238+0000 7f006b7d6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f00640c90b0 con 0x7f00640684e0 2024-04-25T19:18:54.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.238+0000 7f006b7d6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f006415ec80 con 0x7f00640c98b0 2024-04-25T19:18:54.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.238+0000 7f006b7d6640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f00640c9390 con 0x7f0064154ee0 2024-04-25T19:18:54.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.238+0000 7f0069572640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00640684e0 0x7f006415d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:54.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.238+0000 7f0069572640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00640684e0 0x7f006415d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52770/0 (socket says 172.21.15.76:52770) 2024-04-25T19:18:54.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.238+0000 7f0069572640 1 -- 172.21.15.76:0/2055984606 learned_addr learned my addr 172.21.15.76:0/2055984606 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:54.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.238+0000 7f0068d71640 1 -- 172.21.15.76:0/2055984606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00640c98b0 msgr2=0x7f00640c47c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3300/0 2024-04-25T19:18:54.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.238+0000 7f0068d71640 1 --2- 172.21.15.76:0/2055984606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00640c98b0 0x7f00640c47c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:18:54.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.238+0000 7f0069d73640 1 --2- 172.21.15.76:0/2055984606 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0064154ee0 0x7f00641598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:54.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.239+0000 7f0069572640 1 -- 172.21.15.76:0/2055984606 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0064154ee0 msgr2=0x7f00641598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:54.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.239+0000 7f0069572640 1 --2- 172.21.15.76:0/2055984606 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0064154ee0 0x7f00641598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:54.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.239+0000 7f0069572640 1 -- 172.21.15.76:0/2055984606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00640c98b0 msgr2=0x7f00640c47c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:18:54.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.239+0000 7f0069572640 1 --2- 172.21.15.76:0/2055984606 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00640c98b0 0x7f00640c47c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:54.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.239+0000 7f0069572640 1 -- 172.21.15.76:0/2055984606 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006415d670 con 0x7f00640684e0 2024-04-25T19:18:54.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.239+0000 7f0069572640 1 --2- 172.21.15.76:0/2055984606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00640684e0 0x7f006415d130 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f006000e8a0 tx=0x7f006000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:54.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.239+0000 7f0059ffb640 1 -- 172.21.15.76:0/2055984606 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060017070 con 0x7f00640684e0 2024-04-25T19:18:54.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.239+0000 7f006b7d6640 1 -- 172.21.15.76:0/2055984606 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f006415c790 con 0x7f00640684e0 2024-04-25T19:18:54.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.239+0000 7f0059ffb640 1 -- 172.21.15.76:0/2055984606 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f006000c940 con 0x7f00640684e0 2024-04-25T19:18:54.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.239+0000 7f0059ffb640 1 -- 172.21.15.76:0/2055984606 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006000cd30 con 0x7f00640684e0 2024-04-25T19:18:54.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.240+0000 7f006b7d6640 1 -- 172.21.15.76:0/2055984606 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00641f83c0 con 0x7f00640684e0 2024-04-25T19:18:54.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.241+0000 7f006b7d6640 1 --2- 172.21.15.76:0/2055984606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00640c4d00 0x7f0064159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:54.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.241+0000 7f006b7d6640 1 -- 172.21.15.76:0/2055984606 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f006415c540 con 0x7f00640c4d00 2024-04-25T19:18:54.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.241+0000 7f0068d71640 1 --2- 172.21.15.76:0/2055984606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00640c4d00 0x7f0064159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:54.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.242+0000 7f0068d71640 1 -- 172.21.15.76:0/2055984606 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f006406d6a0 con 0x7f00640c4d00 2024-04-25T19:18:54.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.242+0000 7f0059ffb640 1 -- 172.21.15.76:0/2055984606 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f006001a070 con 0x7f00640684e0 2024-04-25T19:18:54.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.242+0000 7f0068d71640 1 --2- 172.21.15.76:0/2055984606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00640c4d00 0x7f0064159df0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f00640c5fa0 tx=0x7f005400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:54.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.242+0000 7f0059ffb640 1 --2- 172.21.15.76:0/2055984606 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0048074460 0x7f0048076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:54.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.243+0000 7f0059ffb640 1 -- 172.21.15.76:0/2055984606 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f0060095530 con 0x7f00640684e0 2024-04-25T19:18:54.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.243+0000 7f0069d73640 1 --2- 172.21.15.76:0/2055984606 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0048074460 0x7f0048076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:54.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.243+0000 7f0059ffb640 1 -- 172.21.15.76:0/2055984606 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054003e10 con 0x7f00640c4d00 2024-04-25T19:18:54.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.243+0000 7f0059ffb640 1 -- 172.21.15.76:0/2055984606 <== mon.0 v2:172.21.15.76:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0054015070 con 0x7f00640c4d00 2024-04-25T19:18:54.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.243+0000 7f0059ffb640 1 -- 172.21.15.76:0/2055984606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00640c4d00 msgr2=0x7f0064159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:54.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.243+0000 7f0059ffb640 1 --2- 172.21.15.76:0/2055984606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00640c4d00 0x7f0064159df0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f00640c5fa0 tx=0x7f005400b8c0 comp rx=0 tx=0).stop 2024-04-25T19:18:54.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.243+0000 7f0059ffb640 1 -- 172.21.15.76:0/2055984606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00640c4d00 msgr2=0x7f0064159df0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:18:54.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.243+0000 7f0059ffb640 1 --2- 172.21.15.76:0/2055984606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00640c4d00 0x7f0064159df0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:54.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.243+0000 7f0069d73640 1 --2- 172.21.15.76:0/2055984606 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0048074460 0x7f0048076970 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f005c033000 tx=0x7f005c009000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:54.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.254+0000 7f006b7d6640 1 --2- 172.21.15.76:0/2055984606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00641505e0 0x7f00641faba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:54.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.254+0000 7f006b7d6640 1 -- 172.21.15.76:0/2055984606 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f00641f85c0 con 0x7f00641505e0 2024-04-25T19:18:54.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.254+0000 7f0068d71640 1 --2- 172.21.15.76:0/2055984606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00641505e0 0x7f00641faba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:54.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.254+0000 7f0068d71640 1 -- 172.21.15.76:0/2055984606 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f006406a390 con 0x7f00641505e0 2024-04-25T19:18:54.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.255+0000 7f0068d71640 1 --2- 172.21.15.76:0/2055984606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00641505e0 0x7f00641faba0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f006415dc80 tx=0x7f0054005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:54.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.255+0000 7f0059ffb640 1 -- 172.21.15.76:0/2055984606 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054025440 con 0x7f00641505e0 2024-04-25T19:18:54.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.255+0000 7f0059ffb640 1 -- 172.21.15.76:0/2055984606 <== mon.0 v2:172.21.15.76:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4885 (secure 0 0 0) 0x7f0054025440 con 0x7f00641505e0 2024-04-25T19:18:54.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.255+0000 7f0059ffb640 1 -- 172.21.15.76:0/2055984606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00641505e0 msgr2=0x7f00641faba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:54.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.255+0000 7f0059ffb640 1 --2- 172.21.15.76:0/2055984606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00641505e0 0x7f00641faba0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f006415dc80 tx=0x7f0054005a20 comp rx=0 tx=0).stop 2024-04-25T19:18:54.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.255+0000 7f0059ffb640 1 -- 172.21.15.76:0/2055984606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00641505e0 msgr2=0x7f00641faba0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:18:54.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.255+0000 7f0059ffb640 1 --2- 172.21.15.76:0/2055984606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00641505e0 0x7f00641faba0 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f006415dc80 tx=0x7f0054005a20 comp rx=0 tx=0).stop 2024-04-25T19:18:54.258 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:18:54.258 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:18:54.258 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:18:54.258 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "leader", 2024-04-25T19:18:54.258 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 28, 2024-04-25T19:18:54.258 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 33367, 2024-04-25T19:18:54.258 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:18:54.258 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:18:54.259 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:18:54.259 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:54.259 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 2, 2024-04-25T19:18:54.259 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:18:54.259 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:18:54.259 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:18:54.259 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:54.259 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:54.259 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:54.259 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:54.260 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:54.260 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:54.260 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:54.260 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:54.260 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:54.260 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:54.260 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:54.260 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:54.260 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:18:54.260 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:18:54.261 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:54.261 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:54.261 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:54.261 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:54.261 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:54.261 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:54.261 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:54.261 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:54.261 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:54.261 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:54.262 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:54.262 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:54.262 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:54.262 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:18:54.262 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:18:54.262 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:18:54.262 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:18:54.262 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:18:54.262 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:18:54.263 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:18:54.263 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:18:54.263 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:18:54.263 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:18:54.263 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:18:54.263 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:18:54.263 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:18:54.263 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:18:54.263 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:18:54.263 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:18:54.263 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:18:54.264 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:54.264 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:54.264 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:54.264 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:54.264 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:54.264 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:54.264 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:54.264 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:54.264 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:54.264 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:54.265 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:54.265 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:54.265 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:18:54.265 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:54.265 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:18:54.265 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.265 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:18:54.265 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:18:54.265 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:54.265 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:54.266 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.266 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:54.266 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:18:54.266 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:54.266 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:54.266 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.266 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:54.266 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:18:54.266 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:54.266 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:54.267 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:54.267 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:54.267 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:18:54.267 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:18:54.267 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:54.267 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:54.267 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:54.267 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:54.267 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.268 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:18:54.268 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:18:54.268 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:54.268 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:54.268 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.268 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:54.268 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:18:54.268 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:54.268 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:54.268 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.269 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:54.269 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:18:54.269 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:54.269 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:54.269 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:54.269 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:54.269 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:18:54.269 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:18:54.269 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:54.269 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:54.270 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:54.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:54.270 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.270 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:18:54.270 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:18:54.270 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:54.270 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:54.270 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.270 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:54.270 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:18:54.271 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:54.271 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:54.271 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.271 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:54.271 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:18:54.271 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:54.271 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:54.271 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:54.271 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:54.271 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:18:54.272 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:18:54.272 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:54.272 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:54.272 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:54.272 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:54.272 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:54.272 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:54.272 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:18:54.272 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:18:54.272 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.273 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:54.273 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:54.273 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:18:54.273 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:54.273 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:54.273 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": [ 2024-04-25T19:18:54.273 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.273 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:54.273 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:54.273 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 2 2024-04-25T19:18:54.274 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:54.274 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:54.274 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": [ 2024-04-25T19:18:54.274 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.274 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:54.274 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:54.274 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:18:54.274 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:54.274 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:54.275 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:18:54.275 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.275 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:54.275 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:54.275 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 2 2024-04-25T19:18:54.275 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:54.275 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:54.275 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:54.275 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:18:54.275 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:18:54.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.256+0000 7f006b7d6640 1 -- 172.21.15.76:0/2055984606 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0048074460 msgr2=0x7f0048076970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:54.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.256+0000 7f006b7d6640 1 --2- 172.21.15.76:0/2055984606 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0048074460 0x7f0048076970 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f005c033000 tx=0x7f005c009000 comp rx=0 tx=0).stop 2024-04-25T19:18:54.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.256+0000 7f006b7d6640 1 -- 172.21.15.76:0/2055984606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00640684e0 msgr2=0x7f006415d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:54.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.256+0000 7f006b7d6640 1 --2- 172.21.15.76:0/2055984606 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00640684e0 0x7f006415d130 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f006000e8a0 tx=0x7f006000ed60 comp rx=0 tx=0).stop 2024-04-25T19:18:54.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.256+0000 7f0069d73640 1 -- 172.21.15.76:0/2055984606 reap_dead start 2024-04-25T19:18:54.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.256+0000 7f0069d73640 1 -- 172.21.15.76:0/2055984606 reap_dead start 2024-04-25T19:18:54.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.256+0000 7f006b7d6640 1 -- 172.21.15.76:0/2055984606 shutdown_connections 2024-04-25T19:18:54.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.256+0000 7f006b7d6640 1 -- 172.21.15.76:0/2055984606 >> 172.21.15.76:0/2055984606 conn(0x7f006406c5c0 msgr2=0x7f006414cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:54.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.256+0000 7f006b7d6640 1 -- 172.21.15.76:0/2055984606 shutdown_connections 2024-04-25T19:18:54.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.256+0000 7f006b7d6640 1 -- 172.21.15.76:0/2055984606 wait complete. 2024-04-25T19:18:54.279 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T19:18:54.279 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-04-25T19:18:54.279 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T19:18:54.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.453+0000 7fc9084d8640 1 Processor -- start 2024-04-25T19:18:54.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.453+0000 7fc9084d8640 1 -- start start 2024-04-25T19:18:54.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.456+0000 7fc9084d8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc900160260 0x7fc90006cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:54.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.456+0000 7fc9084d8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc90006d2b0 0x7fc9000cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:54.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.456+0000 7fc9084d8640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9001609a0 con 0x7fc9001570d0 2024-04-25T19:18:54.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.456+0000 7fc906a75640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc90006d2b0 0x7fc9000cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:54.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.456+0000 7fc906274640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc9001570d0 0x7fc90006e040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:51316/0 (socket says 172.21.15.76:51316) 2024-04-25T19:18:54.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.456+0000 7fc906274640 1 -- 172.21.15.76:0/1171628003 learned_addr learned my addr 172.21.15.76:0/1171628003 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:18:54.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.456+0000 7fc905a73640 1 --2- 172.21.15.76:0/1171628003 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc900160260 0x7fc90006cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:54.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.456+0000 7fc9084d8640 1 -- 172.21.15.76:0/1171628003 --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc9001574e0 con 0x7fc90006d2b0 2024-04-25T19:18:54.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.456+0000 7fc9084d8640 1 -- 172.21.15.76:0/1171628003 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fc900160c80 con 0x7fc900160260 2024-04-25T19:18:54.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.457+0000 7fc905272640 1 -- 172.21.15.76:0/1171628003 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2741035083 0 0) 0x7fc9001609a0 con 0x7fc9001570d0 2024-04-25T19:18:54.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.457+0000 7fc905272640 1 -- 172.21.15.76:0/1171628003 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8ec003520 con 0x7fc9001570d0 2024-04-25T19:18:54.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.457+0000 7fc905a73640 1 -- 172.21.15.76:0/1171628003 >> v2:172.21.15.76:3300/0 conn(0x7fc90006d2b0 msgr2=0x7fc9000cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:18:54.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.457+0000 7fc905a73640 1 --2- 172.21.15.76:0/1171628003 >> v2:172.21.15.76:3300/0 conn(0x7fc90006d2b0 0x7fc9000cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:54.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.457+0000 7fc905a73640 1 -- 172.21.15.76:0/1171628003 >> v1:172.21.15.76:6789/0 conn(0x7fc9001570d0 legacy=0x7fc90006e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:54.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.457+0000 7fc905a73640 1 -- 172.21.15.76:0/1171628003 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc900160f60 con 0x7fc900160260 2024-04-25T19:18:54.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.458+0000 7fc906a75640 1 --2- 172.21.15.76:0/1171628003 >> v2:172.21.15.76:3300/0 conn(0x7fc90006d2b0 0x7fc9000cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:18:54.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.458+0000 7fc905a73640 1 --2- 172.21.15.76:0/1171628003 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc900160260 0x7fc90006cd70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc0097a0 tx=0x7fc8fc032a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c972e056041e622 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:54.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.459+0000 7fc905272640 1 -- 172.21.15.76:0/1171628003 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc036840 con 0x7fc900160260 2024-04-25T19:18:54.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.459+0000 7fc905272640 1 -- 172.21.15.76:0/1171628003 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8fc0369e0 con 0x7fc900160260 2024-04-25T19:18:54.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.459+0000 7fc905272640 1 -- 172.21.15.76:0/1171628003 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc036df0 con 0x7fc900160260 2024-04-25T19:18:54.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.460+0000 7fc9084d8640 1 -- 172.21.15.76:0/1171628003 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc900160260 msgr2=0x7fc90006cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:54.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.460+0000 7fc9084d8640 1 --2- 172.21.15.76:0/1171628003 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc900160260 0x7fc90006cd70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc0097a0 tx=0x7fc8fc032a80 comp rx=0 tx=0).stop 2024-04-25T19:18:54.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.461+0000 7fc9084d8640 1 -- 172.21.15.76:0/1171628003 shutdown_connections 2024-04-25T19:18:54.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.461+0000 7fc9084d8640 1 --2- 172.21.15.76:0/1171628003 >> v2:172.21.15.76:3300/0 conn(0x7fc90006d2b0 0x7fc9000cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:54.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.461+0000 7fc9084d8640 1 --2- 172.21.15.76:0/1171628003 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc900160260 0x7fc90006cd70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:54.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.461+0000 7fc9084d8640 1 -- 172.21.15.76:0/1171628003 >> 172.21.15.76:0/1171628003 conn(0x7fc9000d22d0 msgr2=0x7fc9000d26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:54.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.461+0000 7fc9084d8640 1 -- 172.21.15.76:0/1171628003 shutdown_connections 2024-04-25T19:18:54.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.461+0000 7fc9084d8640 1 -- 172.21.15.76:0/1171628003 wait complete. 2024-04-25T19:18:54.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.464+0000 7fc9084d8640 1 Processor -- start 2024-04-25T19:18:54.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.464+0000 7fc9084d8640 1 -- start start 2024-04-25T19:18:54.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.466+0000 7fc9084d8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc90006d2b0 0x7fc900159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:54.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.466+0000 7fc9084d8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9001570d0 0x7fc9000cad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:54.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.466+0000 7fc9084d8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc900160260 0x7fc9000cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:54.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.466+0000 7fc9084d8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc900160670 con 0x7fc900160260 2024-04-25T19:18:54.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.466+0000 7fc9084d8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc90006db20 con 0x7fc90006d2b0 2024-04-25T19:18:54.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.466+0000 7fc9084d8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fc9000d00a0 con 0x7fc9001570d0 2024-04-25T19:18:54.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.466+0000 7fc906274640 1 -- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc90006d2b0 msgr2=0x7fc900159570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.105:3300/0 2024-04-25T19:18:54.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.466+0000 7fc906274640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc90006d2b0 0x7fc900159570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:18:54.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.466+0000 7fc906a75640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc900160260 0x7fc9000cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:54.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.466+0000 7fc906a75640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc900160260 0x7fc9000cb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52812/0 (socket says 172.21.15.76:52812) 2024-04-25T19:18:54.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.466+0000 7fc906a75640 1 -- 172.21.15.76:0/881114919 learned_addr learned my addr 172.21.15.76:0/881114919 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:18:54.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.466+0000 7fc905a73640 1 --2- 172.21.15.76:0/881114919 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9001570d0 0x7fc9000cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:54.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.467+0000 7fc906a75640 1 -- 172.21.15.76:0/881114919 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9001570d0 msgr2=0x7fc9000cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:54.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.467+0000 7fc906a75640 1 --2- 172.21.15.76:0/881114919 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc9001570d0 0x7fc9000cad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:54.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.467+0000 7fc906a75640 1 -- 172.21.15.76:0/881114919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc90006d2b0 msgr2=0x7fc900159570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T19:18:54.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.467+0000 7fc906a75640 1 --2- 172.21.15.76:0/881114919 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc90006d2b0 0x7fc900159570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:54.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.467+0000 7fc906a75640 1 -- 172.21.15.76:0/881114919 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9001ee4d0 con 0x7fc900160260 2024-04-25T19:18:54.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.467+0000 7fc906a75640 1 --2- 172.21.15.76:0/881114919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc900160260 0x7fc9000cb260 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800cd10 tx=0x7fc8f800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:54.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.467+0000 7fc8f6ffd640 1 -- 172.21.15.76:0/881114919 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8003b90 con 0x7fc900160260 2024-04-25T19:18:54.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.467+0000 7fc8f6ffd640 1 -- 172.21.15.76:0/881114919 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8f8003d50 con 0x7fc900160260 2024-04-25T19:18:54.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.468+0000 7fc9084d8640 1 -- 172.21.15.76:0/881114919 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9001ee770 con 0x7fc900160260 2024-04-25T19:18:54.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.468+0000 7fc8f6ffd640 1 -- 172.21.15.76:0/881114919 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8004260 con 0x7fc900160260 2024-04-25T19:18:54.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.468+0000 7fc9084d8640 1 -- 172.21.15.76:0/881114919 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc90015a480 con 0x7fc900160260 2024-04-25T19:18:54.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.470+0000 7fc9084d8640 1 --2- 172.21.15.76:0/881114919 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc90015c060 0x7fc90015b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:54.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.470+0000 7fc9084d8640 1 -- 172.21.15.76:0/881114919 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fc9000cb7a0 con 0x7fc90015c060 2024-04-25T19:18:54.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.470+0000 7fc906274640 1 --2- 172.21.15.76:0/881114919 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc90015c060 0x7fc90015b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:54.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.471+0000 7fc906274640 1 -- 172.21.15.76:0/881114919 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc90015a1f0 con 0x7fc90015c060 2024-04-25T19:18:54.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.471+0000 7fc906274640 1 --2- 172.21.15.76:0/881114919 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc90015c060 0x7fc90015b070 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0004810 tx=0x7fc8f000c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:54.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.471+0000 7fc8f6ffd640 1 -- 172.21.15.76:0/881114919 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc8f8004400 con 0x7fc900160260 2024-04-25T19:18:54.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.472+0000 7fc8f6ffd640 1 --2- 172.21.15.76:0/881114919 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fc8e4074460 0x7fc8e4076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:54.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.472+0000 7fc8f6ffd640 1 -- 172.21.15.76:0/881114919 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fc8f8095d00 con 0x7fc900160260 2024-04-25T19:18:54.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.472+0000 7fc8f6ffd640 1 -- 172.21.15.76:0/881114919 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0017070 con 0x7fc90015c060 2024-04-25T19:18:54.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.472+0000 7fc8f6ffd640 1 -- 172.21.15.76:0/881114919 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc8f0005ce0 con 0x7fc90015c060 2024-04-25T19:18:54.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.472+0000 7fc8f6ffd640 1 -- 172.21.15.76:0/881114919 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc90015c060 msgr2=0x7fc90015b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:54.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.472+0000 7fc8f6ffd640 1 --2- 172.21.15.76:0/881114919 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc90015c060 0x7fc90015b070 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0004810 tx=0x7fc8f000c810 comp rx=0 tx=0).stop 2024-04-25T19:18:54.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.472+0000 7fc905a73640 1 --2- 172.21.15.76:0/881114919 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fc8e4074460 0x7fc8e4076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:54.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.472+0000 7fc8f6ffd640 1 -- 172.21.15.76:0/881114919 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc90015c060 msgr2=0x7fc90015b070 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:18:54.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.472+0000 7fc8f6ffd640 1 --2- 172.21.15.76:0/881114919 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc90015c060 0x7fc90015b070 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:54.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.473+0000 7fc905a73640 1 --2- 172.21.15.76:0/881114919 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fc8e4074460 0x7fc8e4076970 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc03d000 tx=0x7fc8fc008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:54.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.482+0000 7fc9084d8640 1 --2- 172.21.15.76:0/881114919 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc90015c470 0x7fc90015b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:18:54.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.482+0000 7fc9084d8640 1 -- 172.21.15.76:0/881114919 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fc9001591a0 con 0x7fc90015c470 2024-04-25T19:18:54.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.482+0000 7fc906274640 1 --2- 172.21.15.76:0/881114919 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc90015c470 0x7fc90015b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:18:54.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.483+0000 7fc906274640 1 -- 172.21.15.76:0/881114919 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc9000cc840 con 0x7fc90015c470 2024-04-25T19:18:54.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.484+0000 7fc906274640 1 --2- 172.21.15.76:0/881114919 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc90015c470 0x7fc90015b5b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc900076700 tx=0x7fc8f0000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:18:54.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.484+0000 7fc8f6ffd640 1 -- 172.21.15.76:0/881114919 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0017040 con 0x7fc90015c470 2024-04-25T19:18:54.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.484+0000 7fc8f6ffd640 1 -- 172.21.15.76:0/881114919 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5054 (secure 0 0 0) 0x7fc8f0012400 con 0x7fc90015c470 2024-04-25T19:18:54.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.484+0000 7fc8f6ffd640 1 -- 172.21.15.76:0/881114919 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc90015c470 msgr2=0x7fc90015b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:54.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.484+0000 7fc8f6ffd640 1 --2- 172.21.15.76:0/881114919 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc90015c470 0x7fc90015b5b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc900076700 tx=0x7fc8f0000ed0 comp rx=0 tx=0).stop 2024-04-25T19:18:54.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.484+0000 7fc8f6ffd640 1 -- 172.21.15.76:0/881114919 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc90015c470 msgr2=0x7fc90015b5b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:18:54.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.484+0000 7fc8f6ffd640 1 --2- 172.21.15.76:0/881114919 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fc90015c470 0x7fc90015b5b0 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc900076700 tx=0x7fc8f0000ed0 comp rx=0 tx=0).stop 2024-04-25T19:18:54.486 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:18:54.487 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:18:54.487 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:18:54.487 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "peon", 2024-04-25T19:18:54.487 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 28, 2024-04-25T19:18:54.487 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 138262, 2024-04-25T19:18:54.487 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:18:54.487 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:18:54.487 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:18:54.487 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:54.487 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 3, 2024-04-25T19:18:54.487 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:18:54.487 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:18:54.487 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:18:54.488 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:54.488 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:54.488 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:54.488 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:54.488 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:54.488 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:54.488 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:54.488 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:54.488 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:54.488 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:54.488 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:54.488 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:54.488 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:18:54.488 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:18:54.489 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:54.489 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:54.489 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:54.489 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:54.489 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:54.489 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:54.489 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:54.489 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:54.489 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:54.489 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:54.489 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:54.489 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:54.489 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:54.489 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:18:54.489 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:18:54.490 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:18:54.490 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:18:54.490 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:18:54.490 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:18:54.490 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:18:54.490 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:18:54.490 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:18:54.490 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:18:54.490 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:18:54.490 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:18:54.490 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:18:54.490 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:18:54.490 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:18:54.490 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:18:54.491 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:18:54.491 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:18:54.491 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:18:54.491 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:18:54.491 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:18:54.491 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:18:54.491 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:18:54.491 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:18:54.491 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:18:54.491 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:18:54.491 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:18:54.491 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:18:54.491 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:54.492 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:18:54.492 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:54.492 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:18:54.492 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.492 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:18:54.492 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:18:54.492 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:54.492 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:54.492 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.492 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:54.492 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:18:54.492 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:54.492 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:54.492 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.492 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:54.493 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:18:54.493 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:54.493 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:54.493 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:54.493 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:54.493 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:18:54.493 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:18:54.493 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:54.493 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:54.493 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:54.493 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:54.493 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.493 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:18:54.493 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:18:54.493 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:54.494 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:54.494 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.494 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:54.494 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:18:54.494 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:54.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:54.494 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.494 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:54.494 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:18:54.494 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:54.494 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:54.494 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:54.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:54.495 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:18:54.495 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:18:54.495 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:54.495 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:54.495 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:54.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:54.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.495 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:18:54.495 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:18:54.495 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:18:54.495 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:18:54.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.495 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:18:54.495 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:18:54.495 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:54.496 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:54.496 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.496 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:18:54.496 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:18:54.496 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:18:54.496 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:54.496 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:54.496 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:54.496 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:18:54.496 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:18:54.496 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:18:54.496 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:18:54.496 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:18:54.496 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:54.496 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:54.497 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:54.497 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:18:54.497 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:18:54.497 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.497 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:54.497 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:54.497 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:18:54.497 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:54.497 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:54.497 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": [ 2024-04-25T19:18:54.497 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.497 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:54.497 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:54.497 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 4 2024-04-25T19:18:54.498 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:54.498 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:54.498 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": [ 2024-04-25T19:18:54.498 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.498 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:54.498 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:54.498 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 7 2024-04-25T19:18:54.498 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:54.498 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:54.498 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:18:54.498 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.498 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:54.498 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:54.499 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 6 2024-04-25T19:18:54.499 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:54.499 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:18:54.499 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": [ 2024-04-25T19:18:54.499 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:18:54.499 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:18:54.499 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:18:54.499 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 2 2024-04-25T19:18:54.499 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:18:54.499 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:18:54.499 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:18:54.499 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:18:54.499 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:18:54.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.485+0000 7fc9084d8640 1 -- 172.21.15.76:0/881114919 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fc8e4074460 msgr2=0x7fc8e4076970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:54.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.485+0000 7fc9084d8640 1 --2- 172.21.15.76:0/881114919 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fc8e4074460 0x7fc8e4076970 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc03d000 tx=0x7fc8fc008030 comp rx=0 tx=0).stop 2024-04-25T19:18:54.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.485+0000 7fc9084d8640 1 -- 172.21.15.76:0/881114919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc900160260 msgr2=0x7fc9000cb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:18:54.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.485+0000 7fc906a75640 1 -- 172.21.15.76:0/881114919 reap_dead start 2024-04-25T19:18:54.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.485+0000 7fc9084d8640 1 --2- 172.21.15.76:0/881114919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc900160260 0x7fc9000cb260 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800cd10 tx=0x7fc8f800a650 comp rx=0 tx=0).stop 2024-04-25T19:18:54.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.485+0000 7fc9084d8640 1 -- 172.21.15.76:0/881114919 shutdown_connections 2024-04-25T19:18:54.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.485+0000 7fc9084d8640 1 --2- 172.21.15.76:0/881114919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc900160260 0x7fc9000cb260 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:18:54.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.485+0000 7fc9084d8640 1 -- 172.21.15.76:0/881114919 >> 172.21.15.76:0/881114919 conn(0x7fc9000d22d0 msgr2=0x7fc9000b6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:18:54.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.485+0000 7fc9084d8640 1 -- 172.21.15.76:0/881114919 shutdown_connections 2024-04-25T19:18:54.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:18:54.485+0000 7fc9084d8640 1 -- 172.21.15.76:0/881114919 wait complete. 2024-04-25T19:18:54.506 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-04-25T19:18:54.506 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-04-25T19:18:55.611 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:18:55.611 INFO:tasks.workunit.client.0.smithi076.stdout: Children see throughput for 1 rewriters = 833959.81 kB/sec 2024-04-25T19:18:55.611 INFO:tasks.workunit.client.0.smithi076.stdout: Parent sees throughput for 1 rewriters = 833942.40 kB/sec 2024-04-25T19:18:55.612 INFO:tasks.workunit.client.0.smithi076.stdout: Min throughput per process = 833959.81 kB/sec 2024-04-25T19:18:55.612 INFO:tasks.workunit.client.0.smithi076.stdout: Max throughput per process = 833959.81 kB/sec 2024-04-25T19:18:55.612 INFO:tasks.workunit.client.0.smithi076.stdout: Avg throughput per process = 833959.81 kB/sec 2024-04-25T19:18:55.612 INFO:tasks.workunit.client.0.smithi076.stdout: Min xfer = 10485760.00 kB 2024-04-25T19:18:56.676 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:18:56.679 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:19:03.658 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:19:03.659 INFO:tasks.workunit.client.0.smithi076.stdout: Children see throughput for 1 readers = 7942788.00 kB/sec 2024-04-25T19:19:03.659 INFO:tasks.workunit.client.0.smithi076.stdout: Parent sees throughput for 1 readers = 7941272.52 kB/sec 2024-04-25T19:19:03.659 INFO:tasks.workunit.client.0.smithi076.stdout: Min throughput per process = 7942788.00 kB/sec 2024-04-25T19:19:03.659 INFO:tasks.workunit.client.0.smithi076.stdout: Max throughput per process = 7942788.00 kB/sec 2024-04-25T19:19:03.659 INFO:tasks.workunit.client.0.smithi076.stdout: Avg throughput per process = 7942788.00 kB/sec 2024-04-25T19:19:03.659 INFO:tasks.workunit.client.0.smithi076.stdout: Min xfer = 10485760.00 kB 2024-04-25T19:19:07.078 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:19:07.079 INFO:tasks.workunit.client.0.smithi076.stdout: Children see throughput for 1 re-readers = 7957266.50 kB/sec 2024-04-25T19:19:07.079 INFO:tasks.workunit.client.0.smithi076.stdout: Parent sees throughput for 1 re-readers = 7956053.43 kB/sec 2024-04-25T19:19:07.079 INFO:tasks.workunit.client.0.smithi076.stdout: Min throughput per process = 7957266.50 kB/sec 2024-04-25T19:19:07.079 INFO:tasks.workunit.client.0.smithi076.stdout: Max throughput per process = 7957266.50 kB/sec 2024-04-25T19:19:07.079 INFO:tasks.workunit.client.0.smithi076.stdout: Avg throughput per process = 7957266.50 kB/sec 2024-04-25T19:19:07.079 INFO:tasks.workunit.client.0.smithi076.stdout: Min xfer = 10485760.00 kB 2024-04-25T19:19:14.016 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:19:14.016 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:19:14.016 INFO:tasks.workunit.client.0.smithi076.stdout: 2024-04-25T19:19:14.016 INFO:tasks.workunit.client.0.smithi076.stdout:iozone test complete. 2024-04-25T19:19:14.017 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T19:19:14.017 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T19:19:14.053 INFO:tasks.workunit:Stopping ['suites/iozone.sh'] on client.0... 2024-04-25T19:19:14.053 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T19:19:14.361 DEBUG:teuthology.parallel:result is None 2024-04-25T19:19:14.362 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T19:19:14.394 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T19:19:14.394 DEBUG:teuthology.run_tasks:Unwinding manager mon_thrash 2024-04-25T19:19:14.406 INFO:tasks.mon_thrash:joining mon_thrasher 2024-04-25T19:19:14.507 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-04-25T19:19:14.507 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-04-25T19:19:14.507 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T19:19:14.507 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T19:19:14.510 INFO:tasks.ceph.mon.b:Started 2024-04-25T19:19:14.510 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T19:19:14.510 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:19:14.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.618+0000 7f9301586640 1 Processor -- start 2024-04-25T19:19:14.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.618+0000 7f9301586640 1 -- start start 2024-04-25T19:19:14.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.619+0000 7f9301586640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f92fc0cea70 0x7f92fc0cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:14.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.619+0000 7f9301586640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92fc15ce80 con 0x7f92fc0d0810 2024-04-25T19:19:14.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.619+0000 7f9301586640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92fc15d160 con 0x7f92fc06e160 2024-04-25T19:19:14.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.619+0000 7f9301586640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f92fc15d440 con 0x7f92fc0cea70 2024-04-25T19:19:14.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.619+0000 7f92faffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f92fc0d0810 0x7f92fc06f000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47944/0 (socket says 172.21.15.76:47944) 2024-04-25T19:19:14.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.619+0000 7f92faffd640 1 -- 172.21.15.76:0/2247552800 learned_addr learned my addr 172.21.15.76:0/2247552800 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:14.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.620+0000 7f92f3fff640 1 --2- 172.21.15.76:0/2247552800 >> v2:172.21.15.105:3300/0 conn(0x7f92fc0cea70 0x7f92fc0cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:14.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.620+0000 7f92fa7fc640 1 -- 172.21.15.76:0/2247552800 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1080257905 0 0) 0x7f92fc15ce80 con 0x7f92fc0d0810 2024-04-25T19:19:14.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.620+0000 7f92fa7fc640 1 -- 172.21.15.76:0/2247552800 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92e8003520 con 0x7f92fc0d0810 2024-04-25T19:19:14.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.620+0000 7f92f3fff640 1 -- 172.21.15.76:0/2247552800 >> v2:172.21.15.105:3300/0 conn(0x7f92fc0cea70 msgr2=0x7f92fc0cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T19:19:14.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.620+0000 7f92f3fff640 1 -- 172.21.15.76:0/2247552800 >> v2:172.21.15.105:3300/0 conn(0x7f92fc0cea70 msgr2=0x7f92fc0cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T19:19:14.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.620+0000 7f92f3fff640 1 --2- 172.21.15.76:0/2247552800 >> v2:172.21.15.105:3300/0 conn(0x7f92fc0cea70 0x7f92fc0cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T19:19:14.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.620+0000 7f92f3fff640 1 --2- 172.21.15.76:0/2247552800 >> v2:172.21.15.105:3300/0 conn(0x7f92fc0cea70 0x7f92fc0cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T19:19:14.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.620+0000 7f92fa7fc640 1 -- 172.21.15.76:0/2247552800 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4194542162 0 0) 0x7f92e8003520 con 0x7f92fc0d0810 2024-04-25T19:19:14.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.620+0000 7f92fa7fc640 1 -- 172.21.15.76:0/2247552800 >> v2:172.21.15.105:3300/0 conn(0x7f92fc0cea70 msgr2=0x7f92fc0cee80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:19:14.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.620+0000 7f92fa7fc640 1 --2- 172.21.15.76:0/2247552800 >> v2:172.21.15.105:3300/0 conn(0x7f92fc0cea70 0x7f92fc0cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.620+0000 7f92fa7fc640 1 -- 172.21.15.76:0/2247552800 >> v1:172.21.15.105:6789/0 conn(0x7f92fc06e160 legacy=0x7f92fc06e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:14.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.620+0000 7f92fa7fc640 1 -- 172.21.15.76:0/2247552800 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92fc15d720 con 0x7f92fc0d0810 2024-04-25T19:19:14.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.620+0000 7f92fa7fc640 1 -- 172.21.15.76:0/2247552800 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f92ec001670 con 0x7f92fc0d0810 2024-04-25T19:19:14.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.620+0000 7f92fa7fc640 1 -- 172.21.15.76:0/2247552800 >> v1:172.21.15.76:6789/0 conn(0x7f92fc0d0810 legacy=0x7f92fc06f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:14.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.620+0000 7f92fa7fc640 1 --2- 172.21.15.76:0/2247552800 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92e80039b0 0x7f92e8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:14.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.620+0000 7f92fa7fc640 1 --2- 172.21.15.76:0/2247552800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92e8004800 0x7f92e8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:14.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.620+0000 7f92fa7fc640 1 --2- 172.21.15.76:0/2247552800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92e8005270 0x7f92e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:14.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.620+0000 7f92fa7fc640 1 -- 172.21.15.76:0/2247552800 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc15d440 con 0x7f92e8005270 2024-04-25T19:19:14.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.620+0000 7f92fa7fc640 1 -- 172.21.15.76:0/2247552800 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f92e800c200 con 0x7f92e8004800 2024-04-25T19:19:14.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.620+0000 7f92fa7fc640 1 -- 172.21.15.76:0/2247552800 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f92e800c400 con 0x7f92e80039b0 2024-04-25T19:19:14.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.620+0000 7f92fb7fe640 1 --2- 172.21.15.76:0/2247552800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92e8005270 0x7f92e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:14.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.620+0000 7f92faffd640 1 --2- 172.21.15.76:0/2247552800 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92e80039b0 0x7f92e8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:14.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.620+0000 7f92f3fff640 1 --2- 172.21.15.76:0/2247552800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92e8004800 0x7f92e8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:14.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.620+0000 7f92fb7fe640 1 -- 172.21.15.76:0/2247552800 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92e80039b0 msgr2=0x7f92e8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:14.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.620+0000 7f92fb7fe640 1 --2- 172.21.15.76:0/2247552800 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92e80039b0 0x7f92e8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.621+0000 7f92fb7fe640 1 -- 172.21.15.76:0/2247552800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92e8004800 msgr2=0x7f92e8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:14.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.621+0000 7f92fb7fe640 1 --2- 172.21.15.76:0/2247552800 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92e8004800 0x7f92e8004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.621+0000 7f92fb7fe640 1 -- 172.21.15.76:0/2247552800 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e8003520 con 0x7f92e8005270 2024-04-25T19:19:14.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.621+0000 7f92fb7fe640 1 -- 172.21.15.76:0/2247552800 reap_dead start 2024-04-25T19:19:14.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.621+0000 7f92fb7fe640 1 --2- 172.21.15.76:0/2247552800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92e8005270 0x7f92e800b9c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f92f4001e50 tx=0x7f92e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94efccb343136ab4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:14.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.621+0000 7f92fa7fc640 1 -- 172.21.15.76:0/2247552800 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f4001a70 con 0x7f92e8005270 2024-04-25T19:19:14.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.621+0000 7f92fa7fc640 1 -- 172.21.15.76:0/2247552800 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92f4003970 con 0x7f92e8005270 2024-04-25T19:19:14.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.621+0000 7f92fa7fc640 1 -- 172.21.15.76:0/2247552800 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f4003d60 con 0x7f92e8005270 2024-04-25T19:19:14.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.622+0000 7f9301586640 1 -- 172.21.15.76:0/2247552800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92e8005270 msgr2=0x7f92e800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:14.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.622+0000 7f9301586640 1 --2- 172.21.15.76:0/2247552800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92e8005270 0x7f92e800b9c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f92f4001e50 tx=0x7f92e8005830 comp rx=0 tx=0).stop 2024-04-25T19:19:14.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.622+0000 7f9301586640 1 -- 172.21.15.76:0/2247552800 shutdown_connections 2024-04-25T19:19:14.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.622+0000 7f9301586640 1 --2- 172.21.15.76:0/2247552800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92e8005270 0x7f92e800b9c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.622+0000 7f9301586640 1 -- 172.21.15.76:0/2247552800 >> 172.21.15.76:0/2247552800 conn(0x7f92fc0b3c00 msgr2=0x7f92fc0b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:14.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.622+0000 7f9301586640 1 -- 172.21.15.76:0/2247552800 shutdown_connections 2024-04-25T19:19:14.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.622+0000 7f9301586640 1 -- 172.21.15.76:0/2247552800 wait complete. 2024-04-25T19:19:14.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.624+0000 7f9301586640 1 Processor -- start 2024-04-25T19:19:14.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.624+0000 7f9301586640 1 -- start start 2024-04-25T19:19:14.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.625+0000 7f9301586640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92e8005270 0x7f92fc1f6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:14.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.625+0000 7f9301586640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92fc159b80 0x7f92fc15a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:14.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.625+0000 7f9301586640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92fc159260 0x7f92fc156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:14.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.625+0000 7f9301586640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc15cb70 con 0x7f92fc159260 2024-04-25T19:19:14.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.625+0000 7f9301586640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f92fc06dc40 con 0x7f92fc159b80 2024-04-25T19:19:14.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.625+0000 7f9301586640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f92fc1570e0 con 0x7f92e8005270 2024-04-25T19:19:14.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.625+0000 7f92fb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92fc159260 0x7f92fc156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:14.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.625+0000 7f92fb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92fc159260 0x7f92fc156240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39560/0 (socket says 172.21.15.76:39560) 2024-04-25T19:19:14.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.625+0000 7f92fb7fe640 1 -- 172.21.15.76:0/2706407600 learned_addr learned my addr 172.21.15.76:0/2706407600 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:14.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.625+0000 7f92faffd640 1 --2- 172.21.15.76:0/2706407600 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92e8005270 0x7f92fc1f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:14.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.625+0000 7f92f3fff640 1 --2- 172.21.15.76:0/2706407600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92fc159b80 0x7f92fc15a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:14.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.625+0000 7f92fb7fe640 1 -- 172.21.15.76:0/2706407600 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92e8005270 msgr2=0x7f92fc1f6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:14.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.625+0000 7f92fb7fe640 1 --2- 172.21.15.76:0/2706407600 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92e8005270 0x7f92fc1f6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.625+0000 7f92fb7fe640 1 -- 172.21.15.76:0/2706407600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92fc159b80 msgr2=0x7f92fc15a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:14.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.625+0000 7f92fb7fe640 1 --2- 172.21.15.76:0/2706407600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92fc159b80 0x7f92fc15a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.625+0000 7f92fb7fe640 1 -- 172.21.15.76:0/2706407600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92fc0b4fa0 con 0x7f92fc159260 2024-04-25T19:19:14.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.626+0000 7f92fb7fe640 1 --2- 172.21.15.76:0/2706407600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92fc159260 0x7f92fc156240 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f92f402a920 tx=0x7f92f40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:14.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.626+0000 7f92f37fe640 1 -- 172.21.15.76:0/2706407600 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f4001a70 con 0x7f92fc159260 2024-04-25T19:19:14.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.626+0000 7f92f37fe640 1 -- 172.21.15.76:0/2706407600 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92f402ac70 con 0x7f92fc159260 2024-04-25T19:19:14.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.626+0000 7f9301586640 1 -- 172.21.15.76:0/2706407600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92fc157b00 con 0x7f92fc159260 2024-04-25T19:19:14.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.626+0000 7f92f37fe640 1 -- 172.21.15.76:0/2706407600 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f4006d70 con 0x7f92fc159260 2024-04-25T19:19:14.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.626+0000 7f9301586640 1 -- 172.21.15.76:0/2706407600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92fc0d1650 con 0x7f92fc159260 2024-04-25T19:19:14.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.627+0000 7f9301586640 1 -- 172.21.15.76:0/2706407600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92fc0b4d10 con 0x7f92fc159260 2024-04-25T19:19:14.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.627+0000 7f92f37fe640 1 -- 172.21.15.76:0/2706407600 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f92f4007030 con 0x7f92fc159260 2024-04-25T19:19:14.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.628+0000 7f92f37fe640 1 --2- 172.21.15.76:0/2706407600 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f92dc0744e0 0x7f92dc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:14.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.628+0000 7f92f37fe640 1 -- 172.21.15.76:0/2706407600 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f92f40b4bc0 con 0x7f92fc159260 2024-04-25T19:19:14.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.628+0000 7f92faffd640 1 --2- 172.21.15.76:0/2706407600 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f92dc0744e0 0x7f92dc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:14.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.628+0000 7f92faffd640 1 --2- 172.21.15.76:0/2706407600 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f92dc0744e0 0x7f92dc0769f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f92ec007fd0 tx=0x7f92ec009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:14.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.632+0000 7f92f37fe640 1 -- 172.21.15.76:0/2706407600 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f92f4043060 con 0x7f92fc159260 2024-04-25T19:19:14.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.877+0000 7f9301586640 1 -- 172.21.15.76:0/2706407600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f92fc06a390 con 0x7f92fc159260 2024-04-25T19:19:14.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.878+0000 7f92f37fe640 1 -- 172.21.15.76:0/2706407600 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1559 (secure 0 0 0) 0x7f92f407f360 con 0x7f92fc159260 2024-04-25T19:19:14.880 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":28,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:19:14.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.880+0000 7f9301586640 1 -- 172.21.15.76:0/2706407600 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f92dc0744e0 msgr2=0x7f92dc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:14.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.880+0000 7f9301586640 1 --2- 172.21.15.76:0/2706407600 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f92dc0744e0 0x7f92dc0769f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f92ec007fd0 tx=0x7f92ec009040 comp rx=0 tx=0).stop 2024-04-25T19:19:14.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.880+0000 7f9301586640 1 -- 172.21.15.76:0/2706407600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92fc159260 msgr2=0x7f92fc156240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:14.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.880+0000 7f9301586640 1 --2- 172.21.15.76:0/2706407600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92fc159260 0x7f92fc156240 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f92f402a920 tx=0x7f92f40014a0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.880+0000 7f9301586640 1 -- 172.21.15.76:0/2706407600 shutdown_connections 2024-04-25T19:19:14.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.880+0000 7f9301586640 1 --2- 172.21.15.76:0/2706407600 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f92dc0744e0 0x7f92dc0769f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.880+0000 7f9301586640 1 --2- 172.21.15.76:0/2706407600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92fc159260 0x7f92fc156240 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.880+0000 7f9301586640 1 --2- 172.21.15.76:0/2706407600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92fc159b80 0x7f92fc15a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.880+0000 7f9301586640 1 --2- 172.21.15.76:0/2706407600 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f92e8005270 0x7f92fc1f6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:14.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.880+0000 7f9301586640 1 -- 172.21.15.76:0/2706407600 >> 172.21.15.76:0/2706407600 conn(0x7f92fc0b3c00 msgr2=0x7f92fc0ca500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:14.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.880+0000 7f9301586640 1 -- 172.21.15.76:0/2706407600 shutdown_connections 2024-04-25T19:19:14.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:14.880+0000 7f9301586640 1 -- 172.21.15.76:0/2706407600 wait complete. 2024-04-25T19:19:17.894 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:19:18.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.019+0000 7f663893c640 1 Processor -- start 2024-04-25T19:19:18.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.020+0000 7f663893c640 1 -- start start 2024-04-25T19:19:18.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.021+0000 7f663893c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66340ccbd0 0x7f66340c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.021+0000 7f663893c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6634160b80 con 0x7f66340ce700 2024-04-25T19:19:18.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.021+0000 7f663893c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66340c9030 con 0x7f66340c9570 2024-04-25T19:19:18.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.021+0000 7f663893c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f66340c9310 con 0x7f66340ccbd0 2024-04-25T19:19:18.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.022+0000 7f663259c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f66340ce700 0x7f663415ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47956/0 (socket says 172.21.15.76:47956) 2024-04-25T19:19:18.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.022+0000 7f663259c640 1 -- 172.21.15.76:0/1680063345 learned_addr learned my addr 172.21.15.76:0/1680063345 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:18.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.022+0000 7f6631d9b640 1 --2- 172.21.15.76:0/1680063345 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66340ccbd0 0x7f66340c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.022+0000 7f663159a640 1 -- 172.21.15.76:0/1680063345 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2084421998 0 0) 0x7f6634160b80 con 0x7f66340ce700 2024-04-25T19:19:18.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.022+0000 7f663159a640 1 -- 172.21.15.76:0/1680063345 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f661c003520 con 0x7f66340ce700 2024-04-25T19:19:18.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.022+0000 7f6631d9b640 1 -- 172.21.15.76:0/1680063345 >> v1:172.21.15.105:6789/0 conn(0x7f66340c9570 legacy=0x7f6634167370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.022+0000 7f6631d9b640 1 -- 172.21.15.76:0/1680063345 >> v1:172.21.15.76:6789/0 conn(0x7f66340ce700 legacy=0x7f663415ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.022+0000 7f6631d9b640 1 -- 172.21.15.76:0/1680063345 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66340cc3d0 con 0x7f66340ccbd0 2024-04-25T19:19:18.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.023+0000 7f6631d9b640 1 --2- 172.21.15.76:0/1680063345 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66340ccbd0 0x7f66340c5d30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f66280097d0 tx=0x7f662802a2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8abfc45b3cebad2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:18.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.023+0000 7f663159a640 1 -- 172.21.15.76:0/1680063345 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6628037070 con 0x7f66340ccbd0 2024-04-25T19:19:18.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.024+0000 7f663159a640 1 -- 172.21.15.76:0/1680063345 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6628032640 con 0x7f66340ccbd0 2024-04-25T19:19:18.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.024+0000 7f663159a640 1 -- 172.21.15.76:0/1680063345 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6628032a10 con 0x7f66340ccbd0 2024-04-25T19:19:18.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.025+0000 7f663893c640 1 -- 172.21.15.76:0/1680063345 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66340ccbd0 msgr2=0x7f66340c5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.025+0000 7f663893c640 1 --2- 172.21.15.76:0/1680063345 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66340ccbd0 0x7f66340c5d30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f66280097d0 tx=0x7f662802a2c0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.025+0000 7f663893c640 1 -- 172.21.15.76:0/1680063345 shutdown_connections 2024-04-25T19:19:18.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.025+0000 7f663893c640 1 --2- 172.21.15.76:0/1680063345 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66340ccbd0 0x7f66340c5d30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.025+0000 7f663893c640 1 -- 172.21.15.76:0/1680063345 >> 172.21.15.76:0/1680063345 conn(0x7f6634070800 msgr2=0x7f6634070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:18.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.025+0000 7f663893c640 1 -- 172.21.15.76:0/1680063345 shutdown_connections 2024-04-25T19:19:18.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.025+0000 7f663893c640 1 -- 172.21.15.76:0/1680063345 wait complete. 2024-04-25T19:19:18.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.028+0000 7f663893c640 1 Processor -- start 2024-04-25T19:19:18.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.028+0000 7f663893c640 1 -- start start 2024-04-25T19:19:18.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.030+0000 7f663893c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66340c9570 0x7f66341e79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.030+0000 7f663893c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66340ccbd0 0x7f66341e6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.030+0000 7f663893c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66341f37d0 0x7f6634159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.030+0000 7f663893c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f66340cd310 con 0x7f66340c9570 2024-04-25T19:19:18.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.030+0000 7f663893c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f66340c9980 con 0x7f66340ccbd0 2024-04-25T19:19:18.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.030+0000 7f663893c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f66340c8520 con 0x7f66341f37d0 2024-04-25T19:19:18.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.030+0000 7f6631d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66340ccbd0 0x7f66341e6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.031+0000 7f6631d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66340ccbd0 0x7f66341e6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:50236/0 (socket says 172.21.15.76:50236) 2024-04-25T19:19:18.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.031+0000 7f6631d9b640 1 -- 172.21.15.76:0/3282033095 learned_addr learned my addr 172.21.15.76:0/3282033095 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:18.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.031+0000 7f663259c640 1 --2- 172.21.15.76:0/3282033095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66340c9570 0x7f66341e79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.031+0000 7f6632d9d640 1 --2- 172.21.15.76:0/3282033095 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66341f37d0 0x7f6634159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.031+0000 7f663259c640 1 -- 172.21.15.76:0/3282033095 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66341f37d0 msgr2=0x7f6634159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.031+0000 7f663259c640 1 --2- 172.21.15.76:0/3282033095 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66341f37d0 0x7f6634159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.031+0000 7f663259c640 1 -- 172.21.15.76:0/3282033095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66340ccbd0 msgr2=0x7f66341e6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.031+0000 7f663259c640 1 --2- 172.21.15.76:0/3282033095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66340ccbd0 0x7f66341e6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.031+0000 7f663259c640 1 -- 172.21.15.76:0/3282033095 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66340780a0 con 0x7f66340c9570 2024-04-25T19:19:18.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.031+0000 7f663259c640 1 --2- 172.21.15.76:0/3282033095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66340c9570 0x7f66341e79b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6624015c20 tx=0x7f6624013560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:18.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.032+0000 7f6622ffd640 1 -- 172.21.15.76:0/3282033095 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662401d070 con 0x7f66340c9570 2024-04-25T19:19:18.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.032+0000 7f6622ffd640 1 -- 172.21.15.76:0/3282033095 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6624013e10 con 0x7f66340c9570 2024-04-25T19:19:18.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.032+0000 7f6622ffd640 1 -- 172.21.15.76:0/3282033095 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6624012030 con 0x7f66340c9570 2024-04-25T19:19:18.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.032+0000 7f663893c640 1 -- 172.21.15.76:0/3282033095 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f663415d6e0 con 0x7f66340c9570 2024-04-25T19:19:18.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.033+0000 7f663893c640 1 -- 172.21.15.76:0/3282033095 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66341f2fd0 con 0x7f66340c9570 2024-04-25T19:19:18.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.034+0000 7f663893c640 1 -- 172.21.15.76:0/3282033095 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66341f2ae0 con 0x7f66340c9570 2024-04-25T19:19:18.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.034+0000 7f6622ffd640 1 -- 172.21.15.76:0/3282033095 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6624020070 con 0x7f66340c9570 2024-04-25T19:19:18.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.035+0000 7f6622ffd640 1 --2- 172.21.15.76:0/3282033095 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f66140744b0 0x7f66140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.035+0000 7f6622ffd640 1 -- 172.21.15.76:0/3282033095 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f662409d820 con 0x7f66340c9570 2024-04-25T19:19:18.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.035+0000 7f6631d9b640 1 --2- 172.21.15.76:0/3282033095 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f66140744b0 0x7f66140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.036+0000 7f6631d9b640 1 --2- 172.21.15.76:0/3282033095 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f66140744b0 0x7f66140769c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f663415f2b0 tx=0x7f662802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:18.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.040+0000 7f6622ffd640 1 -- 172.21.15.76:0/3282033095 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6624067860 con 0x7f66340c9570 2024-04-25T19:19:18.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.310+0000 7f663893c640 1 -- 172.21.15.76:0/3282033095 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f663406a390 con 0x7f66340c9570 2024-04-25T19:19:18.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.310+0000 7f6622ffd640 1 -- 172.21.15.76:0/3282033095 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1564 (secure 0 0 0) 0x7f662406bb60 con 0x7f66340c9570 2024-04-25T19:19:18.312 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":30,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:19:18.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.312+0000 7f663893c640 1 -- 172.21.15.76:0/3282033095 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f66140744b0 msgr2=0x7f66140769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.312+0000 7f663893c640 1 --2- 172.21.15.76:0/3282033095 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f66140744b0 0x7f66140769c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f663415f2b0 tx=0x7f662802e000 comp rx=0 tx=0).stop 2024-04-25T19:19:18.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.312+0000 7f663893c640 1 -- 172.21.15.76:0/3282033095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66340c9570 msgr2=0x7f66341e79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.312+0000 7f663893c640 1 --2- 172.21.15.76:0/3282033095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66340c9570 0x7f66341e79b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6624015c20 tx=0x7f6624013560 comp rx=0 tx=0).stop 2024-04-25T19:19:18.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.313+0000 7f663893c640 1 -- 172.21.15.76:0/3282033095 shutdown_connections 2024-04-25T19:19:18.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.313+0000 7f663893c640 1 --2- 172.21.15.76:0/3282033095 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f66140744b0 0x7f66140769c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.313+0000 7f663893c640 1 --2- 172.21.15.76:0/3282033095 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f66341f37d0 0x7f6634159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.313+0000 7f663893c640 1 --2- 172.21.15.76:0/3282033095 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66340ccbd0 0x7f66341e6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.313+0000 7f663893c640 1 --2- 172.21.15.76:0/3282033095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66340c9570 0x7f66341e79b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.313+0000 7f663893c640 1 -- 172.21.15.76:0/3282033095 >> 172.21.15.76:0/3282033095 conn(0x7f6634070800 msgr2=0x7f663415ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:18.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.313+0000 7f663893c640 1 -- 172.21.15.76:0/3282033095 shutdown_connections 2024-04-25T19:19:18.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.313+0000 7f663893c640 1 -- 172.21.15.76:0/3282033095 wait complete. 2024-04-25T19:19:18.325 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T19:19:18.325 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-04-25T19:19:18.326 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-04-25T19:19:18.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.434+0000 7f6997e51640 1 Processor -- start 2024-04-25T19:19:18.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.434+0000 7f6997e51640 1 -- start start 2024-04-25T19:19:18.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.436+0000 7f6997e51640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f69900714c0 0x7f6990154a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.436+0000 7f6997e51640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f699006d120 0x7f699015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.436+0000 7f6997e51640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6990154f80 con 0x7f699015e9c0 2024-04-25T19:19:18.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.436+0000 7f6997e51640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f69900cb490 con 0x7f699006d120 2024-04-25T19:19:18.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.436+0000 7f6997e51640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f69900ca7f0 con 0x7f69900714c0 2024-04-25T19:19:18.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.436+0000 7f69963ee640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f699015e9c0 0x7f69900c8440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47960/0 (socket says 172.21.15.76:47960) 2024-04-25T19:19:18.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.436+0000 7f69963ee640 1 -- 172.21.15.76:0/1593543304 learned_addr learned my addr 172.21.15.76:0/1593543304 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:18.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.436+0000 7f69953ec640 1 --2- 172.21.15.76:0/1593543304 >> v2:172.21.15.105:3300/0 conn(0x7f699006d120 0x7f699015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.436+0000 7f6995bed640 1 --2- 172.21.15.76:0/1593543304 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f69900714c0 0x7f6990154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.436+0000 7f6994beb640 1 -- 172.21.15.76:0/1593543304 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2079530617 0 0) 0x7f6990154f80 con 0x7f699015e9c0 2024-04-25T19:19:18.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.436+0000 7f6994beb640 1 -- 172.21.15.76:0/1593543304 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f697c003520 con 0x7f699015e9c0 2024-04-25T19:19:18.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.436+0000 7f6995bed640 1 -- 172.21.15.76:0/1593543304 >> v2:172.21.15.105:3300/0 conn(0x7f699006d120 msgr2=0x7f699015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:18.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.436+0000 7f6995bed640 1 --2- 172.21.15.76:0/1593543304 >> v2:172.21.15.105:3300/0 conn(0x7f699006d120 0x7f699015e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.436+0000 7f6995bed640 1 -- 172.21.15.76:0/1593543304 >> v1:172.21.15.76:6789/0 conn(0x7f699015e9c0 legacy=0x7f69900c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.436+0000 7f6995bed640 1 -- 172.21.15.76:0/1593543304 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699006c7b0 con 0x7f69900714c0 2024-04-25T19:19:18.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.436+0000 7f69953ec640 1 --2- 172.21.15.76:0/1593543304 >> v2:172.21.15.105:3300/0 conn(0x7f699006d120 0x7f699015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:19:18.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.437+0000 7f6995bed640 1 --2- 172.21.15.76:0/1593543304 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f69900714c0 0x7f6990154a40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f698c0097a0 tx=0x7f698c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db6520f8ab1e6483 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:18.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.437+0000 7f6994beb640 1 -- 172.21.15.76:0/1593543304 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c037070 con 0x7f69900714c0 2024-04-25T19:19:18.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.437+0000 7f6994beb640 1 -- 172.21.15.76:0/1593543304 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f698c032440 con 0x7f69900714c0 2024-04-25T19:19:18.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.437+0000 7f6994beb640 1 -- 172.21.15.76:0/1593543304 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c0327f0 con 0x7f69900714c0 2024-04-25T19:19:18.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.438+0000 7f6997e51640 1 -- 172.21.15.76:0/1593543304 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f69900714c0 msgr2=0x7f6990154a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.438+0000 7f6997e51640 1 --2- 172.21.15.76:0/1593543304 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f69900714c0 0x7f6990154a40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f698c0097a0 tx=0x7f698c02a210 comp rx=0 tx=0).stop 2024-04-25T19:19:18.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.438+0000 7f6997e51640 1 -- 172.21.15.76:0/1593543304 shutdown_connections 2024-04-25T19:19:18.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.438+0000 7f6997e51640 1 --2- 172.21.15.76:0/1593543304 >> v2:172.21.15.105:3300/0 conn(0x7f699006d120 0x7f699015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.438+0000 7f6997e51640 1 --2- 172.21.15.76:0/1593543304 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f69900714c0 0x7f6990154a40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.438+0000 7f6997e51640 1 -- 172.21.15.76:0/1593543304 >> 172.21.15.76:0/1593543304 conn(0x7f699006d6d0 msgr2=0x7f699006dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:18.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.438+0000 7f6997e51640 1 -- 172.21.15.76:0/1593543304 shutdown_connections 2024-04-25T19:19:18.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.438+0000 7f6997e51640 1 -- 172.21.15.76:0/1593543304 wait complete. 2024-04-25T19:19:18.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.440+0000 7f6997e51640 1 Processor -- start 2024-04-25T19:19:18.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.440+0000 7f6997e51640 1 -- start start 2024-04-25T19:19:18.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.441+0000 7f6997e51640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699006d120 0x7f699015b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.441+0000 7f6997e51640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f69900714c0 0x7f69900cef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.441+0000 7f6997e51640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f699015e9c0 0x7f69901f4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.441+0000 7f6997e51640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f699015df60 con 0x7f699006d120 2024-04-25T19:19:18.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.441+0000 7f6997e51640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f69900ca510 con 0x7f699015e9c0 2024-04-25T19:19:18.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.441+0000 7f6997e51640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f69900724b0 con 0x7f69900714c0 2024-04-25T19:19:18.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.441+0000 7f6995bed640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699006d120 0x7f699015b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.441+0000 7f6995bed640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699006d120 0x7f699015b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39580/0 (socket says 172.21.15.76:39580) 2024-04-25T19:19:18.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.441+0000 7f6995bed640 1 -- 172.21.15.76:0/1825811824 learned_addr learned my addr 172.21.15.76:0/1825811824 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:18.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.442+0000 7f69963ee640 1 --2- 172.21.15.76:0/1825811824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f699015e9c0 0x7f69901f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.442+0000 7f69953ec640 1 --2- 172.21.15.76:0/1825811824 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f69900714c0 0x7f69900cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.442+0000 7f6995bed640 1 -- 172.21.15.76:0/1825811824 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f69900714c0 msgr2=0x7f69900cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.442+0000 7f6995bed640 1 --2- 172.21.15.76:0/1825811824 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f69900714c0 0x7f69900cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.442+0000 7f6995bed640 1 -- 172.21.15.76:0/1825811824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f699015e9c0 msgr2=0x7f69901f4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.442+0000 7f6995bed640 1 --2- 172.21.15.76:0/1825811824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f699015e9c0 0x7f69901f4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.442+0000 7f6995bed640 1 -- 172.21.15.76:0/1825811824 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699015a830 con 0x7f699006d120 2024-04-25T19:19:18.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.442+0000 7f69953ec640 1 --2- 172.21.15.76:0/1825811824 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f69900714c0 0x7f69900cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:19:18.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.442+0000 7f6995bed640 1 --2- 172.21.15.76:0/1825811824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699006d120 0x7f699015b2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f698c035040 tx=0x7f698c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:18.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.442+0000 7f69867fc640 1 -- 172.21.15.76:0/1825811824 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c037040 con 0x7f699006d120 2024-04-25T19:19:18.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.442+0000 7f69867fc640 1 -- 172.21.15.76:0/1825811824 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f698c004230 con 0x7f699006d120 2024-04-25T19:19:18.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.442+0000 7f6997e51640 1 -- 172.21.15.76:0/1825811824 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6990158e30 con 0x7f699006d120 2024-04-25T19:19:18.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.442+0000 7f69867fc640 1 -- 172.21.15.76:0/1825811824 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c006e10 con 0x7f699006d120 2024-04-25T19:19:18.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.442+0000 7f6997e51640 1 -- 172.21.15.76:0/1825811824 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f699015aaf0 con 0x7f699006d120 2024-04-25T19:19:18.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.443+0000 7f6997e51640 1 --2- 172.21.15.76:0/1825811824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699015acf0 0x7f69901f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.443+0000 7f6997e51640 1 -- 172.21.15.76:0/1825811824 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6990158be0 con 0x7f699015acf0 2024-04-25T19:19:18.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.443+0000 7f69953ec640 1 --2- 172.21.15.76:0/1825811824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699015acf0 0x7f69901f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.443+0000 7f69953ec640 1 -- 172.21.15.76:0/1825811824 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f69900ca970 con 0x7f699015acf0 2024-04-25T19:19:18.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.443+0000 7f69953ec640 1 --2- 172.21.15.76:0/1825811824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699015acf0 0x7f69901f4970 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f699014e7c0 tx=0x7f6980010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:18.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.444+0000 7f69867fc640 1 -- 172.21.15.76:0/1825811824 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698000f040 con 0x7f699015acf0 2024-04-25T19:19:18.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.444+0000 7f69867fc640 1 -- 172.21.15.76:0/1825811824 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f698c03a070 con 0x7f699006d120 2024-04-25T19:19:18.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.444+0000 7f69867fc640 1 --2- 172.21.15.76:0/1825811824 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f69740743e0 0x7f69740768f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.444+0000 7f69867fc640 1 -- 172.21.15.76:0/1825811824 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f698c0b56a0 con 0x7f699006d120 2024-04-25T19:19:18.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.444+0000 7f69867fc640 1 -- 172.21.15.76:0/1825811824 <== mon.0 v2:172.21.15.76:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6980004220 con 0x7f699015acf0 2024-04-25T19:19:18.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.444+0000 7f69867fc640 1 -- 172.21.15.76:0/1825811824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699015acf0 msgr2=0x7f69901f4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.444+0000 7f69867fc640 1 --2- 172.21.15.76:0/1825811824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699015acf0 0x7f69901f4970 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f699014e7c0 tx=0x7f6980010c20 comp rx=0 tx=0).stop 2024-04-25T19:19:18.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.444+0000 7f69867fc640 1 -- 172.21.15.76:0/1825811824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699015acf0 msgr2=0x7f69901f4970 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:19:18.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.444+0000 7f69867fc640 1 --2- 172.21.15.76:0/1825811824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699015acf0 0x7f69901f4970 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.445+0000 7f69963ee640 1 --2- 172.21.15.76:0/1825811824 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f69740743e0 0x7f69740768f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.445+0000 7f69963ee640 1 --2- 172.21.15.76:0/1825811824 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f69740743e0 0x7f69740768f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f69900cec90 tx=0x7f6988009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:18.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.451+0000 7f6997e51640 1 --2- 172.21.15.76:0/1825811824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6990158680 0x7f69900cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.451+0000 7f6997e51640 1 -- 172.21.15.76:0/1825811824 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6990154b60 con 0x7f6990158680 2024-04-25T19:19:18.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.451+0000 7f69953ec640 1 --2- 172.21.15.76:0/1825811824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6990158680 0x7f69900cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.451+0000 7f69953ec640 1 -- 172.21.15.76:0/1825811824 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f699006a390 con 0x7f6990158680 2024-04-25T19:19:18.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.452+0000 7f69953ec640 1 --2- 172.21.15.76:0/1825811824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6990158680 0x7f69900cdb80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f699014e7c0 tx=0x7f69800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:18.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.454+0000 7f69867fc640 1 -- 172.21.15.76:0/1825811824 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980006250 con 0x7f6990158680 2024-04-25T19:19:18.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.454+0000 7f69867fc640 1 -- 172.21.15.76:0/1825811824 <== mon.0 v2:172.21.15.76:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4896 (secure 0 0 0) 0x7f6980005e80 con 0x7f6990158680 2024-04-25T19:19:18.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.454+0000 7f69867fc640 1 -- 172.21.15.76:0/1825811824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6990158680 msgr2=0x7f69900cdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.454+0000 7f69867fc640 1 --2- 172.21.15.76:0/1825811824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6990158680 0x7f69900cdb80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f699014e7c0 tx=0x7f69800046c0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.454+0000 7f69867fc640 1 -- 172.21.15.76:0/1825811824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6990158680 msgr2=0x7f69900cdb80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:19:18.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.454+0000 7f69867fc640 1 --2- 172.21.15.76:0/1825811824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6990158680 0x7f69900cdb80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.456 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:19:18.456 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:19:18.456 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:19:18.456 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "leader", 2024-04-25T19:19:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 30, 2024-04-25T19:19:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 57566, 2024-04-25T19:19:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:19:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:19:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-04-25T19:19:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:19:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:19:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 0, 2024-04-25T19:19:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:19:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:19:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:19:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:19:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:19:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:19:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:19:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:19:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:19:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:19:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:19:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:19:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:19:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:19:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:19:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:19:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:19:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:19:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:19:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:19:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:19:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:19:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:19:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:19:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:19:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:19:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:19:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:19:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:19:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:19:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:19:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:19:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:19:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:19:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:19:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:19:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:19:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:19:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:19:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:19:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:19:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:19:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:19:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:19:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:19:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:19:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:19:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:19:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:19:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:19:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:19:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:19:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:19:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:19:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:19:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:19:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:19:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:19:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:19:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:19:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:19:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:19:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:19:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:19:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:19:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:19:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:19:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:19:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:19:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:19:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:19:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:19:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:19:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:19:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:19:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:19:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:19:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:19:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:19:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:19:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:19:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:19:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:19:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:19:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:19:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:19:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:19:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:19:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:19:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:19:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:19:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:19:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:19:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:19:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:19:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:19:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:19:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:19:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:19:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:19:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:19:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:19:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:19:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:19:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:19:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:19:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:19:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:19:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:19:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:19:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:19:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:19:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:19:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:19:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:19:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:19:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:19:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:19:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:19:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:19:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": [ 2024-04-25T19:19:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:19:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:19:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 2 2024-04-25T19:19:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:19:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:19:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": [ 2024-04-25T19:19:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:19:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:19:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:19:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:19:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:19:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:19:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:19:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:19:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 2 2024-04-25T19:19:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:19:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:19:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:19:18.469 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:19:18.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.455+0000 7f6997e51640 1 -- 172.21.15.76:0/1825811824 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f69740743e0 msgr2=0x7f69740768f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.455+0000 7f6997e51640 1 --2- 172.21.15.76:0/1825811824 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f69740743e0 0x7f69740768f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f69900cec90 tx=0x7f6988009040 comp rx=0 tx=0).stop 2024-04-25T19:19:18.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.455+0000 7f6997e51640 1 -- 172.21.15.76:0/1825811824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699006d120 msgr2=0x7f699015b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.455+0000 7f6997e51640 1 --2- 172.21.15.76:0/1825811824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699006d120 0x7f699015b2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f698c035040 tx=0x7f698c0046c0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.455+0000 7f69963ee640 1 -- 172.21.15.76:0/1825811824 reap_dead start 2024-04-25T19:19:18.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.455+0000 7f69963ee640 1 -- 172.21.15.76:0/1825811824 reap_dead start 2024-04-25T19:19:18.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.455+0000 7f6997e51640 1 -- 172.21.15.76:0/1825811824 shutdown_connections 2024-04-25T19:19:18.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.455+0000 7f6997e51640 1 -- 172.21.15.76:0/1825811824 >> 172.21.15.76:0/1825811824 conn(0x7f699006d6d0 msgr2=0x7f699006dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:18.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.455+0000 7f6997e51640 1 -- 172.21.15.76:0/1825811824 shutdown_connections 2024-04-25T19:19:18.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.455+0000 7f6997e51640 1 -- 172.21.15.76:0/1825811824 wait complete. 2024-04-25T19:19:18.471 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T19:19:18.471 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-04-25T19:19:18.471 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-04-25T19:19:18.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.580+0000 7f5f00764640 1 Processor -- start 2024-04-25T19:19:18.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.580+0000 7f5f00764640 1 -- start start 2024-04-25T19:19:18.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.581+0000 7f5f00764640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5ef806e870 0x7f5ef80cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.581+0000 7f5f00764640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ef815ce80 con 0x7f5ef80d0810 2024-04-25T19:19:18.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.581+0000 7f5f00764640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ef815d160 con 0x7f5ef80cea70 2024-04-25T19:19:18.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.581+0000 7f5f00764640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f5ef815d440 con 0x7f5ef806e870 2024-04-25T19:19:18.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.581+0000 7f5efe500640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f5ef80d0810 0x7f5ef806f000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47968/0 (socket says 172.21.15.76:47968) 2024-04-25T19:19:18.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.581+0000 7f5efe500640 1 -- 172.21.15.76:0/748672534 learned_addr learned my addr 172.21.15.76:0/748672534 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:18.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.581+0000 7f5efed01640 1 --2- 172.21.15.76:0/748672534 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5ef806e870 0x7f5ef80cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.581+0000 7f5efd4fe640 1 -- 172.21.15.76:0/748672534 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1490660730 0 0) 0x7f5ef815ce80 con 0x7f5ef80d0810 2024-04-25T19:19:18.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.582+0000 7f5efd4fe640 1 -- 172.21.15.76:0/748672534 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ee4003520 con 0x7f5ef80d0810 2024-04-25T19:19:18.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.582+0000 7f5efd4fe640 1 -- 172.21.15.76:0/748672534 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2029516668 0 0) 0x7f5ef815d160 con 0x7f5ef80cea70 2024-04-25T19:19:18.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.582+0000 7f5efd4fe640 1 -- 172.21.15.76:0/748672534 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ef815ce80 con 0x7f5ef80cea70 2024-04-25T19:19:18.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.582+0000 7f5efed01640 1 -- 172.21.15.76:0/748672534 >> v1:172.21.15.105:6789/0 conn(0x7f5ef80cea70 legacy=0x7f5ef806e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.582+0000 7f5efed01640 1 -- 172.21.15.76:0/748672534 >> v1:172.21.15.76:6789/0 conn(0x7f5ef80d0810 legacy=0x7f5ef806f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.582+0000 7f5efed01640 1 -- 172.21.15.76:0/748672534 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef815d720 con 0x7f5ef806e870 2024-04-25T19:19:18.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.582+0000 7f5efed01640 1 --2- 172.21.15.76:0/748672534 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5ef806e870 0x7f5ef80cee80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00097d0 tx=0x7f5ef002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32858b99f10adfa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:18.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.583+0000 7f5efd4fe640 1 -- 172.21.15.76:0/748672534 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0037070 con 0x7f5ef806e870 2024-04-25T19:19:18.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.583+0000 7f5efd4fe640 1 -- 172.21.15.76:0/748672534 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef0032440 con 0x7f5ef806e870 2024-04-25T19:19:18.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.583+0000 7f5efd4fe640 1 -- 172.21.15.76:0/748672534 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0032810 con 0x7f5ef806e870 2024-04-25T19:19:18.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.583+0000 7f5f00764640 1 -- 172.21.15.76:0/748672534 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5ef806e870 msgr2=0x7f5ef80cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.583+0000 7f5f00764640 1 --2- 172.21.15.76:0/748672534 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5ef806e870 0x7f5ef80cee80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00097d0 tx=0x7f5ef002a210 comp rx=0 tx=0).stop 2024-04-25T19:19:18.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.583+0000 7f5f00764640 1 -- 172.21.15.76:0/748672534 shutdown_connections 2024-04-25T19:19:18.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.583+0000 7f5f00764640 1 --2- 172.21.15.76:0/748672534 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5ef806e870 0x7f5ef80cee80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.583+0000 7f5f00764640 1 -- 172.21.15.76:0/748672534 >> 172.21.15.76:0/748672534 conn(0x7f5ef80b3c00 msgr2=0x7f5ef80b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:18.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.583+0000 7f5f00764640 1 -- 172.21.15.76:0/748672534 shutdown_connections 2024-04-25T19:19:18.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.583+0000 7f5f00764640 1 -- 172.21.15.76:0/748672534 wait complete. 2024-04-25T19:19:18.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.585+0000 7f5f00764640 1 Processor -- start 2024-04-25T19:19:18.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.585+0000 7f5f00764640 1 -- start start 2024-04-25T19:19:18.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.586+0000 7f5f00764640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ef806e870 0x7f5ef81f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.586+0000 7f5f00764640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5ef80cea70 0x7f5ef815bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.586+0000 7f5f00764640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef80d0810 0x7f5ef8150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.586+0000 7f5f00764640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef815cb70 con 0x7f5ef806e870 2024-04-25T19:19:18.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.586+0000 7f5f00764640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef806dc40 con 0x7f5ef80d0810 2024-04-25T19:19:18.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.586+0000 7f5f00764640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f5ef81570e0 con 0x7f5ef80cea70 2024-04-25T19:19:18.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.586+0000 7f5efe500640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ef806e870 0x7f5ef81f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.586+0000 7f5efe500640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ef806e870 0x7f5ef81f2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39610/0 (socket says 172.21.15.76:39610) 2024-04-25T19:19:18.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.586+0000 7f5efe500640 1 -- 172.21.15.76:0/1488565790 learned_addr learned my addr 172.21.15.76:0/1488565790 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:18.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.587+0000 7f5efed01640 1 --2- 172.21.15.76:0/1488565790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef80d0810 0x7f5ef8150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.587+0000 7f5efdcff640 1 --2- 172.21.15.76:0/1488565790 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5ef80cea70 0x7f5ef815bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.587+0000 7f5efe500640 1 -- 172.21.15.76:0/1488565790 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5ef80cea70 msgr2=0x7f5ef815bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.587+0000 7f5efe500640 1 --2- 172.21.15.76:0/1488565790 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5ef80cea70 0x7f5ef815bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.587+0000 7f5efe500640 1 -- 172.21.15.76:0/1488565790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef80d0810 msgr2=0x7f5ef8150510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.587+0000 7f5efe500640 1 --2- 172.21.15.76:0/1488565790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef80d0810 0x7f5ef8150510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.587+0000 7f5efe500640 1 -- 172.21.15.76:0/1488565790 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef8151b50 con 0x7f5ef806e870 2024-04-25T19:19:18.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.587+0000 7f5efe500640 1 --2- 172.21.15.76:0/1488565790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ef806e870 0x7f5ef81f2e10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5ef400cb50 tx=0x7f5ef400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:18.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.587+0000 7f5eeeffd640 1 -- 172.21.15.76:0/1488565790 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4017070 con 0x7f5ef806e870 2024-04-25T19:19:18.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.587+0000 7f5eeeffd640 1 -- 172.21.15.76:0/1488565790 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef400ad40 con 0x7f5ef806e870 2024-04-25T19:19:18.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.587+0000 7f5eeeffd640 1 -- 172.21.15.76:0/1488565790 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4012620 con 0x7f5ef806e870 2024-04-25T19:19:18.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.587+0000 7f5f00764640 1 -- 172.21.15.76:0/1488565790 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef81f3350 con 0x7f5ef806e870 2024-04-25T19:19:18.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.587+0000 7f5f00764640 1 -- 172.21.15.76:0/1488565790 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef81f6340 con 0x7f5ef806e870 2024-04-25T19:19:18.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.588+0000 7f5f00764640 1 --2- 172.21.15.76:0/1488565790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef815dff0 0x7f5ef8150a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.588+0000 7f5f00764640 1 -- 172.21.15.76:0/1488565790 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef806a390 con 0x7f5ef815dff0 2024-04-25T19:19:18.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.588+0000 7f5efdcff640 1 --2- 172.21.15.76:0/1488565790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef815dff0 0x7f5ef8150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.589+0000 7f5efdcff640 1 -- 172.21.15.76:0/1488565790 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5ef8150f90 con 0x7f5ef815dff0 2024-04-25T19:19:18.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.589+0000 7f5efdcff640 1 --2- 172.21.15.76:0/1488565790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef815dff0 0x7f5ef8150a50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8155b20 tx=0x7f5ee800e820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:18.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.589+0000 7f5eeeffd640 1 -- 172.21.15.76:0/1488565790 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8019070 con 0x7f5ef815dff0 2024-04-25T19:19:18.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.589+0000 7f5eeeffd640 1 -- 172.21.15.76:0/1488565790 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5ef401a070 con 0x7f5ef806e870 2024-04-25T19:19:18.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.589+0000 7f5eeeffd640 1 --2- 172.21.15.76:0/1488565790 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5edc0743b0 0x7f5edc0768c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.589+0000 7f5eeeffd640 1 -- 172.21.15.76:0/1488565790 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f5ef4094fd0 con 0x7f5ef806e870 2024-04-25T19:19:18.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.590+0000 7f5efed01640 1 --2- 172.21.15.76:0/1488565790 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5edc0743b0 0x7f5edc0768c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.590+0000 7f5eeeffd640 1 -- 172.21.15.76:0/1488565790 <== mon.1 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5ee80052e0 con 0x7f5ef815dff0 2024-04-25T19:19:18.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.590+0000 7f5eeeffd640 1 -- 172.21.15.76:0/1488565790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef815dff0 msgr2=0x7f5ef8150a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.590+0000 7f5eeeffd640 1 --2- 172.21.15.76:0/1488565790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef815dff0 0x7f5ef8150a50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8155b20 tx=0x7f5ee800e820 comp rx=0 tx=0).stop 2024-04-25T19:19:18.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.590+0000 7f5eeeffd640 1 -- 172.21.15.76:0/1488565790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef815dff0 msgr2=0x7f5ef8150a50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:19:18.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.590+0000 7f5eeeffd640 1 --2- 172.21.15.76:0/1488565790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef815dff0 0x7f5ef8150a50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.590+0000 7f5efed01640 1 --2- 172.21.15.76:0/1488565790 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5edc0743b0 0x7f5edc0768c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0034000 tx=0x7f5ef002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:18.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.596+0000 7f5f00764640 1 --2- 172.21.15.76:0/1488565790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef81514f0 0x7f5ef81f6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.596+0000 7f5f00764640 1 -- 172.21.15.76:0/1488565790 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef80b8070 con 0x7f5ef81514f0 2024-04-25T19:19:18.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.596+0000 7f5efdcff640 1 --2- 172.21.15.76:0/1488565790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef81514f0 0x7f5ef81f6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.597+0000 7f5efdcff640 1 -- 172.21.15.76:0/1488565790 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5ef80b4260 con 0x7f5ef81514f0 2024-04-25T19:19:18.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.597+0000 7f5efdcff640 1 --2- 172.21.15.76:0/1488565790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef81514f0 0x7f5ef81f6620 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8009010 tx=0x7f5ee8000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:18.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.597+0000 7f5eeeffd640 1 -- 172.21.15.76:0/1488565790 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8003800 con 0x7f5ef81514f0 2024-04-25T19:19:18.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.598+0000 7f5eeeffd640 1 -- 172.21.15.76:0/1488565790 <== mon.1 v2:172.21.15.105:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4553 (secure 0 0 0) 0x7f5ee8003800 con 0x7f5ef81514f0 2024-04-25T19:19:18.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.598+0000 7f5eeeffd640 1 -- 172.21.15.76:0/1488565790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef81514f0 msgr2=0x7f5ef81f6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.598+0000 7f5eeeffd640 1 --2- 172.21.15.76:0/1488565790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef81514f0 0x7f5ef81f6620 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8009010 tx=0x7f5ee8000ed0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.598+0000 7f5eeeffd640 1 -- 172.21.15.76:0/1488565790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef81514f0 msgr2=0x7f5ef81f6620 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:19:18.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.598+0000 7f5eeeffd640 1 --2- 172.21.15.76:0/1488565790 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5ef81514f0 0x7f5ef81f6620 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.600 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:19:18.600 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:19:18.600 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:19:18.600 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "peon", 2024-04-25T19:19:18.600 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 30, 2024-04-25T19:19:18.600 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 3989, 2024-04-25T19:19:18.600 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:19:18.600 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:19:18.600 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-04-25T19:19:18.600 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:19:18.600 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:19:18.600 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 0, 2024-04-25T19:19:18.600 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:19:18.601 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:19:18.601 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:19:18.601 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:19:18.601 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:19:18.601 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:19:18.601 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:19:18.601 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:19:18.601 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:19:18.601 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:19:18.601 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:19:18.601 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:19:18.601 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:19:18.601 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:19:18.601 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:19:18.601 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:19:18.602 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:19:18.602 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:19:18.602 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:19:18.602 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:19:18.602 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:19:18.602 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:19:18.602 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:19:18.602 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:19:18.602 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:19:18.602 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:19:18.602 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:19:18.602 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:19:18.602 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:19:18.602 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.603 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:19:18.603 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:19:18.603 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:19:18.603 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:19:18.603 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:19:18.603 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:19:18.603 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:19:18.603 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:19:18.603 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:19:18.603 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:19:18.603 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:19:18.603 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:19:18.603 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:19:18.603 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:19:18.603 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:19:18.604 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:19:18.604 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:19:18.604 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:19:18.604 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:19:18.604 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:19:18.604 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:19:18.604 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:19:18.604 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:19:18.604 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:19:18.604 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:19:18.604 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:19:18.604 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:19:18.604 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:19:18.604 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:19:18.604 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:19:18.605 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.605 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:19:18.605 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.605 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:19:18.605 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:19:18.605 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:19:18.605 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:19:18.605 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.605 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:19:18.605 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:19:18.605 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:19:18.605 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.605 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.605 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:19:18.605 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:19:18.605 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:19:18.605 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:19:18.606 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:19:18.606 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.606 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:19:18.606 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:19:18.606 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:19:18.606 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:19:18.606 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:19:18.606 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.606 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.606 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:19:18.606 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:19:18.606 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:19:18.606 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:19:18.606 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.606 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:19:18.607 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:19:18.607 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:19:18.607 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.607 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.607 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:19:18.607 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:19:18.607 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:19:18.607 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:19:18.607 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:19:18.607 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.607 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:19:18.607 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:19:18.607 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:19:18.607 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:19:18.607 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:19:18.608 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.608 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.608 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:19:18.608 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:19:18.608 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:19:18.608 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:19:18.608 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.608 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:19:18.608 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:19:18.608 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:19:18.608 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.608 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.608 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:19:18.608 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:19:18.608 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:19:18.609 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:19:18.609 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:19:18.609 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.609 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:19:18.609 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:19:18.609 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:19:18.609 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:19:18.609 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:19:18.609 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:19:18.609 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:19:18.609 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.609 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:19:18.609 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:19:18.609 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.609 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:19:18.610 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:19:18.610 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:19:18.610 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:19:18.610 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:19:18.610 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:19:18.610 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.610 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:19:18.610 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:19:18.610 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:19:18.610 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:19:18.610 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:19:18.610 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.610 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:19:18.610 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:19:18.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.598+0000 7f5f00764640 1 -- 172.21.15.76:0/1488565790 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5edc0743b0 msgr2=0x7f5edc0768c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.598+0000 7f5f00764640 1 --2- 172.21.15.76:0/1488565790 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5edc0743b0 0x7f5edc0768c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0034000 tx=0x7f5ef002e000 comp rx=0 tx=0).stop 2024-04-25T19:19:18.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.598+0000 7f5f00764640 1 -- 172.21.15.76:0/1488565790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ef806e870 msgr2=0x7f5ef81f2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.598+0000 7f5f00764640 1 --2- 172.21.15.76:0/1488565790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ef806e870 0x7f5ef81f2e10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5ef400cb50 tx=0x7f5ef400a430 comp rx=0 tx=0).stop 2024-04-25T19:19:18.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.598+0000 7f5efed01640 1 -- 172.21.15.76:0/1488565790 reap_dead start 2024-04-25T19:19:18.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.598+0000 7f5efed01640 1 -- 172.21.15.76:0/1488565790 reap_dead start 2024-04-25T19:19:18.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.598+0000 7f5f00764640 1 -- 172.21.15.76:0/1488565790 shutdown_connections 2024-04-25T19:19:18.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.598+0000 7f5f00764640 1 -- 172.21.15.76:0/1488565790 >> 172.21.15.76:0/1488565790 conn(0x7f5ef80b3c00 msgr2=0x7f5ef80d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:18.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.598+0000 7f5f00764640 1 -- 172.21.15.76:0/1488565790 shutdown_connections 2024-04-25T19:19:18.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.598+0000 7f5f00764640 1 -- 172.21.15.76:0/1488565790 wait complete. 2024-04-25T19:19:18.612 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T19:19:18.612 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-04-25T19:19:18.613 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-04-25T19:19:18.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.720+0000 7efc22630640 1 Processor -- start 2024-04-25T19:19:18.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.720+0000 7efc22630640 1 -- start start 2024-04-25T19:19:18.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.721+0000 7efc22630640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc1c153b70 0x7efc1c15d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.721+0000 7efc22630640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc1c153f80 con 0x7efc1c159840 2024-04-25T19:19:18.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.721+0000 7efc22630640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc1c0715d0 con 0x7efc1c0c5b20 2024-04-25T19:19:18.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.721+0000 7efc22630640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7efc1c15ce30 con 0x7efc1c153b70 2024-04-25T19:19:18.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.722+0000 7efc1bfff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7efc1c159840 0x7efc1c0c96c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47982/0 (socket says 172.21.15.76:47982) 2024-04-25T19:19:18.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.722+0000 7efc1bfff640 1 -- 172.21.15.76:0/2342687953 learned_addr learned my addr 172.21.15.76:0/2342687953 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:18.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.722+0000 7efc20bcd640 1 --2- 172.21.15.76:0/2342687953 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc1c153b70 0x7efc1c15d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.722+0000 7efc1affd640 1 -- 172.21.15.76:0/2342687953 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3990766308 0 0) 0x7efc1c153f80 con 0x7efc1c159840 2024-04-25T19:19:18.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.722+0000 7efc1affd640 1 -- 172.21.15.76:0/2342687953 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc04003520 con 0x7efc1c159840 2024-04-25T19:19:18.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.722+0000 7efc1affd640 1 -- 172.21.15.76:0/2342687953 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3441028746 0 0) 0x7efc1c0715d0 con 0x7efc1c0c5b20 2024-04-25T19:19:18.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.722+0000 7efc1affd640 1 -- 172.21.15.76:0/2342687953 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc1c153f80 con 0x7efc1c0c5b20 2024-04-25T19:19:18.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.722+0000 7efc20bcd640 1 -- 172.21.15.76:0/2342687953 >> v1:172.21.15.105:6789/0 conn(0x7efc1c0c5b20 legacy=0x7efc1c0c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.722+0000 7efc20bcd640 1 -- 172.21.15.76:0/2342687953 >> v1:172.21.15.76:6789/0 conn(0x7efc1c159840 legacy=0x7efc1c0c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.722+0000 7efc1affd640 1 -- 172.21.15.76:0/2342687953 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1476304078 0 0) 0x7efc04003520 con 0x7efc1c159840 2024-04-25T19:19:18.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.722+0000 7efc20bcd640 1 -- 172.21.15.76:0/2342687953 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc1c0718b0 con 0x7efc1c153b70 2024-04-25T19:19:18.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.723+0000 7efc20bcd640 1 --2- 172.21.15.76:0/2342687953 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc1c153b70 0x7efc1c15d880 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efc14009dc0 tx=0x7efc1402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e88e79bfa17ff089 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:18.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.723+0000 7efc1affd640 1 -- 172.21.15.76:0/2342687953 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0c001670 con 0x7efc1c153b70 2024-04-25T19:19:18.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.723+0000 7efc1affd640 1 -- 172.21.15.76:0/2342687953 <== mon.2 v2:172.21.15.105:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc14007540 con 0x7efc1c153b70 2024-04-25T19:19:18.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.723+0000 7efc1affd640 1 -- 172.21.15.76:0/2342687953 <== mon.2 v2:172.21.15.105:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc140078f0 con 0x7efc1c153b70 2024-04-25T19:19:18.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.724+0000 7efc22630640 1 -- 172.21.15.76:0/2342687953 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc1c153b70 msgr2=0x7efc1c15d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.724+0000 7efc22630640 1 --2- 172.21.15.76:0/2342687953 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc1c153b70 0x7efc1c15d880 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efc14009dc0 tx=0x7efc1402a660 comp rx=0 tx=0).stop 2024-04-25T19:19:18.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.724+0000 7efc22630640 1 -- 172.21.15.76:0/2342687953 shutdown_connections 2024-04-25T19:19:18.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.724+0000 7efc22630640 1 --2- 172.21.15.76:0/2342687953 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc1c153b70 0x7efc1c15d880 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.724+0000 7efc22630640 1 -- 172.21.15.76:0/2342687953 >> 172.21.15.76:0/2342687953 conn(0x7efc1c0cecc0 msgr2=0x7efc1c0cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:18.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.724+0000 7efc22630640 1 -- 172.21.15.76:0/2342687953 shutdown_connections 2024-04-25T19:19:18.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.724+0000 7efc22630640 1 -- 172.21.15.76:0/2342687953 wait complete. 2024-04-25T19:19:18.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.725+0000 7efc22630640 1 Processor -- start 2024-04-25T19:19:18.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.726+0000 7efc22630640 1 -- start start 2024-04-25T19:19:18.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.727+0000 7efc22630640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc1c0c5b20 0x7efc1c06d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.727+0000 7efc22630640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc1c153b70 0x7efc1c1f3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.727+0000 7efc22630640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc1c159840 0x7efc1c0ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.727+0000 7efc22630640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efc1c15d080 con 0x7efc1c0c5b20 2024-04-25T19:19:18.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.727+0000 7efc22630640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efc1c15ddc0 con 0x7efc1c159840 2024-04-25T19:19:18.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.727+0000 7efc22630640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7efc1c071270 con 0x7efc1c153b70 2024-04-25T19:19:18.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.727+0000 7efc1bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc1c0c5b20 0x7efc1c06d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.727+0000 7efc1bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc1c0c5b20 0x7efc1c06d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39622/0 (socket says 172.21.15.76:39622) 2024-04-25T19:19:18.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.727+0000 7efc1bfff640 1 -- 172.21.15.76:0/466711156 learned_addr learned my addr 172.21.15.76:0/466711156 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:18.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.727+0000 7efc1b7fe640 1 --2- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc1c153b70 0x7efc1c1f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.727+0000 7efc20bcd640 1 --2- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc1c159840 0x7efc1c0ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.727+0000 7efc1bfff640 1 -- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc1c153b70 msgr2=0x7efc1c1f3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.727+0000 7efc1bfff640 1 --2- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc1c153b70 0x7efc1c1f3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.727+0000 7efc1bfff640 1 -- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc1c159840 msgr2=0x7efc1c0ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.727+0000 7efc1bfff640 1 --2- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc1c159840 0x7efc1c0ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.727+0000 7efc1bfff640 1 -- 172.21.15.76:0/466711156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc1c1f3670 con 0x7efc1c0c5b20 2024-04-25T19:19:18.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.727+0000 7efc1b7fe640 1 --2- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc1c153b70 0x7efc1c1f3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:19:18.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.727+0000 7efc20bcd640 1 --2- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efc1c159840 0x7efc1c0ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:19:18.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.727+0000 7efc1bfff640 1 --2- 172.21.15.76:0/466711156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc1c0c5b20 0x7efc1c06d030 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efc0c00c520 tx=0x7efc0c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:18.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.728+0000 7efc18ff9640 1 -- 172.21.15.76:0/466711156 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0c017070 con 0x7efc1c0c5b20 2024-04-25T19:19:18.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.728+0000 7efc18ff9640 1 -- 172.21.15.76:0/466711156 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc0c005ce0 con 0x7efc1c0c5b20 2024-04-25T19:19:18.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.728+0000 7efc22630640 1 -- 172.21.15.76:0/466711156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc1c14df00 con 0x7efc1c0c5b20 2024-04-25T19:19:18.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.728+0000 7efc18ff9640 1 -- 172.21.15.76:0/466711156 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0c0072b0 con 0x7efc1c0c5b20 2024-04-25T19:19:18.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.728+0000 7efc22630640 1 -- 172.21.15.76:0/466711156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc1c15ca10 con 0x7efc1c0c5b20 2024-04-25T19:19:18.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.728+0000 7efc22630640 1 --2- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc1c15bf70 0x7efc1c14c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.728+0000 7efc22630640 1 -- 172.21.15.76:0/466711156 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7efc1c14e480 con 0x7efc1c15bf70 2024-04-25T19:19:18.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.729+0000 7efc1b7fe640 1 --2- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc1c15bf70 0x7efc1c14c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.729+0000 7efc1b7fe640 1 -- 172.21.15.76:0/466711156 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efc1c076950 con 0x7efc1c15bf70 2024-04-25T19:19:18.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.729+0000 7efc1b7fe640 1 --2- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc1c15bf70 0x7efc1c14c4d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efc1c06c5e0 tx=0x7efc1000ff70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:18.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.729+0000 7efc18ff9640 1 -- 172.21.15.76:0/466711156 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efc0c01a070 con 0x7efc1c0c5b20 2024-04-25T19:19:18.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.730+0000 7efc18ff9640 1 --2- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7efc080744b0 0x7efc080769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.730+0000 7efc18ff9640 1 -- 172.21.15.76:0/466711156 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc100052e0 con 0x7efc1c15bf70 2024-04-25T19:19:18.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.730+0000 7efc18ff9640 1 -- 172.21.15.76:0/466711156 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7efc0c0945f0 con 0x7efc1c0c5b20 2024-04-25T19:19:18.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.730+0000 7efc20bcd640 1 --2- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7efc080744b0 0x7efc080769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.730+0000 7efc18ff9640 1 -- 172.21.15.76:0/466711156 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7efc10018070 con 0x7efc1c15bf70 2024-04-25T19:19:18.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.730+0000 7efc18ff9640 1 -- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc1c15bf70 msgr2=0x7efc1c14c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.730+0000 7efc18ff9640 1 --2- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc1c15bf70 0x7efc1c14c4d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efc1c06c5e0 tx=0x7efc1000ff70 comp rx=0 tx=0).stop 2024-04-25T19:19:18.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.730+0000 7efc18ff9640 1 -- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc1c15bf70 msgr2=0x7efc1c14c4d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:19:18.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.730+0000 7efc18ff9640 1 --2- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc1c15bf70 0x7efc1c14c4d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.730+0000 7efc20bcd640 1 --2- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7efc080744b0 0x7efc080769c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efc14034000 tx=0x7efc1402f000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:18.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.736+0000 7efc22630640 1 --2- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc1c15c380 0x7efc1c14ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.736+0000 7efc22630640 1 -- 172.21.15.76:0/466711156 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7efc1c14cf50 con 0x7efc1c15c380 2024-04-25T19:19:18.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.737+0000 7efc1b7fe640 1 --2- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc1c15c380 0x7efc1c14ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.737+0000 7efc1b7fe640 1 -- 172.21.15.76:0/466711156 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7efc1c06a390 con 0x7efc1c15c380 2024-04-25T19:19:18.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.737+0000 7efc1b7fe640 1 --2- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc1c15c380 0x7efc1c14ca10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efc1c06c5e0 tx=0x7efc10001150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:18.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.737+0000 7efc18ff9640 1 -- 172.21.15.76:0/466711156 <== mon.2 v2:172.21.15.105:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1000dce0 con 0x7efc1c15c380 2024-04-25T19:19:18.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.738+0000 7efc18ff9640 1 -- 172.21.15.76:0/466711156 <== mon.2 v2:172.21.15.105:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5065 (secure 0 0 0) 0x7efc1000dce0 con 0x7efc1c15c380 2024-04-25T19:19:18.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.738+0000 7efc18ff9640 1 -- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc1c15c380 msgr2=0x7efc1c14ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.738+0000 7efc18ff9640 1 --2- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc1c15c380 0x7efc1c14ca10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efc1c06c5e0 tx=0x7efc10001150 comp rx=0 tx=0).stop 2024-04-25T19:19:18.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.738+0000 7efc18ff9640 1 -- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc1c15c380 msgr2=0x7efc1c14ca10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-04-25T19:19:18.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.738+0000 7efc18ff9640 1 --2- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efc1c15c380 0x7efc1c14ca10 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efc1c06c5e0 tx=0x7efc10001150 comp rx=0 tx=0).stop 2024-04-25T19:19:18.740 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:19:18.740 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:19:18.740 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:19:18.740 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "peon", 2024-04-25T19:19:18.741 INFO:teuthology.orchestra.run.smithi076.stdout: "election_epoch": 30, 2024-04-25T19:19:18.741 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 162515, 2024-04-25T19:19:18.741 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum": [ 2024-04-25T19:19:18.741 INFO:teuthology.orchestra.run.smithi076.stdout: 0, 2024-04-25T19:19:18.741 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-04-25T19:19:18.741 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-04-25T19:19:18.741 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:19:18.741 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_age": 0, 2024-04-25T19:19:18.741 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:19:18.741 INFO:teuthology.orchestra.run.smithi076.stdout: "required_con": "2450521708007866372", 2024-04-25T19:19:18.741 INFO:teuthology.orchestra.run.smithi076.stdout: "required_mon": [ 2024-04-25T19:19:18.741 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:19:18.741 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:19:18.741 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:19:18.742 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:19:18.742 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:19:18.742 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:19:18.742 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:19:18.742 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:19:18.742 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:19:18.742 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:19:18.742 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:19:18.742 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:19:18.742 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_con": "4540701547738038271", 2024-04-25T19:19:18.742 INFO:teuthology.orchestra.run.smithi076.stdout: "quorum_mon": [ 2024-04-25T19:19:18.742 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:19:18.742 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:19:18.743 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:19:18.743 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:19:18.743 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:19:18.743 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:19:18.743 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:19:18.743 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:19:18.743 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:19:18.743 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:19:18.743 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:19:18.743 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:19:18.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.743 INFO:teuthology.orchestra.run.smithi076.stdout: "outside_quorum": [], 2024-04-25T19:19:18.743 INFO:teuthology.orchestra.run.smithi076.stdout: "extra_probe_peers": [], 2024-04-25T19:19:18.743 INFO:teuthology.orchestra.run.smithi076.stdout: "sync_provider": [], 2024-04-25T19:19:18.743 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": { 2024-04-25T19:19:18.743 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 1, 2024-04-25T19:19:18.743 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "d5adf865-03c7-4588-a151-044ededbff0b", 2024-04-25T19:19:18.744 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:19:18.744 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-04-25T19:14:27.121750Z", 2024-04-25T19:19:18.744 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release": 19, 2024-04-25T19:19:18.744 INFO:teuthology.orchestra.run.smithi076.stdout: "min_mon_release_name": "squid", 2024-04-25T19:19:18.744 INFO:teuthology.orchestra.run.smithi076.stdout: "election_strategy": 1, 2024-04-25T19:19:18.744 INFO:teuthology.orchestra.run.smithi076.stdout: "disallowed_leaders: ": "", 2024-04-25T19:19:18.744 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false, 2024-04-25T19:19:18.744 INFO:teuthology.orchestra.run.smithi076.stdout: "tiebreaker_mon": "", 2024-04-25T19:19:18.744 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_ranks: ": "", 2024-04-25T19:19:18.744 INFO:teuthology.orchestra.run.smithi076.stdout: "features": { 2024-04-25T19:19:18.744 INFO:teuthology.orchestra.run.smithi076.stdout: "persistent": [ 2024-04-25T19:19:18.744 INFO:teuthology.orchestra.run.smithi076.stdout: "kraken", 2024-04-25T19:19:18.744 INFO:teuthology.orchestra.run.smithi076.stdout: "luminous", 2024-04-25T19:19:18.744 INFO:teuthology.orchestra.run.smithi076.stdout: "mimic", 2024-04-25T19:19:18.744 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap-prune", 2024-04-25T19:19:18.745 INFO:teuthology.orchestra.run.smithi076.stdout: "nautilus", 2024-04-25T19:19:18.745 INFO:teuthology.orchestra.run.smithi076.stdout: "octopus", 2024-04-25T19:19:18.745 INFO:teuthology.orchestra.run.smithi076.stdout: "pacific", 2024-04-25T19:19:18.745 INFO:teuthology.orchestra.run.smithi076.stdout: "elector-pinging", 2024-04-25T19:19:18.745 INFO:teuthology.orchestra.run.smithi076.stdout: "quincy", 2024-04-25T19:19:18.745 INFO:teuthology.orchestra.run.smithi076.stdout: "reef", 2024-04-25T19:19:18.745 INFO:teuthology.orchestra.run.smithi076.stdout: "squid" 2024-04-25T19:19:18.745 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:19:18.745 INFO:teuthology.orchestra.run.smithi076.stdout: "optional": [] 2024-04-25T19:19:18.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.745 INFO:teuthology.orchestra.run.smithi076.stdout: "mons": [ 2024-04-25T19:19:18.745 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.745 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 0, 2024-04-25T19:19:18.745 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "a", 2024-04-25T19:19:18.745 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:19:18.746 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:19:18.746 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.746 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:19:18.746 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:3300", 2024-04-25T19:19:18.746 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:19:18.746 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.746 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.746 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:19:18.746 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789", 2024-04-25T19:19:18.746 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:19:18.746 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:19:18.746 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:19:18.746 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.746 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6789/0", 2024-04-25T19:19:18.746 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6789/0", 2024-04-25T19:19:18.747 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:19:18.747 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:19:18.747 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:19:18.747 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.747 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.747 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 1, 2024-04-25T19:19:18.747 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "b", 2024-04-25T19:19:18.747 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:19:18.747 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:19:18.747 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.747 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:19:18.747 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3300", 2024-04-25T19:19:18.747 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:19:18.747 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.747 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.747 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:19:18.748 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789", 2024-04-25T19:19:18.748 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:19:18.748 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:19:18.748 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:19:18.748 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.748 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6789/0", 2024-04-25T19:19:18.748 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6789/0", 2024-04-25T19:19:18.748 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:19:18.748 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:19:18.748 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:19:18.748 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.748 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.748 INFO:teuthology.orchestra.run.smithi076.stdout: "rank": 2, 2024-04-25T19:19:18.748 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "c", 2024-04-25T19:19:18.748 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-04-25T19:19:18.749 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-04-25T19:19:18.749 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.749 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-04-25T19:19:18.749 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:3301", 2024-04-25T19:19:18.749 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:19:18.749 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.749 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.749 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-04-25T19:19:18.749 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790", 2024-04-25T19:19:18.749 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 0 2024-04-25T19:19:18.749 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:19:18.749 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:19:18.749 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.749 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.105:6790/0", 2024-04-25T19:19:18.749 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.105:6790/0", 2024-04-25T19:19:18.749 INFO:teuthology.orchestra.run.smithi076.stdout: "priority": 0, 2024-04-25T19:19:18.749 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0, 2024-04-25T19:19:18.750 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "{}" 2024-04-25T19:19:18.750 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:19:18.750 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:19:18.750 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.750 INFO:teuthology.orchestra.run.smithi076.stdout: "feature_map": { 2024-04-25T19:19:18.750 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": [ 2024-04-25T19:19:18.750 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.750 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:19:18.750 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:19:18.750 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 1 2024-04-25T19:19:18.750 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:19:18.750 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:19:18.750 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": [ 2024-04-25T19:19:18.750 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.751 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:19:18.751 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:19:18.751 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 4 2024-04-25T19:19:18.751 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:19:18.751 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:19:18.751 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": [ 2024-04-25T19:19:18.751 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.751 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:19:18.751 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:19:18.751 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 7 2024-04-25T19:19:18.751 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:19:18.751 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:19:18.751 INFO:teuthology.orchestra.run.smithi076.stdout: "client": [ 2024-04-25T19:19:18.751 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.751 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:19:18.751 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:19:18.752 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 6 2024-04-25T19:19:18.752 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:19:18.752 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-04-25T19:19:18.752 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": [ 2024-04-25T19:19:18.752 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-04-25T19:19:18.752 INFO:teuthology.orchestra.run.smithi076.stdout: "features": "0x3f03cffffffdffff", 2024-04-25T19:19:18.752 INFO:teuthology.orchestra.run.smithi076.stdout: "release": "luminous", 2024-04-25T19:19:18.752 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 2 2024-04-25T19:19:18.752 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-04-25T19:19:18.752 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-04-25T19:19:18.752 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-04-25T19:19:18.752 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": false 2024-04-25T19:19:18.752 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:19:18.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.738+0000 7efc22630640 1 -- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7efc080744b0 msgr2=0x7efc080769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.738+0000 7efc22630640 1 --2- 172.21.15.76:0/466711156 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7efc080744b0 0x7efc080769c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efc14034000 tx=0x7efc1402f000 comp rx=0 tx=0).stop 2024-04-25T19:19:18.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.738+0000 7efc22630640 1 -- 172.21.15.76:0/466711156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc1c0c5b20 msgr2=0x7efc1c06d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.738+0000 7efc22630640 1 --2- 172.21.15.76:0/466711156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc1c0c5b20 0x7efc1c06d030 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efc0c00c520 tx=0x7efc0c00c9e0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.738+0000 7efc20bcd640 1 -- 172.21.15.76:0/466711156 reap_dead start 2024-04-25T19:19:18.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.738+0000 7efc20bcd640 1 -- 172.21.15.76:0/466711156 reap_dead start 2024-04-25T19:19:18.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.738+0000 7efc22630640 1 -- 172.21.15.76:0/466711156 shutdown_connections 2024-04-25T19:19:18.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.738+0000 7efc22630640 1 -- 172.21.15.76:0/466711156 >> 172.21.15.76:0/466711156 conn(0x7efc1c0cecc0 msgr2=0x7efc1c158660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:18.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.738+0000 7efc22630640 1 -- 172.21.15.76:0/466711156 shutdown_connections 2024-04-25T19:19:18.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.738+0000 7efc22630640 1 -- 172.21.15.76:0/466711156 wait complete. 2024-04-25T19:19:18.754 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-04-25T19:19:18.755 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-04-25T19:19:18.755 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-04-25T19:19:18.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.862+0000 7f02a85e3640 1 Processor -- start 2024-04-25T19:19:18.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.862+0000 7f02a85e3640 1 -- start start 2024-04-25T19:19:18.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.863+0000 7f02a85e3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f02a00714c0 0x7f02a006c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.863+0000 7f02a85e3640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02a006d160 con 0x7f02a00c7e20 2024-04-25T19:19:18.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.863+0000 7f02a85e3640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02a015d660 con 0x7f02a00c9730 2024-04-25T19:19:18.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.863+0000 7f02a85e3640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f02a0071fa0 con 0x7f02a00714c0 2024-04-25T19:19:18.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.863+0000 7f02a6b80640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f02a00c7e20 0x7f02a01673e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47990/0 (socket says 172.21.15.76:47990) 2024-04-25T19:19:18.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.863+0000 7f02a6b80640 1 -- 172.21.15.76:0/707785930 learned_addr learned my addr 172.21.15.76:0/707785930 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:18.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.864+0000 7f02a637f640 1 --2- 172.21.15.76:0/707785930 >> v2:172.21.15.105:3300/0 conn(0x7f02a00714c0 0x7f02a006c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.864+0000 7f02a537d640 1 -- 172.21.15.76:0/707785930 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2398334212 0 0) 0x7f02a006d160 con 0x7f02a00c7e20 2024-04-25T19:19:18.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.864+0000 7f02a537d640 1 -- 172.21.15.76:0/707785930 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f028c003520 con 0x7f02a00c7e20 2024-04-25T19:19:18.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.864+0000 7f02a537d640 1 -- 172.21.15.76:0/707785930 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3009433215 0 0) 0x7f028c003520 con 0x7f02a00c7e20 2024-04-25T19:19:18.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.864+0000 7f02a637f640 1 -- 172.21.15.76:0/707785930 >> v1:172.21.15.105:6789/0 conn(0x7f02a00c9730 legacy=0x7f02a00c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.864+0000 7f02a637f640 1 -- 172.21.15.76:0/707785930 >> v1:172.21.15.76:6789/0 conn(0x7f02a00c7e20 legacy=0x7f02a01673e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.864+0000 7f02a637f640 1 -- 172.21.15.76:0/707785930 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a015d940 con 0x7f02a00714c0 2024-04-25T19:19:18.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.865+0000 7f02a637f640 1 --2- 172.21.15.76:0/707785930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02a00714c0 0x7f02a006c5c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f029c0097a0 tx=0x7f029c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc427b84707056b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:18.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.865+0000 7f02a537d640 1 -- 172.21.15.76:0/707785930 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c037070 con 0x7f02a00714c0 2024-04-25T19:19:18.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.865+0000 7f02a537d640 1 -- 172.21.15.76:0/707785930 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f029c02ac50 con 0x7f02a00714c0 2024-04-25T19:19:18.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.865+0000 7f02a537d640 1 -- 172.21.15.76:0/707785930 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02980019f0 con 0x7f02a00714c0 2024-04-25T19:19:18.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.866+0000 7f02a85e3640 1 -- 172.21.15.76:0/707785930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02a00714c0 msgr2=0x7f02a006c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.866+0000 7f02a85e3640 1 --2- 172.21.15.76:0/707785930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02a00714c0 0x7f02a006c5c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f029c0097a0 tx=0x7f029c02a1c0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.866+0000 7f02a85e3640 1 -- 172.21.15.76:0/707785930 shutdown_connections 2024-04-25T19:19:18.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.866+0000 7f02a85e3640 1 --2- 172.21.15.76:0/707785930 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02a00714c0 0x7f02a006c5c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.866+0000 7f02a85e3640 1 -- 172.21.15.76:0/707785930 >> 172.21.15.76:0/707785930 conn(0x7f02a006d6d0 msgr2=0x7f02a006dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:18.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.866+0000 7f02a85e3640 1 -- 172.21.15.76:0/707785930 shutdown_connections 2024-04-25T19:19:18.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.866+0000 7f02a85e3640 1 -- 172.21.15.76:0/707785930 wait complete. 2024-04-25T19:19:18.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.867+0000 7f02a85e3640 1 Processor -- start 2024-04-25T19:19:18.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.867+0000 7f02a85e3640 1 -- start start 2024-04-25T19:19:18.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.868+0000 7f02a85e3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02a00c7e20 0x7f02a01f6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.869+0000 7f02a85e3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02a00c9730 0x7f02a014be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.869+0000 7f02a85e3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02a00cbe00 0x7f02a01e8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.869+0000 7f02a85e3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f02a00cc6f0 con 0x7f02a00c7e20 2024-04-25T19:19:18.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.869+0000 7f02a85e3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f02a00cc570 con 0x7f02a00c9730 2024-04-25T19:19:18.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.869+0000 7f02a85e3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f02a0156bf0 con 0x7f02a00cbe00 2024-04-25T19:19:18.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.869+0000 7f02a637f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02a00c7e20 0x7f02a01f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.869+0000 7f02a637f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02a00c7e20 0x7f02a01f6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39628/0 (socket says 172.21.15.76:39628) 2024-04-25T19:19:18.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.869+0000 7f02a637f640 1 -- 172.21.15.76:0/4138535810 learned_addr learned my addr 172.21.15.76:0/4138535810 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:18.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.869+0000 7f02a5b7e640 1 --2- 172.21.15.76:0/4138535810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02a00c9730 0x7f02a014be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.869+0000 7f02a6b80640 1 --2- 172.21.15.76:0/4138535810 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02a00cbe00 0x7f02a01e8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.869+0000 7f02a637f640 1 -- 172.21.15.76:0/4138535810 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02a00cbe00 msgr2=0x7f02a01e8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.869+0000 7f02a637f640 1 --2- 172.21.15.76:0/4138535810 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02a00cbe00 0x7f02a01e8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.869+0000 7f02a637f640 1 -- 172.21.15.76:0/4138535810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02a00c9730 msgr2=0x7f02a014be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:18.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.869+0000 7f02a637f640 1 --2- 172.21.15.76:0/4138535810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02a00c9730 0x7f02a014be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:18.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.869+0000 7f02a637f640 1 -- 172.21.15.76:0/4138535810 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a01f65c0 con 0x7f02a00c7e20 2024-04-25T19:19:18.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.869+0000 7f02a6b80640 1 --2- 172.21.15.76:0/4138535810 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02a00cbe00 0x7f02a01e8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:19:18.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.869+0000 7f02a5b7e640 1 --2- 172.21.15.76:0/4138535810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02a00c9730 0x7f02a014be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:19:18.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.869+0000 7f02a637f640 1 --2- 172.21.15.76:0/4138535810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02a00c7e20 0x7f02a01f6080 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f029c035040 tx=0x7f029c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:18.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.869+0000 7f0296ffd640 1 -- 172.21.15.76:0/4138535810 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c042020 con 0x7f02a00c7e20 2024-04-25T19:19:18.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.869+0000 7f0296ffd640 1 -- 172.21.15.76:0/4138535810 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f029c03d470 con 0x7f02a00c7e20 2024-04-25T19:19:18.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.869+0000 7f02a85e3640 1 -- 172.21.15.76:0/4138535810 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a00cb150 con 0x7f02a00c7e20 2024-04-25T19:19:18.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.869+0000 7f0296ffd640 1 -- 172.21.15.76:0/4138535810 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c037040 con 0x7f02a00c7e20 2024-04-25T19:19:18.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.869+0000 7f02a85e3640 1 -- 172.21.15.76:0/4138535810 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a00ca910 con 0x7f02a00c7e20 2024-04-25T19:19:18.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.870+0000 7f02a85e3640 1 -- 172.21.15.76:0/4138535810 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02a01f6790 con 0x7f02a00c7e20 2024-04-25T19:19:18.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.871+0000 7f0296ffd640 1 -- 172.21.15.76:0/4138535810 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f029c045020 con 0x7f02a00c7e20 2024-04-25T19:19:18.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.871+0000 7f0296ffd640 1 --2- 172.21.15.76:0/4138535810 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f02840744b0 0x7f02840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:18.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.872+0000 7f0296ffd640 1 -- 172.21.15.76:0/4138535810 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f029c0b5a60 con 0x7f02a00c7e20 2024-04-25T19:19:18.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.872+0000 7f02a5b7e640 1 --2- 172.21.15.76:0/4138535810 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f02840744b0 0x7f02840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:18.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.872+0000 7f02a5b7e640 1 --2- 172.21.15.76:0/4138535810 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f02840744b0 0x7f02840769c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f02a01f06c0 tx=0x7f0290009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:18.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:18.876+0000 7f0296ffd640 1 -- 172.21.15.76:0/4138535810 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f029c04c030 con 0x7f02a00c7e20 2024-04-25T19:19:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:19.118+0000 7f02a85e3640 1 -- 172.21.15.76:0/4138535810 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f02a006a390 con 0x7f02a00c7e20 2024-04-25T19:19:19.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:19.119+0000 7f0296ffd640 1 -- 172.21.15.76:0/4138535810 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f029c080170 con 0x7f02a00c7e20 2024-04-25T19:19:19.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:19.121+0000 7f02a85e3640 1 -- 172.21.15.76:0/4138535810 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f02840744b0 msgr2=0x7f02840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:19.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:19.121+0000 7f02a85e3640 1 --2- 172.21.15.76:0/4138535810 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f02840744b0 0x7f02840769c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f02a01f06c0 tx=0x7f0290009040 comp rx=0 tx=0).stop 2024-04-25T19:19:19.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:19.121+0000 7f02a85e3640 1 -- 172.21.15.76:0/4138535810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02a00c7e20 msgr2=0x7f02a01f6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:19.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:19.121+0000 7f02a85e3640 1 --2- 172.21.15.76:0/4138535810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02a00c7e20 0x7f02a01f6080 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f029c035040 tx=0x7f029c0046c0 comp rx=0 tx=0).stop 2024-04-25T19:19:19.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:19.122+0000 7f02a85e3640 1 -- 172.21.15.76:0/4138535810 shutdown_connections 2024-04-25T19:19:19.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:19.122+0000 7f02a85e3640 1 --2- 172.21.15.76:0/4138535810 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f02840744b0 0x7f02840769c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:19.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:19.122+0000 7f02a85e3640 1 --2- 172.21.15.76:0/4138535810 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02a00cbe00 0x7f02a01e8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:19.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:19.122+0000 7f02a85e3640 1 --2- 172.21.15.76:0/4138535810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02a00c9730 0x7f02a014be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:19.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:19.122+0000 7f02a85e3640 1 --2- 172.21.15.76:0/4138535810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02a00c7e20 0x7f02a01f6080 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:19.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:19.122+0000 7f02a85e3640 1 -- 172.21.15.76:0/4138535810 >> 172.21.15.76:0/4138535810 conn(0x7f02a006d6d0 msgr2=0x7f02a00c8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:19.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:19.122+0000 7f02a85e3640 1 -- 172.21.15.76:0/4138535810 shutdown_connections 2024-04-25T19:19:19.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:19.122+0000 7f02a85e3640 1 -- 172.21.15.76:0/4138535810 wait complete. 2024-04-25T19:19:19.137 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-04-25T19:19:26.990 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:19:26.993 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:19:29.138 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:19:29.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.258+0000 7f5d37fce640 1 Processor -- start 2024-04-25T19:19:29.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.258+0000 7f5d37fce640 1 -- start start 2024-04-25T19:19:29.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.259+0000 7f5d37fce640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d300c7750 0x7f5d30153590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.260+0000 7f5d37fce640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d300c8020 con 0x7f5d300c7b60 2024-04-25T19:19:29.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.260+0000 7f5d37fce640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d301598a0 con 0x7f5d3015cd80 2024-04-25T19:19:29.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.260+0000 7f5d37fce640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f5d30159b40 con 0x7f5d300c7750 2024-04-25T19:19:29.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.260+0000 7f5d35569640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f5d300c7b60 0x7f5d300cdd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59220/0 (socket says 172.21.15.76:59220) 2024-04-25T19:19:29.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.260+0000 7f5d35569640 1 -- 172.21.15.76:0/1019226869 learned_addr learned my addr 172.21.15.76:0/1019226869 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:29.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.260+0000 7f5d35d6a640 1 --2- 172.21.15.76:0/1019226869 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d300c7750 0x7f5d30153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.260+0000 7f5d34d68640 1 -- 172.21.15.76:0/1019226869 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2136875173 0 0) 0x7f5d300c8020 con 0x7f5d300c7b60 2024-04-25T19:19:29.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.260+0000 7f5d34d68640 1 -- 172.21.15.76:0/1019226869 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d1c003520 con 0x7f5d300c7b60 2024-04-25T19:19:29.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.260+0000 7f5d34d68640 1 -- 172.21.15.76:0/1019226869 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3526712425 0 0) 0x7f5d301598a0 con 0x7f5d3015cd80 2024-04-25T19:19:29.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.260+0000 7f5d34d68640 1 -- 172.21.15.76:0/1019226869 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d300c8020 con 0x7f5d3015cd80 2024-04-25T19:19:29.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.260+0000 7f5d34d68640 1 -- 172.21.15.76:0/1019226869 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1594090937 0 0) 0x7f5d1c003520 con 0x7f5d300c7b60 2024-04-25T19:19:29.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.260+0000 7f5d34d68640 1 -- 172.21.15.76:0/1019226869 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d300c7750 msgr2=0x7f5d30153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:29.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.260+0000 7f5d34d68640 1 --2- 172.21.15.76:0/1019226869 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d300c7750 0x7f5d30153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.260+0000 7f5d34d68640 1 -- 172.21.15.76:0/1019226869 >> v1:172.21.15.105:6789/0 conn(0x7f5d3015cd80 legacy=0x7f5d3015d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:29.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.260+0000 7f5d34d68640 1 -- 172.21.15.76:0/1019226869 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d30159e20 con 0x7f5d300c7b60 2024-04-25T19:19:29.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.260+0000 7f5d34d68640 1 -- 172.21.15.76:0/1019226869 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f5d20001670 con 0x7f5d300c7b60 2024-04-25T19:19:29.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.260+0000 7f5d34d68640 1 -- 172.21.15.76:0/1019226869 >> v1:172.21.15.76:6789/0 conn(0x7f5d300c7b60 legacy=0x7f5d300cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:29.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.260+0000 7f5d34d68640 1 --2- 172.21.15.76:0/1019226869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d1c003b70 0x7f5d1c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.260+0000 7f5d34d68640 1 --2- 172.21.15.76:0/1019226869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d1c004800 0x7f5d1c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.260+0000 7f5d34d68640 1 --2- 172.21.15.76:0/1019226869 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d1c0051a0 0x7f5d1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.261+0000 7f5d34d68640 1 -- 172.21.15.76:0/1019226869 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5d30159b40 con 0x7f5d1c003b70 2024-04-25T19:19:29.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.261+0000 7f5d34d68640 1 -- 172.21.15.76:0/1019226869 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1c00c090 con 0x7f5d1c004800 2024-04-25T19:19:29.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.261+0000 7f5d34d68640 1 -- 172.21.15.76:0/1019226869 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f5d1c00c290 con 0x7f5d1c0051a0 2024-04-25T19:19:29.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.261+0000 7f5d35d6a640 1 --2- 172.21.15.76:0/1019226869 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d300c7750 0x7f5d30153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:19:29.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.261+0000 7f5d35569640 1 --2- 172.21.15.76:0/1019226869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d1c004800 0x7f5d1c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.261+0000 7f5d3656b640 1 --2- 172.21.15.76:0/1019226869 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d1c0051a0 0x7f5d1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.261+0000 7f5d35d6a640 1 --2- 172.21.15.76:0/1019226869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d1c003b70 0x7f5d1c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.261+0000 7f5d35d6a640 1 -- 172.21.15.76:0/1019226869 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d1c0051a0 msgr2=0x7f5d1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:29.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.261+0000 7f5d35d6a640 1 --2- 172.21.15.76:0/1019226869 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d1c0051a0 0x7f5d1c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.261+0000 7f5d35d6a640 1 -- 172.21.15.76:0/1019226869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d1c004800 msgr2=0x7f5d1c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:29.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.261+0000 7f5d35d6a640 1 --2- 172.21.15.76:0/1019226869 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d1c004800 0x7f5d1c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.261+0000 7f5d35d6a640 1 -- 172.21.15.76:0/1019226869 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d1c003520 con 0x7f5d1c003b70 2024-04-25T19:19:29.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.262+0000 7f5d35d6a640 1 --2- 172.21.15.76:0/1019226869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d1c003b70 0x7f5d1c003fc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5d30071f20 tx=0x7f5d1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1698c1c2b3786da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:29.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.262+0000 7f5d34d68640 1 -- 172.21.15.76:0/1019226869 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c036070 con 0x7f5d1c003b70 2024-04-25T19:19:29.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.262+0000 7f5d34d68640 1 -- 172.21.15.76:0/1019226869 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d2c004030 con 0x7f5d1c003b70 2024-04-25T19:19:29.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.262+0000 7f5d34d68640 1 -- 172.21.15.76:0/1019226869 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c0097f0 con 0x7f5d1c003b70 2024-04-25T19:19:29.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.262+0000 7f5d37fce640 1 -- 172.21.15.76:0/1019226869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d1c003b70 msgr2=0x7f5d1c003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:29.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.262+0000 7f5d37fce640 1 --2- 172.21.15.76:0/1019226869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d1c003b70 0x7f5d1c003fc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5d30071f20 tx=0x7f5d1c005830 comp rx=0 tx=0).stop 2024-04-25T19:19:29.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.263+0000 7f5d3656b640 1 -- 172.21.15.76:0/1019226869 reap_dead start 2024-04-25T19:19:29.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.263+0000 7f5d37fce640 1 -- 172.21.15.76:0/1019226869 shutdown_connections 2024-04-25T19:19:29.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.263+0000 7f5d37fce640 1 -- 172.21.15.76:0/1019226869 >> 172.21.15.76:0/1019226869 conn(0x7f5d30070800 msgr2=0x7f5d30070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:29.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.263+0000 7f5d37fce640 1 -- 172.21.15.76:0/1019226869 shutdown_connections 2024-04-25T19:19:29.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.263+0000 7f5d37fce640 1 -- 172.21.15.76:0/1019226869 wait complete. 2024-04-25T19:19:29.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.264+0000 7f5d37fce640 1 Processor -- start 2024-04-25T19:19:29.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.264+0000 7f5d37fce640 1 -- start start 2024-04-25T19:19:29.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.266+0000 7f5d37fce640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d1c003b70 0x7f5d3014a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.266+0000 7f5d37fce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d301f1640 0x7f5d301f1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.266+0000 7f5d37fce640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d30148e10 0x7f5d30149220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.266+0000 7f5d37fce640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5d300c8520 con 0x7f5d30148e10 2024-04-25T19:19:29.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.266+0000 7f5d37fce640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3015a3e0 con 0x7f5d301f1640 2024-04-25T19:19:29.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.266+0000 7f5d37fce640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f5d300c86a0 con 0x7f5d1c003b70 2024-04-25T19:19:29.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.266+0000 7f5d3656b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d30148e10 0x7f5d30149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.266+0000 7f5d3656b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d30148e10 0x7f5d30149220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40304/0 (socket says 172.21.15.76:40304) 2024-04-25T19:19:29.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.266+0000 7f5d3656b640 1 -- 172.21.15.76:0/2018918148 learned_addr learned my addr 172.21.15.76:0/2018918148 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:29.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.266+0000 7f5d35d6a640 1 --2- 172.21.15.76:0/2018918148 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d1c003b70 0x7f5d3014a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.266+0000 7f5d35569640 1 --2- 172.21.15.76:0/2018918148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d301f1640 0x7f5d301f1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.266+0000 7f5d3656b640 1 -- 172.21.15.76:0/2018918148 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d1c003b70 msgr2=0x7f5d3014a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:29.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.266+0000 7f5d3656b640 1 --2- 172.21.15.76:0/2018918148 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d1c003b70 0x7f5d3014a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.266+0000 7f5d3656b640 1 -- 172.21.15.76:0/2018918148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d301f1640 msgr2=0x7f5d301f1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:29.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.266+0000 7f5d3656b640 1 --2- 172.21.15.76:0/2018918148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d301f1640 0x7f5d301f1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.266+0000 7f5d3656b640 1 -- 172.21.15.76:0/2018918148 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d30150480 con 0x7f5d30148e10 2024-04-25T19:19:29.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.266+0000 7f5d3656b640 1 --2- 172.21.15.76:0/2018918148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d30148e10 0x7f5d30149220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800f8d0 tx=0x7f5d2800fd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:29.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.266+0000 7f5d267fc640 1 -- 172.21.15.76:0/2018918148 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d28003b80 con 0x7f5d30148e10 2024-04-25T19:19:29.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.266+0000 7f5d267fc640 1 -- 172.21.15.76:0/2018918148 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d28018070 con 0x7f5d30148e10 2024-04-25T19:19:29.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.266+0000 7f5d267fc640 1 -- 172.21.15.76:0/2018918148 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d280050d0 con 0x7f5d30148e10 2024-04-25T19:19:29.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.266+0000 7f5d37fce640 1 -- 172.21.15.76:0/2018918148 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d301f2190 con 0x7f5d30148e10 2024-04-25T19:19:29.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.267+0000 7f5d37fce640 1 -- 172.21.15.76:0/2018918148 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3014fcd0 con 0x7f5d30148e10 2024-04-25T19:19:29.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.268+0000 7f5d37fce640 1 -- 172.21.15.76:0/2018918148 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d301f1f90 con 0x7f5d30148e10 2024-04-25T19:19:29.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.268+0000 7f5d267fc640 1 -- 172.21.15.76:0/2018918148 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5d28005270 con 0x7f5d30148e10 2024-04-25T19:19:29.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.269+0000 7f5d267fc640 1 --2- 172.21.15.76:0/2018918148 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5d140744e0 0x7f5d140769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.269+0000 7f5d267fc640 1 -- 172.21.15.76:0/2018918148 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f5d28094f10 con 0x7f5d30148e10 2024-04-25T19:19:29.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.269+0000 7f5d35d6a640 1 --2- 172.21.15.76:0/2018918148 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5d140744e0 0x7f5d140769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.269+0000 7f5d35d6a640 1 --2- 172.21.15.76:0/2018918148 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5d140744e0 0x7f5d140769f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5d30071f20 tx=0x7f5d2c008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:29.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.273+0000 7f5d267fc640 1 -- 172.21.15.76:0/2018918148 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5d2805f0f0 con 0x7f5d30148e10 2024-04-25T19:19:29.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.487+0000 7f5d37fce640 1 -- 172.21.15.76:0/2018918148 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d3006a390 con 0x7f5d30148e10 2024-04-25T19:19:29.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.488+0000 7f5d267fc640 1 -- 172.21.15.76:0/2018918148 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6915 (secure 0 0 0) 0x7f5d280633f0 con 0x7f5d30148e10 2024-04-25T19:19:29.489 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:19:29.489 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/3003966182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3003966182},{"type":"v1","addr":"172.21.15.76:6836","nonce":3003966182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T19:15:18.769449+0000","modified":"2024-04-25T19:15:48.500543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4284,"mds_1":4278,"mds_2":4241,"mds_3":4238,"mds_4":4235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4235":{"gid":4235,"name":"b","rank":4,"incarnation":15,"state":"up:active","state_seq":10,"addr":"172.21.15.105:6836/1809877606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1809877606},{"type":"v1","addr":"172.21.15.105:6836","nonce":1809877606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4238":{"gid":4238,"name":"f","rank":3,"incarnation":13,"state":"up:active","state_seq":9,"addr":"172.21.15.105:6838/2724142282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2724142282},{"type":"v1","addr":"172.21.15.105:6838","nonce":2724142282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4241":{"gid":4241,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.105:6839/1798232582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6837","nonce":1798232582},{"type":"v1","addr":"172.21.15.105:6839","nonce":1798232582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.76:6833/2101494685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2101494685},{"type":"v1","addr":"172.21.15.76:6833","nonce":2101494685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4284":{"gid":4284,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6837/4226277412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":4226277412},{"type":"v1","addr":"172.21.15.76:6837","nonce":4226277412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4235,4238,4241,4278,4284]},"id":1}]} 2024-04-25T19:19:29.490 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 16 2024-04-25T19:19:29.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.490+0000 7f5d37fce640 1 -- 172.21.15.76:0/2018918148 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5d140744e0 msgr2=0x7f5d140769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:29.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.490+0000 7f5d37fce640 1 --2- 172.21.15.76:0/2018918148 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5d140744e0 0x7f5d140769f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5d30071f20 tx=0x7f5d2c008030 comp rx=0 tx=0).stop 2024-04-25T19:19:29.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.490+0000 7f5d37fce640 1 -- 172.21.15.76:0/2018918148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d30148e10 msgr2=0x7f5d30149220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:29.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.490+0000 7f5d37fce640 1 --2- 172.21.15.76:0/2018918148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d30148e10 0x7f5d30149220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800f8d0 tx=0x7f5d2800fd90 comp rx=0 tx=0).stop 2024-04-25T19:19:29.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.490+0000 7f5d37fce640 1 -- 172.21.15.76:0/2018918148 shutdown_connections 2024-04-25T19:19:29.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.490+0000 7f5d37fce640 1 --2- 172.21.15.76:0/2018918148 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5d140744e0 0x7f5d140769f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.490+0000 7f5d37fce640 1 --2- 172.21.15.76:0/2018918148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d30148e10 0x7f5d30149220 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.490+0000 7f5d37fce640 1 --2- 172.21.15.76:0/2018918148 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d301f1640 0x7f5d301f1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.490+0000 7f5d37fce640 1 --2- 172.21.15.76:0/2018918148 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5d1c003b70 0x7f5d3014a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.490+0000 7f5d37fce640 1 -- 172.21.15.76:0/2018918148 >> 172.21.15.76:0/2018918148 conn(0x7f5d30070800 msgr2=0x7f5d300c97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:29.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.490+0000 7f5d37fce640 1 -- 172.21.15.76:0/2018918148 shutdown_connections 2024-04-25T19:19:29.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.490+0000 7f5d37fce640 1 -- 172.21.15.76:0/2018918148 wait complete. 2024-04-25T19:19:29.503 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-04-25T19:19:29.503 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-04-25T19:19:29.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.609+0000 7f20f0b4d640 1 Processor -- start 2024-04-25T19:19:29.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.609+0000 7f20f0b4d640 1 -- start start 2024-04-25T19:19:29.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.610+0000 7f20f0b4d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f20ec0c6040 0x7f20ec154c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.610+0000 7f20f0b4d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20ec0cb820 con 0x7f20ec0cab50 2024-04-25T19:19:29.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.610+0000 7f20f0b4d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20ec0cbac0 con 0x7f20ec0caf60 2024-04-25T19:19:29.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.610+0000 7f20f0b4d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f20ec0cc730 con 0x7f20ec0c6040 2024-04-25T19:19:29.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.610+0000 7f20e3fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f20ec0cab50 0x7f20ec0cd020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35690/0 (socket says 172.21.15.76:35690) 2024-04-25T19:19:29.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.610+0000 7f20e3fff640 1 -- 172.21.15.76:0/2700877641 learned_addr learned my addr 172.21.15.76:0/2700877641 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:29.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.610+0000 7f20ea59c640 1 --2- 172.21.15.76:0/2700877641 >> v2:172.21.15.105:3300/0 conn(0x7f20ec0c6040 0x7f20ec154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.610+0000 7f20e9d9b640 1 -- 172.21.15.76:0/2700877641 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3666236504 0 0) 0x7f20ec0cb820 con 0x7f20ec0cab50 2024-04-25T19:19:29.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.611+0000 7f20e9d9b640 1 -- 172.21.15.76:0/2700877641 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20d8003520 con 0x7f20ec0cab50 2024-04-25T19:19:29.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.611+0000 7f20ea59c640 1 -- 172.21.15.76:0/2700877641 >> v1:172.21.15.105:6789/0 conn(0x7f20ec0caf60 legacy=0x7f20ec15aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:29.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.611+0000 7f20ea59c640 1 -- 172.21.15.76:0/2700877641 >> v1:172.21.15.76:6789/0 conn(0x7f20ec0cab50 legacy=0x7f20ec0cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:29.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.611+0000 7f20e9d9b640 1 -- 172.21.15.76:0/2700877641 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 801608871 0 0) 0x7f20d8003520 con 0x7f20ec0cab50 2024-04-25T19:19:29.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.611+0000 7f20ea59c640 1 -- 172.21.15.76:0/2700877641 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ec0cbda0 con 0x7f20ec0c6040 2024-04-25T19:19:29.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.611+0000 7f20ea59c640 1 --2- 172.21.15.76:0/2700877641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20ec0c6040 0x7f20ec154c20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f20dc0097a0 tx=0x7f20dc02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb8f5325aefd9aa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:29.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.612+0000 7f20e9d9b640 1 -- 172.21.15.76:0/2700877641 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc037070 con 0x7f20ec0c6040 2024-04-25T19:19:29.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.612+0000 7f20e9d9b640 1 -- 172.21.15.76:0/2700877641 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20dc02ace0 con 0x7f20ec0c6040 2024-04-25T19:19:29.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.612+0000 7f20e9d9b640 1 -- 172.21.15.76:0/2700877641 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d4001690 con 0x7f20ec0c6040 2024-04-25T19:19:29.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.612+0000 7f20f0b4d640 1 -- 172.21.15.76:0/2700877641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20ec0c6040 msgr2=0x7f20ec154c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:29.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.612+0000 7f20f0b4d640 1 --2- 172.21.15.76:0/2700877641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20ec0c6040 0x7f20ec154c20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f20dc0097a0 tx=0x7f20dc02a250 comp rx=0 tx=0).stop 2024-04-25T19:19:29.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.612+0000 7f20f0b4d640 1 -- 172.21.15.76:0/2700877641 shutdown_connections 2024-04-25T19:19:29.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.612+0000 7f20f0b4d640 1 --2- 172.21.15.76:0/2700877641 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20ec0c6040 0x7f20ec154c20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.612+0000 7f20f0b4d640 1 -- 172.21.15.76:0/2700877641 >> 172.21.15.76:0/2700877641 conn(0x7f20ec072b60 msgr2=0x7f20ec072f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:29.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.612+0000 7f20f0b4d640 1 -- 172.21.15.76:0/2700877641 shutdown_connections 2024-04-25T19:19:29.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.612+0000 7f20f0b4d640 1 -- 172.21.15.76:0/2700877641 wait complete. 2024-04-25T19:19:29.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.614+0000 7f20f0b4d640 1 Processor -- start 2024-04-25T19:19:29.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.614+0000 7f20f0b4d640 1 -- start start 2024-04-25T19:19:29.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.615+0000 7f20f0b4d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20ec0c6040 0x7f20ec147ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.615+0000 7f20f0b4d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f20ec0cab50 0x7f20ec14e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.615+0000 7f20f0b4d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20ec0caf60 0x7f20ec1e7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.615+0000 7f20f0b4d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec0cd730 con 0x7f20ec0caf60 2024-04-25T19:19:29.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.615+0000 7f20f0b4d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec0cfe00 con 0x7f20ec0c6040 2024-04-25T19:19:29.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.615+0000 7f20f0b4d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f20ec0cd8b0 con 0x7f20ec0cab50 2024-04-25T19:19:29.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.616+0000 7f20ead9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20ec0caf60 0x7f20ec1e7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.616+0000 7f20ead9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20ec0caf60 0x7f20ec1e7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50370/0 (socket says 172.21.15.76:50370) 2024-04-25T19:19:29.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.616+0000 7f20ead9d640 1 -- 172.21.15.76:0/1820913726 learned_addr learned my addr 172.21.15.76:0/1820913726 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:29.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.616+0000 7f20ea59c640 1 --2- 172.21.15.76:0/1820913726 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20ec0c6040 0x7f20ec147ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.616+0000 7f20e3fff640 1 --2- 172.21.15.76:0/1820913726 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f20ec0cab50 0x7f20ec14e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.616+0000 7f20ead9d640 1 -- 172.21.15.76:0/1820913726 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f20ec0cab50 msgr2=0x7f20ec14e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:29.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.616+0000 7f20ead9d640 1 --2- 172.21.15.76:0/1820913726 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f20ec0cab50 0x7f20ec14e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.616+0000 7f20ead9d640 1 -- 172.21.15.76:0/1820913726 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20ec0c6040 msgr2=0x7f20ec147ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:29.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.616+0000 7f20ead9d640 1 --2- 172.21.15.76:0/1820913726 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20ec0c6040 0x7f20ec147ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.616+0000 7f20ead9d640 1 -- 172.21.15.76:0/1820913726 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ec148430 con 0x7f20ec0caf60 2024-04-25T19:19:29.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.616+0000 7f20ead9d640 1 --2- 172.21.15.76:0/1820913726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20ec0caf60 0x7f20ec1e7140 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f20e400c6b0 tx=0x7f20e400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:29.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.616+0000 7f20e2ffd640 1 -- 172.21.15.76:0/1820913726 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e4003b70 con 0x7f20ec0caf60 2024-04-25T19:19:29.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.616+0000 7f20e2ffd640 1 -- 172.21.15.76:0/1820913726 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20e4003d10 con 0x7f20ec0caf60 2024-04-25T19:19:29.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.616+0000 7f20f0b4d640 1 -- 172.21.15.76:0/1820913726 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20ec14ee40 con 0x7f20ec0caf60 2024-04-25T19:19:29.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.616+0000 7f20e2ffd640 1 -- 172.21.15.76:0/1820913726 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e4005310 con 0x7f20ec0caf60 2024-04-25T19:19:29.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.616+0000 7f20f0b4d640 1 -- 172.21.15.76:0/1820913726 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20ec1495a0 con 0x7f20ec0caf60 2024-04-25T19:19:29.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.617+0000 7f20f0b4d640 1 -- 172.21.15.76:0/1820913726 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20ec06a390 con 0x7f20ec0caf60 2024-04-25T19:19:29.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.618+0000 7f20e2ffd640 1 -- 172.21.15.76:0/1820913726 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f20e40054b0 con 0x7f20ec0caf60 2024-04-25T19:19:29.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.619+0000 7f20e2ffd640 1 --2- 172.21.15.76:0/1820913726 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f20cc0744e0 0x7f20cc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:29.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.619+0000 7f20e2ffd640 1 -- 172.21.15.76:0/1820913726 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f20e4094f80 con 0x7f20ec0caf60 2024-04-25T19:19:29.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.619+0000 7f20ea59c640 1 --2- 172.21.15.76:0/1820913726 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f20cc0744e0 0x7f20cc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:29.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.619+0000 7f20ea59c640 1 --2- 172.21.15.76:0/1820913726 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f20cc0744e0 0x7f20cc0769f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f20dc034000 tx=0x7f20dc02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:29.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.623+0000 7f20e2ffd640 1 -- 172.21.15.76:0/1820913726 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f20e405f040 con 0x7f20ec0caf60 2024-04-25T19:19:29.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.867+0000 7f20f0b4d640 1 -- 172.21.15.76:0/1820913726 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f20ec0c71e0 con 0x7f20ec0caf60 2024-04-25T19:19:29.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.868+0000 7f20e2ffd640 1 -- 172.21.15.76:0/1820913726 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1565 (secure 0 0 0) 0x7f20e4063340 con 0x7f20ec0caf60 2024-04-25T19:19:29.869 INFO:teuthology.orchestra.run.smithi076.stdout:{"election_epoch":30,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","modified":"2024-04-25T19:14:27.121750Z","created":"2024-04-25T19:14:27.121750Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3301","nonce":0},{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-04-25T19:19:29.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.870+0000 7f20f0b4d640 1 -- 172.21.15.76:0/1820913726 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f20cc0744e0 msgr2=0x7f20cc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:29.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.870+0000 7f20f0b4d640 1 --2- 172.21.15.76:0/1820913726 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f20cc0744e0 0x7f20cc0769f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f20dc034000 tx=0x7f20dc02e000 comp rx=0 tx=0).stop 2024-04-25T19:19:29.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.870+0000 7f20f0b4d640 1 -- 172.21.15.76:0/1820913726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20ec0caf60 msgr2=0x7f20ec1e7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:29.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.870+0000 7f20f0b4d640 1 --2- 172.21.15.76:0/1820913726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20ec0caf60 0x7f20ec1e7140 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f20e400c6b0 tx=0x7f20e400cb70 comp rx=0 tx=0).stop 2024-04-25T19:19:29.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.870+0000 7f20f0b4d640 1 -- 172.21.15.76:0/1820913726 shutdown_connections 2024-04-25T19:19:29.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.870+0000 7f20f0b4d640 1 --2- 172.21.15.76:0/1820913726 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f20cc0744e0 0x7f20cc0769f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.870+0000 7f20f0b4d640 1 --2- 172.21.15.76:0/1820913726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20ec0caf60 0x7f20ec1e7140 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.870+0000 7f20f0b4d640 1 --2- 172.21.15.76:0/1820913726 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f20ec0cab50 0x7f20ec14e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.870+0000 7f20f0b4d640 1 --2- 172.21.15.76:0/1820913726 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20ec0c6040 0x7f20ec147ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:29.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.870+0000 7f20f0b4d640 1 -- 172.21.15.76:0/1820913726 >> 172.21.15.76:0/1820913726 conn(0x7f20ec072b60 msgr2=0x7f20ec150290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:29.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.870+0000 7f20f0b4d640 1 -- 172.21.15.76:0/1820913726 shutdown_connections 2024-04-25T19:19:29.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:29.870+0000 7f20f0b4d640 1 -- 172.21.15.76:0/1820913726 wait complete. 2024-04-25T19:19:29.883 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-04-25T19:19:29.883 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-25T19:19:29.895 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-25T19:19:29.895 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:19:29.895 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:19:29.911 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:19:29.911 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:19:29.966 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:19:30.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.113+0000 7f5f9e89f640 1 Processor -- start 2024-04-25T19:19:30.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.113+0000 7f5f9e89f640 1 -- start start 2024-04-25T19:19:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.114+0000 7f5f9e89f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5f98157200 0x7f5f980d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.114+0000 7f5f9e89f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f980d0160 0x7f5f980ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.114+0000 7f5f9e89f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f9815dd00 con 0x7f5f980cfd50 2024-04-25T19:19:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.114+0000 7f5f9e89f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f5f9815cea0 con 0x7f5f98157200 2024-04-25T19:19:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.114+0000 7f5f9e89f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f5f9815d180 con 0x7f5f980d0160 2024-04-25T19:19:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.114+0000 7f5f977fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f5f980cfd50 0x7f5f980cee90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35698/0 (socket says 172.21.15.76:35698) 2024-04-25T19:19:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.114+0000 7f5f977fe640 1 -- 172.21.15.76:0/2958006480 learned_addr learned my addr 172.21.15.76:0/2958006480 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.115+0000 7f5f9ce3c640 1 --2- 172.21.15.76:0/2958006480 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f980d0160 0x7f5f980ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.115+0000 7f5f97fff640 1 --2- 172.21.15.76:0/2958006480 >> v2:172.21.15.105:3300/0 conn(0x7f5f98157200 0x7f5f980d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.115+0000 7f5f96ffd640 1 -- 172.21.15.76:0/2958006480 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4269181822 0 0) 0x7f5f9815dd00 con 0x7f5f980cfd50 2024-04-25T19:19:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.115+0000 7f5f96ffd640 1 -- 172.21.15.76:0/2958006480 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f80003520 con 0x7f5f980cfd50 2024-04-25T19:19:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.115+0000 7f5f96ffd640 1 -- 172.21.15.76:0/2958006480 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 808860616 0 0) 0x7f5f80003520 con 0x7f5f980cfd50 2024-04-25T19:19:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.115+0000 7f5f96ffd640 1 -- 172.21.15.76:0/2958006480 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f980d0160 msgr2=0x7f5f980ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.115+0000 7f5f96ffd640 1 --2- 172.21.15.76:0/2958006480 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f980d0160 0x7f5f980ccb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.115+0000 7f5f96ffd640 1 -- 172.21.15.76:0/2958006480 >> v2:172.21.15.105:3300/0 conn(0x7f5f98157200 msgr2=0x7f5f980d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.115+0000 7f5f96ffd640 1 --2- 172.21.15.76:0/2958006480 >> v2:172.21.15.105:3300/0 conn(0x7f5f98157200 0x7f5f980d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.115+0000 7f5f96ffd640 1 -- 172.21.15.76:0/2958006480 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9815d460 con 0x7f5f980cfd50 2024-04-25T19:19:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.115+0000 7f5f96ffd640 1 -- 172.21.15.76:0/2958006480 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f5f8c001670 con 0x7f5f980cfd50 2024-04-25T19:19:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.115+0000 7f5f96ffd640 1 -- 172.21.15.76:0/2958006480 >> v1:172.21.15.76:6789/0 conn(0x7f5f980cfd50 legacy=0x7f5f980cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.115+0000 7f5f96ffd640 1 --2- 172.21.15.76:0/2958006480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f800039b0 0x7f5f80003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.115+0000 7f5f96ffd640 1 --2- 172.21.15.76:0/2958006480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f80004800 0x7f5f80004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.115+0000 7f5f96ffd640 1 --2- 172.21.15.76:0/2958006480 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f980d0160 0x7f5f80005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.115+0000 7f5f96ffd640 1 -- 172.21.15.76:0/2958006480 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9815cea0 con 0x7f5f80004800 2024-04-25T19:19:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.115+0000 7f5f96ffd640 1 -- 172.21.15.76:0/2958006480 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9815d180 con 0x7f5f800039b0 2024-04-25T19:19:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.115+0000 7f5f96ffd640 1 -- 172.21.15.76:0/2958006480 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f5f8000ba70 con 0x7f5f980d0160 2024-04-25T19:19:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.115+0000 7f5f977fe640 1 --2- 172.21.15.76:0/2958006480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f80004800 0x7f5f80004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.115+0000 7f5f97fff640 1 --2- 172.21.15.76:0/2958006480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f800039b0 0x7f5f80003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.115+0000 7f5f9ce3c640 1 --2- 172.21.15.76:0/2958006480 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f980d0160 0x7f5f80005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.116+0000 7f5f977fe640 1 -- 172.21.15.76:0/2958006480 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f980d0160 msgr2=0x7f5f80005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.116+0000 7f5f977fe640 1 --2- 172.21.15.76:0/2958006480 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f980d0160 0x7f5f80005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.116+0000 7f5f977fe640 1 -- 172.21.15.76:0/2958006480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f800039b0 msgr2=0x7f5f80003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.116+0000 7f5f977fe640 1 --2- 172.21.15.76:0/2958006480 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f800039b0 0x7f5f80003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.116+0000 7f5f977fe640 1 -- 172.21.15.76:0/2958006480 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f80003520 con 0x7f5f80004800 2024-04-25T19:19:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.116+0000 7f5f977fe640 1 --2- 172.21.15.76:0/2958006480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f80004800 0x7f5f80004c10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c00c210 tx=0x7f5f80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a1aacfc32649ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.116+0000 7f5f96ffd640 1 -- 172.21.15.76:0/2958006480 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c007d30 con 0x7f5f80004800 2024-04-25T19:19:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.116+0000 7f5f96ffd640 1 -- 172.21.15.76:0/2958006480 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f8c005ad0 con 0x7f5f80004800 2024-04-25T19:19:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.116+0000 7f5f96ffd640 1 -- 172.21.15.76:0/2958006480 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c007070 con 0x7f5f80004800 2024-04-25T19:19:30.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.117+0000 7f5f9e89f640 1 -- 172.21.15.76:0/2958006480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f80004800 msgr2=0x7f5f80004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:30.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.117+0000 7f5f9e89f640 1 --2- 172.21.15.76:0/2958006480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f80004800 0x7f5f80004c10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c00c210 tx=0x7f5f80005830 comp rx=0 tx=0).stop 2024-04-25T19:19:30.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.117+0000 7f5f9ce3c640 1 -- 172.21.15.76:0/2958006480 reap_dead start 2024-04-25T19:19:30.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.117+0000 7f5f9e89f640 1 -- 172.21.15.76:0/2958006480 shutdown_connections 2024-04-25T19:19:30.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.117+0000 7f5f9e89f640 1 -- 172.21.15.76:0/2958006480 >> 172.21.15.76:0/2958006480 conn(0x7f5f980d2370 msgr2=0x7f5f980d2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:30.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.117+0000 7f5f9e89f640 1 -- 172.21.15.76:0/2958006480 shutdown_connections 2024-04-25T19:19:30.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.117+0000 7f5f9e89f640 1 -- 172.21.15.76:0/2958006480 wait complete. 2024-04-25T19:19:30.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.119+0000 7f5f9e89f640 1 Processor -- start 2024-04-25T19:19:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.119+0000 7f5f9e89f640 1 -- start start 2024-04-25T19:19:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.120+0000 7f5f9e89f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f80004800 0x7f5f981f2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.120+0000 7f5f9e89f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f9815a480 0x7f5f981f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.120+0000 7f5f9e89f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f980b61d0 0x7f5f98151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.120+0000 7f5f9e89f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5f980cf5a0 con 0x7f5f980b61d0 2024-04-25T19:19:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.120+0000 7f5f9e89f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9815da20 con 0x7f5f9815a480 2024-04-25T19:19:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.120+0000 7f5f9e89f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f5f980cf720 con 0x7f5f80004800 2024-04-25T19:19:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.120+0000 7f5f9ce3c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f980b61d0 0x7f5f98151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.120+0000 7f5f9ce3c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f980b61d0 0x7f5f98151390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50390/0 (socket says 172.21.15.76:50390) 2024-04-25T19:19:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.120+0000 7f5f9ce3c640 1 -- 172.21.15.76:0/2332508394 learned_addr learned my addr 172.21.15.76:0/2332508394 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.120+0000 7f5f97fff640 1 --2- 172.21.15.76:0/2332508394 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f80004800 0x7f5f981f2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.120+0000 7f5f977fe640 1 --2- 172.21.15.76:0/2332508394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f9815a480 0x7f5f981f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.121+0000 7f5f9ce3c640 1 -- 172.21.15.76:0/2332508394 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f80004800 msgr2=0x7f5f981f2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.121+0000 7f5f9ce3c640 1 --2- 172.21.15.76:0/2332508394 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f80004800 0x7f5f981f2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.121+0000 7f5f9ce3c640 1 -- 172.21.15.76:0/2332508394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f9815a480 msgr2=0x7f5f981f3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:30.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.121+0000 7f5f9ce3c640 1 --2- 172.21.15.76:0/2332508394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f9815a480 0x7f5f981f3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.121+0000 7f5f9ce3c640 1 -- 172.21.15.76:0/2332508394 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f98155fe0 con 0x7f5f980b61d0 2024-04-25T19:19:30.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.121+0000 7f5f977fe640 1 --2- 172.21.15.76:0/2332508394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f9815a480 0x7f5f981f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:19:30.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.121+0000 7f5f9ce3c640 1 --2- 172.21.15.76:0/2332508394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f980b61d0 0x7f5f98151390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5f980cc530 tx=0x7f5f9000f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:30.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.121+0000 7f5f94ff9640 1 -- 172.21.15.76:0/2332508394 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9000fc80 con 0x7f5f980b61d0 2024-04-25T19:19:30.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.121+0000 7f5f94ff9640 1 -- 172.21.15.76:0/2332508394 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f9001d070 con 0x7f5f980b61d0 2024-04-25T19:19:30.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.121+0000 7f5f9e89f640 1 -- 172.21.15.76:0/2332508394 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f980b4b70 con 0x7f5f980b61d0 2024-04-25T19:19:30.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.121+0000 7f5f94ff9640 1 -- 172.21.15.76:0/2332508394 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90015430 con 0x7f5f980b61d0 2024-04-25T19:19:30.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.121+0000 7f5f9e89f640 1 -- 172.21.15.76:0/2332508394 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f980b53f0 con 0x7f5f980b61d0 2024-04-25T19:19:30.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.122+0000 7f5f9e89f640 1 -- 172.21.15.76:0/2332508394 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f9806a390 con 0x7f5f980b61d0 2024-04-25T19:19:30.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.123+0000 7f5f94ff9640 1 -- 172.21.15.76:0/2332508394 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5f900155d0 con 0x7f5f980b61d0 2024-04-25T19:19:30.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.123+0000 7f5f94ff9640 1 --2- 172.21.15.76:0/2332508394 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5f840744b0 0x7f5f840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:30.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.123+0000 7f5f94ff9640 1 -- 172.21.15.76:0/2332508394 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f5f90095ae0 con 0x7f5f980b61d0 2024-04-25T19:19:30.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.123+0000 7f5f97fff640 1 --2- 172.21.15.76:0/2332508394 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5f840744b0 0x7f5f840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:30.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.124+0000 7f5f97fff640 1 --2- 172.21.15.76:0/2332508394 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5f840744b0 0x7f5f840769c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5f88009bc0 tx=0x7f5f88009190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:30.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.128+0000 7f5f94ff9640 1 -- 172.21.15.76:0/2332508394 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5f90010040 con 0x7f5f980b61d0 2024-04-25T19:19:30.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.303+0000 7f5f9e89f640 1 -- 172.21.15.76:0/2332508394 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f980b7ed0 con 0x7f5f980b61d0 2024-04-25T19:19:30.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.304+0000 7f5f94ff9640 1 -- 172.21.15.76:0/2332508394 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16855 (secure 0 0 0) 0x7f5f90060280 con 0x7f5f980b61d0 2024-04-25T19:19:30.305 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:19:30.306 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":23,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","created":"2024-04-25T19:15:01.693137+0000","modified":"2024-04-25T19:16:17.545036+0000","last_up_change":"2024-04-25T19:15:11.891681+0000","last_in_change":"2024-04-25T19:15:03.898065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:15:13.846514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:15:16.606533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:15:17.719197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72a16c02-48f9-45aa-9447-fcf755ebf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409574998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409574998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409574998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409574998}]},"public_addr":"172.21.15.76:6817/1409574998","cluster_addr":"172.21.15.76:6819/1409574998","heartbeat_back_addr":"172.21.15.76:6823/1409574998","heartbeat_front_addr":"172.21.15.76:6821/1409574998","state":["exists","up"]},{"osd":1,"uuid":"818d50f0-76c6-45f8-bf79-5e304f14c5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6801","nonce":1302663821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6803","nonce":1302663821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6807","nonce":1302663821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6805","nonce":1302663821}]},"public_addr":"172.21.15.76:6801/1302663821","cluster_addr":"172.21.15.76:6803/1302663821","heartbeat_back_addr":"172.21.15.76:6807/1302663821","heartbeat_front_addr":"172.21.15.76:6805/1302663821","state":["exists","up"]},{"osd":2,"uuid":"5714c8f5-ae23-473c-8cf1-db73811acded","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6809","nonce":1992938181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6811","nonce":1992938181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6815","nonce":1992938181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6813","nonce":1992938181}]},"public_addr":"172.21.15.76:6809/1992938181","cluster_addr":"172.21.15.76:6811/1992938181","heartbeat_back_addr":"172.21.15.76:6815/1992938181","heartbeat_front_addr":"172.21.15.76:6813/1992938181","state":["exists","up"]},{"osd":3,"uuid":"13204cdb-d8e6-4f97-bc6e-9561f442decf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6825","nonce":3418257163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6827","nonce":3418257163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6831","nonce":3418257163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6829","nonce":3418257163}]},"public_addr":"172.21.15.76:6825/3418257163","cluster_addr":"172.21.15.76:6827/3418257163","heartbeat_back_addr":"172.21.15.76:6831/3418257163","heartbeat_front_addr":"172.21.15.76:6829/3418257163","state":["exists","up"]},{"osd":4,"uuid":"5bea5166-1986-4d56-b5f5-ecc8cbe7f3ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6801","nonce":2125002468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6803","nonce":2125002468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6807","nonce":2125002468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6805","nonce":2125002468}]},"public_addr":"172.21.15.105:6801/2125002468","cluster_addr":"172.21.15.105:6803/2125002468","heartbeat_back_addr":"172.21.15.105:6807/2125002468","heartbeat_front_addr":"172.21.15.105:6805/2125002468","state":["exists","up"]},{"osd":5,"uuid":"96f89dcc-270d-4421-b8e8-3b92ba5f9844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6819","nonce":3413103121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6821","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6823","nonce":3413103121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6831","nonce":3413103121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6828","nonce":3413103121}]},"public_addr":"172.21.15.105:6819/3413103121","cluster_addr":"172.21.15.105:6823/3413103121","heartbeat_back_addr":"172.21.15.105:6831/3413103121","heartbeat_front_addr":"172.21.15.105:6828/3413103121","state":["exists","up"]},{"osd":6,"uuid":"93eb087a-d9ba-4096-b85d-b74d96e819a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6809","nonce":2721832954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6811","nonce":2721832954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6815","nonce":2721832954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6813","nonce":2721832954}]},"public_addr":"172.21.15.105:6809/2721832954","cluster_addr":"172.21.15.105:6811/2721832954","heartbeat_back_addr":"172.21.15.105:6815/2721832954","heartbeat_front_addr":"172.21.15.105:6813/2721832954","state":["exists","up"]},{"osd":7,"uuid":"2f80f790-ad2d-460e-a37d-47bf1771ef43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6818","nonce":548350430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6822","nonce":548350430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6829","nonce":548350430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6825","nonce":548350430}]},"public_addr":"172.21.15.105:6818/548350430","cluster_addr":"172.21.15.105:6822/548350430","heartbeat_back_addr":"172.21.15.105:6829/548350430","heartbeat_front_addr":"172.21.15.105:6825/548350430","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.106901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.072819+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.126865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.053464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:08.982614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.104825+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.112714+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.107387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":1}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:19:30.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.306+0000 7f5f9e89f640 1 -- 172.21.15.76:0/2332508394 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5f840744b0 msgr2=0x7f5f840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:30.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.306+0000 7f5f9e89f640 1 --2- 172.21.15.76:0/2332508394 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5f840744b0 0x7f5f840769c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5f88009bc0 tx=0x7f5f88009190 comp rx=0 tx=0).stop 2024-04-25T19:19:30.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.306+0000 7f5f9e89f640 1 -- 172.21.15.76:0/2332508394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f980b61d0 msgr2=0x7f5f98151390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:30.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.306+0000 7f5f9e89f640 1 --2- 172.21.15.76:0/2332508394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f980b61d0 0x7f5f98151390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5f980cc530 tx=0x7f5f9000f1b0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.306+0000 7f5f9e89f640 1 -- 172.21.15.76:0/2332508394 shutdown_connections 2024-04-25T19:19:30.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.306+0000 7f5f9e89f640 1 --2- 172.21.15.76:0/2332508394 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5f840744b0 0x7f5f840769c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.306+0000 7f5f9e89f640 1 --2- 172.21.15.76:0/2332508394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f980b61d0 0x7f5f98151390 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.306+0000 7f5f9e89f640 1 --2- 172.21.15.76:0/2332508394 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f9815a480 0x7f5f981f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.306+0000 7f5f9e89f640 1 --2- 172.21.15.76:0/2332508394 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5f80004800 0x7f5f981f2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.306+0000 7f5f9e89f640 1 -- 172.21.15.76:0/2332508394 >> 172.21.15.76:0/2332508394 conn(0x7f5f980d2370 msgr2=0x7f5f98075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:30.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.306+0000 7f5f9e89f640 1 -- 172.21.15.76:0/2332508394 shutdown_connections 2024-04-25T19:19:30.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.306+0000 7f5f9e89f640 1 -- 172.21.15.76:0/2332508394 wait complete. 2024-04-25T19:19:30.319 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T19:15:13.846514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T19:15:16.606533+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T19:15:17.719197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T19:19:30.319 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:19:30.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.428+0000 7f5fa65ce640 1 Processor -- start 2024-04-25T19:19:30.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.428+0000 7f5fa65ce640 1 -- start start 2024-04-25T19:19:30.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.430+0000 7f5fa65ce640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5fa00714c0 0x7f5fa006c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:30.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.430+0000 7f5fa65ce640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5fa00c9730 0x7f5fa006cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:30.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.430+0000 7f5fa65ce640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fa006d160 con 0x7f5fa0156bf0 2024-04-25T19:19:30.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.430+0000 7f5fa65ce640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5fa015d660 con 0x7f5fa00714c0 2024-04-25T19:19:30.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.430+0000 7f5fa65ce640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f5fa0071fa0 con 0x7f5fa00c9730 2024-04-25T19:19:30.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.430+0000 7f5f9ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5fa00714c0 0x7f5fa006c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:30.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.430+0000 7f5fa4b6b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f5fa0156bf0 0x7f5fa00c7710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35714/0 (socket says 172.21.15.76:35714) 2024-04-25T19:19:30.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.430+0000 7f5fa4b6b640 1 -- 172.21.15.76:0/2160161876 learned_addr learned my addr 172.21.15.76:0/2160161876 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:30.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.430+0000 7f5f9f7fe640 1 --2- 172.21.15.76:0/2160161876 >> v2:172.21.15.105:3300/0 conn(0x7f5fa00c9730 0x7f5fa006cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:30.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.430+0000 7f5f9ffff640 1 -- 172.21.15.76:0/2160161876 >> v2:172.21.15.105:3300/0 conn(0x7f5fa00c9730 msgr2=0x7f5fa006cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:30.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.430+0000 7f5f9ffff640 1 --2- 172.21.15.76:0/2160161876 >> v2:172.21.15.105:3300/0 conn(0x7f5fa00c9730 0x7f5fa006cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.430+0000 7f5f9ffff640 1 -- 172.21.15.76:0/2160161876 >> v1:172.21.15.76:6789/0 conn(0x7f5fa0156bf0 legacy=0x7f5fa00c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:30.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.430+0000 7f5f9ffff640 1 -- 172.21.15.76:0/2160161876 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa015d940 con 0x7f5fa00714c0 2024-04-25T19:19:30.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.430+0000 7f5f9effd640 1 -- 172.21.15.76:0/2160161876 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3609792490 0 0) 0x7f5fa006d160 con 0x7f5fa0156bf0 2024-04-25T19:19:30.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.431+0000 7f5f9ffff640 1 --2- 172.21.15.76:0/2160161876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fa00714c0 0x7f5fa006c5c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0073610 tx=0x7f5f9002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d7dc22840b5eda5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:30.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.431+0000 7f5f9effd640 1 -- 172.21.15.76:0/2160161876 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90037070 con 0x7f5fa00714c0 2024-04-25T19:19:30.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.431+0000 7f5f9effd640 1 -- 172.21.15.76:0/2160161876 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f90032440 con 0x7f5fa00714c0 2024-04-25T19:19:30.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.431+0000 7f5f9effd640 1 -- 172.21.15.76:0/2160161876 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90032810 con 0x7f5fa00714c0 2024-04-25T19:19:30.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.432+0000 7f5fa65ce640 1 -- 172.21.15.76:0/2160161876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fa00714c0 msgr2=0x7f5fa006c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:30.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.432+0000 7f5fa65ce640 1 --2- 172.21.15.76:0/2160161876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fa00714c0 0x7f5fa006c5c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0073610 tx=0x7f5f9002a1a0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.432+0000 7f5fa65ce640 1 -- 172.21.15.76:0/2160161876 shutdown_connections 2024-04-25T19:19:30.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.432+0000 7f5fa65ce640 1 --2- 172.21.15.76:0/2160161876 >> v2:172.21.15.105:3300/0 conn(0x7f5fa00c9730 0x7f5fa006cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.432+0000 7f5fa65ce640 1 --2- 172.21.15.76:0/2160161876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fa00714c0 0x7f5fa006c5c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.432+0000 7f5fa65ce640 1 -- 172.21.15.76:0/2160161876 >> 172.21.15.76:0/2160161876 conn(0x7f5fa006d6d0 msgr2=0x7f5fa006dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:30.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.432+0000 7f5fa65ce640 1 -- 172.21.15.76:0/2160161876 shutdown_connections 2024-04-25T19:19:30.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.432+0000 7f5fa65ce640 1 -- 172.21.15.76:0/2160161876 wait complete. 2024-04-25T19:19:30.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.433+0000 7f5fa65ce640 1 Processor -- start 2024-04-25T19:19:30.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.434+0000 7f5fa65ce640 1 -- start start 2024-04-25T19:19:30.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.435+0000 7f5fa65ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fa00c9730 0x7f5fa01f31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:30.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.435+0000 7f5fa65ce640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fa0156bf0 0x7f5fa00cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:30.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.435+0000 7f5fa65ce640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fa00cb960 0x7f5fa0149510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:30.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.435+0000 7f5fa65ce640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa00cc6f0 con 0x7f5fa00cb960 2024-04-25T19:19:30.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.435+0000 7f5fa65ce640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa00cc570 con 0x7f5fa00c9730 2024-04-25T19:19:30.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.435+0000 7f5fa65ce640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f5fa00c7e20 con 0x7f5fa0156bf0 2024-04-25T19:19:30.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.435+0000 7f5fa4b6b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fa00cb960 0x7f5fa0149510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:30.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.435+0000 7f5fa4b6b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fa00cb960 0x7f5fa0149510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50410/0 (socket says 172.21.15.76:50410) 2024-04-25T19:19:30.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.435+0000 7f5fa4b6b640 1 -- 172.21.15.76:0/3394887165 learned_addr learned my addr 172.21.15.76:0/3394887165 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:30.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.435+0000 7f5f9f7fe640 1 --2- 172.21.15.76:0/3394887165 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fa0156bf0 0x7f5fa00cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:30.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.435+0000 7f5f9ffff640 1 --2- 172.21.15.76:0/3394887165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fa00c9730 0x7f5fa01f31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:30.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.435+0000 7f5fa4b6b640 1 -- 172.21.15.76:0/3394887165 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fa0156bf0 msgr2=0x7f5fa00cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:30.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.435+0000 7f5fa4b6b640 1 --2- 172.21.15.76:0/3394887165 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fa0156bf0 0x7f5fa00cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.435+0000 7f5fa4b6b640 1 -- 172.21.15.76:0/3394887165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fa00c9730 msgr2=0x7f5fa01f31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:30.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.435+0000 7f5fa4b6b640 1 --2- 172.21.15.76:0/3394887165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fa00c9730 0x7f5fa01f31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.435+0000 7f5fa4b6b640 1 -- 172.21.15.76:0/3394887165 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa01f36f0 con 0x7f5fa00cb960 2024-04-25T19:19:30.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.436+0000 7f5fa4b6b640 1 --2- 172.21.15.76:0/3394887165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fa00cb960 0x7f5fa0149510 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800c6d0 tx=0x7f5f9800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:30.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.436+0000 7f5f9cff9640 1 -- 172.21.15.76:0/3394887165 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f98018070 con 0x7f5fa00cb960 2024-04-25T19:19:30.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.436+0000 7f5f9cff9640 1 -- 172.21.15.76:0/3394887165 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f98004d60 con 0x7f5fa00cb960 2024-04-25T19:19:30.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.436+0000 7f5fa65ce640 1 -- 172.21.15.76:0/3394887165 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fa00cb6f0 con 0x7f5fa00cb960 2024-04-25T19:19:30.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.436+0000 7f5f9cff9640 1 -- 172.21.15.76:0/3394887165 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9800a880 con 0x7f5fa00cb960 2024-04-25T19:19:30.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.436+0000 7f5fa65ce640 1 -- 172.21.15.76:0/3394887165 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fa0159760 con 0x7f5fa00cb960 2024-04-25T19:19:30.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.437+0000 7f5fa65ce640 1 -- 172.21.15.76:0/3394887165 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fa01f38c0 con 0x7f5fa00cb960 2024-04-25T19:19:30.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.438+0000 7f5f9cff9640 1 -- 172.21.15.76:0/3394887165 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5f9801b070 con 0x7f5fa00cb960 2024-04-25T19:19:30.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.438+0000 7f5f9cff9640 1 --2- 172.21.15.76:0/3394887165 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5f8c0744e0 0x7f5f8c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:30.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.438+0000 7f5f9cff9640 1 -- 172.21.15.76:0/3394887165 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f5f98094a20 con 0x7f5fa00cb960 2024-04-25T19:19:30.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.438+0000 7f5f9ffff640 1 --2- 172.21.15.76:0/3394887165 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5f8c0744e0 0x7f5f8c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:30.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.439+0000 7f5f9ffff640 1 --2- 172.21.15.76:0/3394887165 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5f8c0744e0 0x7f5f8c0769f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5f90034000 tx=0x7f5f9002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:30.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.443+0000 7f5f9cff9640 1 -- 172.21.15.76:0/3394887165 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5fa01f38c0 con 0x7f5fa00cb960 2024-04-25T19:19:30.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.620+0000 7f5fa65ce640 1 -- 172.21.15.76:0/3394887165 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5fa006a390 con 0x7f5fa00cb960 2024-04-25T19:19:30.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.621+0000 7f5f9cff9640 1 -- 172.21.15.76:0/3394887165 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f5f9805f130 con 0x7f5fa00cb960 2024-04-25T19:19:30.622 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-04-25T19:19:30.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.623+0000 7f5fa65ce640 1 -- 172.21.15.76:0/3394887165 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5f8c0744e0 msgr2=0x7f5f8c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:30.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.623+0000 7f5fa65ce640 1 --2- 172.21.15.76:0/3394887165 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5f8c0744e0 0x7f5f8c0769f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5f90034000 tx=0x7f5f9002e000 comp rx=0 tx=0).stop 2024-04-25T19:19:30.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.623+0000 7f5fa65ce640 1 -- 172.21.15.76:0/3394887165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fa00cb960 msgr2=0x7f5fa0149510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:30.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.623+0000 7f5fa65ce640 1 --2- 172.21.15.76:0/3394887165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fa00cb960 0x7f5fa0149510 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800c6d0 tx=0x7f5f9800cb90 comp rx=0 tx=0).stop 2024-04-25T19:19:30.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.623+0000 7f5fa65ce640 1 -- 172.21.15.76:0/3394887165 shutdown_connections 2024-04-25T19:19:30.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.623+0000 7f5fa65ce640 1 --2- 172.21.15.76:0/3394887165 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5f8c0744e0 0x7f5f8c0769f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.623+0000 7f5fa65ce640 1 --2- 172.21.15.76:0/3394887165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fa00cb960 0x7f5fa0149510 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.623+0000 7f5fa65ce640 1 --2- 172.21.15.76:0/3394887165 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fa0156bf0 0x7f5fa00cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.623+0000 7f5fa65ce640 1 --2- 172.21.15.76:0/3394887165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fa00c9730 0x7f5fa01f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.623+0000 7f5fa65ce640 1 -- 172.21.15.76:0/3394887165 >> 172.21.15.76:0/3394887165 conn(0x7f5fa006d6d0 msgr2=0x7f5fa01561d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:30.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.623+0000 7f5fa65ce640 1 -- 172.21.15.76:0/3394887165 shutdown_connections 2024-04-25T19:19:30.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.623+0000 7f5fa65ce640 1 -- 172.21.15.76:0/3394887165 wait complete. 2024-04-25T19:19:30.636 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T19:19:30.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.745+0000 7f3aacbb2640 1 Processor -- start 2024-04-25T19:19:30.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.745+0000 7f3aacbb2640 1 -- start start 2024-04-25T19:19:30.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.747+0000 7f3aacbb2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3aa815d8b0 0x7f3aa8168730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:30.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.747+0000 7f3aacbb2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3aa80cbd50 con 0x7f3aa80cb880 2024-04-25T19:19:30.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.747+0000 7f3aacbb2640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3aa80cc010 con 0x7f3aa80cad30 2024-04-25T19:19:30.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.747+0000 7f3aacbb2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f3aa8161340 con 0x7f3aa815d8b0 2024-04-25T19:19:30.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.747+0000 7f3aa659c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f3aa80cb880 0x7f3aa8160c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35730/0 (socket says 172.21.15.76:35730) 2024-04-25T19:19:30.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.747+0000 7f3aa6d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3aa815d8b0 0x7f3aa8168730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:30.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.747+0000 7f3aa659c640 1 -- 172.21.15.76:0/1533525969 learned_addr learned my addr 172.21.15.76:0/1533525969 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:30.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.747+0000 7f3aa6d9d640 1 -- 172.21.15.76:0/1533525969 >> v1:172.21.15.105:6789/0 conn(0x7f3aa80cad30 legacy=0x7f3aa8072ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:30.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.747+0000 7f3aa6d9d640 1 -- 172.21.15.76:0/1533525969 >> v1:172.21.15.76:6789/0 conn(0x7f3aa80cb880 legacy=0x7f3aa8160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:30.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.747+0000 7f3aa559a640 1 -- 172.21.15.76:0/1533525969 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 559157212 0 0) 0x7f3aa80cbd50 con 0x7f3aa80cb880 2024-04-25T19:19:30.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.747+0000 7f3aa6d9d640 1 -- 172.21.15.76:0/1533525969 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa80cc2f0 con 0x7f3aa815d8b0 2024-04-25T19:19:30.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.748+0000 7f3aa6d9d640 1 --2- 172.21.15.76:0/1533525969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3aa815d8b0 0x7f3aa8168730 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00097a0 tx=0x7f3aa002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f9db273d5fa0c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:30.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.748+0000 7f3aa559a640 1 -- 172.21.15.76:0/1533525969 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0037070 con 0x7f3aa815d8b0 2024-04-25T19:19:30.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.748+0000 7f3aa559a640 1 -- 172.21.15.76:0/1533525969 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3aa0032440 con 0x7f3aa815d8b0 2024-04-25T19:19:30.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.748+0000 7f3aa559a640 1 -- 172.21.15.76:0/1533525969 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0032810 con 0x7f3aa815d8b0 2024-04-25T19:19:30.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.749+0000 7f3aacbb2640 1 -- 172.21.15.76:0/1533525969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3aa815d8b0 msgr2=0x7f3aa8168730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:30.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.749+0000 7f3aacbb2640 1 --2- 172.21.15.76:0/1533525969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3aa815d8b0 0x7f3aa8168730 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00097a0 tx=0x7f3aa002a270 comp rx=0 tx=0).stop 2024-04-25T19:19:30.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.749+0000 7f3aacbb2640 1 -- 172.21.15.76:0/1533525969 shutdown_connections 2024-04-25T19:19:30.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.749+0000 7f3aacbb2640 1 --2- 172.21.15.76:0/1533525969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3aa815d8b0 0x7f3aa8168730 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.749+0000 7f3aacbb2640 1 -- 172.21.15.76:0/1533525969 >> 172.21.15.76:0/1533525969 conn(0x7f3aa8070b00 msgr2=0x7f3aa8070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:30.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.749+0000 7f3aacbb2640 1 -- 172.21.15.76:0/1533525969 shutdown_connections 2024-04-25T19:19:30.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.749+0000 7f3aacbb2640 1 -- 172.21.15.76:0/1533525969 wait complete. 2024-04-25T19:19:30.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.751+0000 7f3aacbb2640 1 Processor -- start 2024-04-25T19:19:30.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.751+0000 7f3aacbb2640 1 -- start start 2024-04-25T19:19:30.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.752+0000 7f3aacbb2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3aa80cad30 0x7f3aa81f3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:30.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.752+0000 7f3aacbb2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aa80cb880 0x7f3aa814fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:30.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.752+0000 7f3aacbb2640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3aa815d8b0 0x7f3aa8151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:30.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.752+0000 7f3aacbb2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa806e7f0 con 0x7f3aa80cad30 2024-04-25T19:19:30.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.752+0000 7f3aacbb2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa81607a0 con 0x7f3aa80cb880 2024-04-25T19:19:30.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.752+0000 7f3aacbb2640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f3aa80cc5d0 con 0x7f3aa815d8b0 2024-04-25T19:19:30.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.752+0000 7f3aa659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3aa80cad30 0x7f3aa81f3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:30.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.752+0000 7f3aa659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3aa80cad30 0x7f3aa81f3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50426/0 (socket says 172.21.15.76:50426) 2024-04-25T19:19:30.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.752+0000 7f3aa659c640 1 -- 172.21.15.76:0/3412430960 learned_addr learned my addr 172.21.15.76:0/3412430960 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:30.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.752+0000 7f3aa5d9b640 1 --2- 172.21.15.76:0/3412430960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aa80cb880 0x7f3aa814fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:30.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.752+0000 7f3aa6d9d640 1 --2- 172.21.15.76:0/3412430960 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3aa815d8b0 0x7f3aa8151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:30.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.752+0000 7f3aa659c640 1 -- 172.21.15.76:0/3412430960 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3aa815d8b0 msgr2=0x7f3aa8151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:30.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.752+0000 7f3aa659c640 1 --2- 172.21.15.76:0/3412430960 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3aa815d8b0 0x7f3aa8151500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.752+0000 7f3aa659c640 1 -- 172.21.15.76:0/3412430960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aa80cb880 msgr2=0x7f3aa814fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:30.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.752+0000 7f3aa659c640 1 --2- 172.21.15.76:0/3412430960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aa80cb880 0x7f3aa814fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.753+0000 7f3aa659c640 1 -- 172.21.15.76:0/3412430960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa81f3b80 con 0x7f3aa80cad30 2024-04-25T19:19:30.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.753+0000 7f3aa659c640 1 --2- 172.21.15.76:0/3412430960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3aa80cad30 0x7f3aa81f3640 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3a9800c7c0 tx=0x7f3a9800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:30.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.753+0000 7f3a96ffd640 1 -- 172.21.15.76:0/3412430960 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98018070 con 0x7f3aa80cad30 2024-04-25T19:19:30.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.753+0000 7f3a96ffd640 1 -- 172.21.15.76:0/3412430960 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a9800a5e0 con 0x7f3aa80cad30 2024-04-25T19:19:30.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.753+0000 7f3aacbb2640 1 -- 172.21.15.76:0/3412430960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa81506f0 con 0x7f3aa80cad30 2024-04-25T19:19:30.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.753+0000 7f3a96ffd640 1 -- 172.21.15.76:0/3412430960 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9800a9d0 con 0x7f3aa80cad30 2024-04-25T19:19:30.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.753+0000 7f3aacbb2640 1 -- 172.21.15.76:0/3412430960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa8153260 con 0x7f3aa80cad30 2024-04-25T19:19:30.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.754+0000 7f3aacbb2640 1 -- 172.21.15.76:0/3412430960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aa81f3d50 con 0x7f3aa80cad30 2024-04-25T19:19:30.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.755+0000 7f3a96ffd640 1 -- 172.21.15.76:0/3412430960 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3a9801b070 con 0x7f3aa80cad30 2024-04-25T19:19:30.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.755+0000 7f3a96ffd640 1 --2- 172.21.15.76:0/3412430960 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f3a880744e0 0x7f3a880769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:30.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.755+0000 7f3a96ffd640 1 -- 172.21.15.76:0/3412430960 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f3a98094750 con 0x7f3aa80cad30 2024-04-25T19:19:30.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.755+0000 7f3aa5d9b640 1 --2- 172.21.15.76:0/3412430960 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f3a880744e0 0x7f3a880769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:30.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.756+0000 7f3aa5d9b640 1 --2- 172.21.15.76:0/3412430960 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f3a880744e0 0x7f3a880769f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3aa806da20 tx=0x7f3a9c009060 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:30.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.760+0000 7f3a96ffd640 1 -- 172.21.15.76:0/3412430960 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3aa81f3d50 con 0x7f3aa80cad30 2024-04-25T19:19:30.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.934+0000 7f3aacbb2640 1 -- 172.21.15.76:0/3412430960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3aa806a390 con 0x7f3aa80cad30 2024-04-25T19:19:30.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.934+0000 7f3a96ffd640 1 -- 172.21.15.76:0/3412430960 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f3a980650c0 con 0x7f3aa80cad30 2024-04-25T19:19:30.936 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-04-25T19:19:30.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.936+0000 7f3aacbb2640 1 -- 172.21.15.76:0/3412430960 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f3a880744e0 msgr2=0x7f3a880769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:30.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.936+0000 7f3aacbb2640 1 --2- 172.21.15.76:0/3412430960 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f3a880744e0 0x7f3a880769f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3aa806da20 tx=0x7f3a9c009060 comp rx=0 tx=0).stop 2024-04-25T19:19:30.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.936+0000 7f3aacbb2640 1 -- 172.21.15.76:0/3412430960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3aa80cad30 msgr2=0x7f3aa81f3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:30.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.936+0000 7f3aacbb2640 1 --2- 172.21.15.76:0/3412430960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3aa80cad30 0x7f3aa81f3640 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3a9800c7c0 tx=0x7f3a9800cc80 comp rx=0 tx=0).stop 2024-04-25T19:19:30.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.937+0000 7f3aacbb2640 1 -- 172.21.15.76:0/3412430960 shutdown_connections 2024-04-25T19:19:30.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.937+0000 7f3aacbb2640 1 --2- 172.21.15.76:0/3412430960 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f3a880744e0 0x7f3a880769f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.937+0000 7f3aacbb2640 1 --2- 172.21.15.76:0/3412430960 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f3aa815d8b0 0x7f3aa8151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.937+0000 7f3aacbb2640 1 --2- 172.21.15.76:0/3412430960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aa80cb880 0x7f3aa814fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.937+0000 7f3aacbb2640 1 --2- 172.21.15.76:0/3412430960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3aa80cad30 0x7f3aa81f3640 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:30.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.937+0000 7f3aacbb2640 1 -- 172.21.15.76:0/3412430960 >> 172.21.15.76:0/3412430960 conn(0x7f3aa8070b00 msgr2=0x7f3aa8156c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:30.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.937+0000 7f3aacbb2640 1 -- 172.21.15.76:0/3412430960 shutdown_connections 2024-04-25T19:19:30.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:30.937+0000 7f3aacbb2640 1 -- 172.21.15.76:0/3412430960 wait complete. 2024-04-25T19:19:30.949 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T19:19:31.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.059+0000 7fb7bf6d8640 1 Processor -- start 2024-04-25T19:19:31.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.059+0000 7fb7bf6d8640 1 -- start start 2024-04-25T19:19:31.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.060+0000 7fb7bf6d8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb7b80747a0 0x7fb7b8154d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:31.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.060+0000 7fb7bf6d8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7b80c9700 0x7fb7b80c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:31.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.060+0000 7fb7bf6d8640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7b815b4c0 con 0x7fb7b80c9290 2024-04-25T19:19:31.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.060+0000 7fb7bf6d8640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb7b8072330 con 0x7fb7b80747a0 2024-04-25T19:19:31.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.060+0000 7fb7bf6d8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb7b80c8250 con 0x7fb7b80c9700 2024-04-25T19:19:31.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.060+0000 7fb7bcc73640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb7b80747a0 0x7fb7b8154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:31.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.060+0000 7fb7bd474640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb7b80c9290 0x7fb7b815df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35736/0 (socket says 172.21.15.76:35736) 2024-04-25T19:19:31.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.060+0000 7fb7bd474640 1 -- 172.21.15.76:0/1683807438 learned_addr learned my addr 172.21.15.76:0/1683807438 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:31.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.061+0000 7fb7bdc75640 1 --2- 172.21.15.76:0/1683807438 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7b80c9700 0x7fb7b80c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:31.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.061+0000 7fb7bcc73640 1 -- 172.21.15.76:0/1683807438 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7b80c9700 msgr2=0x7fb7b80c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:31.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.061+0000 7fb7bcc73640 1 --2- 172.21.15.76:0/1683807438 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7b80c9700 0x7fb7b80c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.061+0000 7fb7bcc73640 1 -- 172.21.15.76:0/1683807438 >> v1:172.21.15.76:6789/0 conn(0x7fb7b80c9290 legacy=0x7fb7b815df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:31.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.061+0000 7fb7affff640 1 -- 172.21.15.76:0/1683807438 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 571346853 0 0) 0x7fb7b815b4c0 con 0x7fb7b80c9290 2024-04-25T19:19:31.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.061+0000 7fb7bcc73640 1 -- 172.21.15.76:0/1683807438 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b815b780 con 0x7fb7b80747a0 2024-04-25T19:19:31.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.061+0000 7fb7bdc75640 1 --2- 172.21.15.76:0/1683807438 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7b80c9700 0x7fb7b80c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:19:31.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.061+0000 7fb7bcc73640 1 --2- 172.21.15.76:0/1683807438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7b80747a0 0x7fb7b8154d20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb7b80c83d0 tx=0x7fb7a802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c76f201183b24f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:31.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.061+0000 7fb7affff640 1 -- 172.21.15.76:0/1683807438 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8037070 con 0x7fb7b80747a0 2024-04-25T19:19:31.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.062+0000 7fb7affff640 1 -- 172.21.15.76:0/1683807438 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7a8032440 con 0x7fb7b80747a0 2024-04-25T19:19:31.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.062+0000 7fb7affff640 1 -- 172.21.15.76:0/1683807438 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8032810 con 0x7fb7b80747a0 2024-04-25T19:19:31.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.062+0000 7fb7bf6d8640 1 -- 172.21.15.76:0/1683807438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7b80747a0 msgr2=0x7fb7b8154d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:31.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.062+0000 7fb7bf6d8640 1 --2- 172.21.15.76:0/1683807438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7b80747a0 0x7fb7b8154d20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb7b80c83d0 tx=0x7fb7a802a210 comp rx=0 tx=0).stop 2024-04-25T19:19:31.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.062+0000 7fb7bf6d8640 1 -- 172.21.15.76:0/1683807438 shutdown_connections 2024-04-25T19:19:31.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.062+0000 7fb7bf6d8640 1 --2- 172.21.15.76:0/1683807438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7b80747a0 0x7fb7b8154d20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.062+0000 7fb7bf6d8640 1 --2- 172.21.15.76:0/1683807438 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7b80c9700 0x7fb7b80c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.062+0000 7fb7bf6d8640 1 -- 172.21.15.76:0/1683807438 >> 172.21.15.76:0/1683807438 conn(0x7fb7b80cae10 msgr2=0x7fb7b80cb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:31.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.062+0000 7fb7bf6d8640 1 -- 172.21.15.76:0/1683807438 shutdown_connections 2024-04-25T19:19:31.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.062+0000 7fb7bf6d8640 1 -- 172.21.15.76:0/1683807438 wait complete. 2024-04-25T19:19:31.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.064+0000 7fb7bf6d8640 1 Processor -- start 2024-04-25T19:19:31.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.064+0000 7fb7bf6d8640 1 -- start start 2024-04-25T19:19:31.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.065+0000 7fb7bf6d8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7b80747a0 0x7fb7b81f5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:31.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.065+0000 7fb7bf6d8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7b80c9700 0x7fb7b8159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:31.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.065+0000 7fb7bf6d8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7b814d2a0 0x7fb7b814c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:31.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.065+0000 7fb7bf6d8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b8070050 con 0x7fb7b80747a0 2024-04-25T19:19:31.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.065+0000 7fb7bf6d8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b80c8780 con 0x7fb7b814d2a0 2024-04-25T19:19:31.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.065+0000 7fb7bf6d8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb7b80cbbe0 con 0x7fb7b80c9700 2024-04-25T19:19:31.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.065+0000 7fb7bd474640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7b80747a0 0x7fb7b81f5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:31.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.065+0000 7fb7bd474640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7b80747a0 0x7fb7b81f5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50446/0 (socket says 172.21.15.76:50446) 2024-04-25T19:19:31.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.065+0000 7fb7bd474640 1 -- 172.21.15.76:0/3087152640 learned_addr learned my addr 172.21.15.76:0/3087152640 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:31.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.066+0000 7fb7bdc75640 1 --2- 172.21.15.76:0/3087152640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7b814d2a0 0x7fb7b814c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:31.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.066+0000 7fb7bcc73640 1 --2- 172.21.15.76:0/3087152640 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7b80c9700 0x7fb7b8159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:31.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.066+0000 7fb7bd474640 1 -- 172.21.15.76:0/3087152640 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7b80c9700 msgr2=0x7fb7b8159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:31.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.066+0000 7fb7bd474640 1 --2- 172.21.15.76:0/3087152640 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7b80c9700 0x7fb7b8159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.066+0000 7fb7bd474640 1 -- 172.21.15.76:0/3087152640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7b814d2a0 msgr2=0x7fb7b814c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:31.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.066+0000 7fb7bd474640 1 --2- 172.21.15.76:0/3087152640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7b814d2a0 0x7fb7b814c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.066+0000 7fb7bd474640 1 -- 172.21.15.76:0/3087152640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b8150400 con 0x7fb7b80747a0 2024-04-25T19:19:31.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.066+0000 7fb7bdc75640 1 --2- 172.21.15.76:0/3087152640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7b814d2a0 0x7fb7b814c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:19:31.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.066+0000 7fb7bcc73640 1 --2- 172.21.15.76:0/3087152640 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7b80c9700 0x7fb7b8159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:19:31.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.066+0000 7fb7bd474640 1 --2- 172.21.15.76:0/3087152640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7b80747a0 0x7fb7b81f5cf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400c430 tx=0x7fb7b400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:31.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.066+0000 7fb7adffb640 1 -- 172.21.15.76:0/3087152640 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b40074e0 con 0x7fb7b80747a0 2024-04-25T19:19:31.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.066+0000 7fb7adffb640 1 -- 172.21.15.76:0/3087152640 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7b40047e0 con 0x7fb7b80747a0 2024-04-25T19:19:31.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.066+0000 7fb7adffb640 1 -- 172.21.15.76:0/3087152640 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4004bd0 con 0x7fb7b80747a0 2024-04-25T19:19:31.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.066+0000 7fb7bf6d8640 1 -- 172.21.15.76:0/3087152640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b81506a0 con 0x7fb7b80747a0 2024-04-25T19:19:31.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.066+0000 7fb7bf6d8640 1 -- 172.21.15.76:0/3087152640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b81f2db0 con 0x7fb7b80747a0 2024-04-25T19:19:31.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.067+0000 7fb7bf6d8640 1 -- 172.21.15.76:0/3087152640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7b806a390 con 0x7fb7b80747a0 2024-04-25T19:19:31.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.068+0000 7fb7adffb640 1 -- 172.21.15.76:0/3087152640 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb7b4011070 con 0x7fb7b80747a0 2024-04-25T19:19:31.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.068+0000 7fb7adffb640 1 --2- 172.21.15.76:0/3087152640 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb79c0744b0 0x7fb79c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:31.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.068+0000 7fb7adffb640 1 -- 172.21.15.76:0/3087152640 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fb7b4094f50 con 0x7fb7b80747a0 2024-04-25T19:19:31.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.068+0000 7fb7bcc73640 1 --2- 172.21.15.76:0/3087152640 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb79c0744b0 0x7fb79c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:31.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.069+0000 7fb7bcc73640 1 --2- 172.21.15.76:0/3087152640 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb79c0744b0 0x7fb79c0769c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8034000 tx=0x7fb7a802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:31.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.073+0000 7fb7adffb640 1 -- 172.21.15.76:0/3087152640 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb7b405f660 con 0x7fb7b80747a0 2024-04-25T19:19:31.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.247+0000 7fb7bf6d8640 1 -- 172.21.15.76:0/3087152640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb7b806fe30 con 0x7fb7b80747a0 2024-04-25T19:19:31.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.247+0000 7fb7adffb640 1 -- 172.21.15.76:0/3087152640 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fb7b405ef60 con 0x7fb7b80747a0 2024-04-25T19:19:31.249 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-04-25T19:19:31.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.250+0000 7fb7bf6d8640 1 -- 172.21.15.76:0/3087152640 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb79c0744b0 msgr2=0x7fb79c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:31.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.250+0000 7fb7bf6d8640 1 --2- 172.21.15.76:0/3087152640 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb79c0744b0 0x7fb79c0769c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8034000 tx=0x7fb7a802e000 comp rx=0 tx=0).stop 2024-04-25T19:19:31.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.250+0000 7fb7bf6d8640 1 -- 172.21.15.76:0/3087152640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7b80747a0 msgr2=0x7fb7b81f5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:31.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.250+0000 7fb7bf6d8640 1 --2- 172.21.15.76:0/3087152640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7b80747a0 0x7fb7b81f5cf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400c430 tx=0x7fb7b400c8f0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.250+0000 7fb7bf6d8640 1 -- 172.21.15.76:0/3087152640 shutdown_connections 2024-04-25T19:19:31.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.250+0000 7fb7bf6d8640 1 --2- 172.21.15.76:0/3087152640 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb79c0744b0 0x7fb79c0769c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.250+0000 7fb7bf6d8640 1 --2- 172.21.15.76:0/3087152640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7b814d2a0 0x7fb7b814c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.250+0000 7fb7bf6d8640 1 --2- 172.21.15.76:0/3087152640 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb7b80c9700 0x7fb7b8159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.250+0000 7fb7bf6d8640 1 --2- 172.21.15.76:0/3087152640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7b80747a0 0x7fb7b81f5cf0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.250+0000 7fb7bf6d8640 1 -- 172.21.15.76:0/3087152640 >> 172.21.15.76:0/3087152640 conn(0x7fb7b80cae10 msgr2=0x7fb7b80cc520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:31.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.250+0000 7fb7bf6d8640 1 -- 172.21.15.76:0/3087152640 shutdown_connections 2024-04-25T19:19:31.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.250+0000 7fb7bf6d8640 1 -- 172.21.15.76:0/3087152640 wait complete. 2024-04-25T19:19:31.263 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T19:19:31.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.370+0000 7f4148396640 1 Processor -- start 2024-04-25T19:19:31.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.370+0000 7f4148396640 1 -- start start 2024-04-25T19:19:31.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.372+0000 7f4148396640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f414006de20 0x7f414006d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:31.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.372+0000 7f4148396640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f41400cc9e0 0x7f4140156f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:31.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.372+0000 7f4148396640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f414006e310 con 0x7f414015d780 2024-04-25T19:19:31.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.372+0000 7f4148396640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4140157450 con 0x7f414006de20 2024-04-25T19:19:31.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.372+0000 7f4148396640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f414015d2f0 con 0x7f41400cc9e0 2024-04-25T19:19:31.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.372+0000 7f4146132640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f414006de20 0x7f414006d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:31.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.372+0000 7f4146132640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f414006de20 0x7f414006d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50450/0 (socket says 172.21.15.76:50450) 2024-04-25T19:19:31.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.372+0000 7f4146132640 1 -- 172.21.15.76:0/2808503396 learned_addr learned my addr 172.21.15.76:0/2808503396 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:31.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.372+0000 7f4145931640 1 --2- 172.21.15.76:0/2808503396 >> v2:172.21.15.105:3300/0 conn(0x7f41400cc9e0 0x7f4140156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:31.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.372+0000 7f4146132640 1 -- 172.21.15.76:0/2808503396 >> v2:172.21.15.105:3300/0 conn(0x7f41400cc9e0 msgr2=0x7f4140156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:31.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.372+0000 7f4146132640 1 --2- 172.21.15.76:0/2808503396 >> v2:172.21.15.105:3300/0 conn(0x7f41400cc9e0 0x7f4140156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.372+0000 7f4146132640 1 -- 172.21.15.76:0/2808503396 >> v1:172.21.15.105:6789/0 conn(0x7f414015d780 legacy=0x7f41400cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:31.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.372+0000 7f4146132640 1 -- 172.21.15.76:0/2808503396 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414015ccc0 con 0x7f414006de20 2024-04-25T19:19:31.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.373+0000 7f4146132640 1 --2- 172.21.15.76:0/2808503396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f414006de20 0x7f414006d6d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f41400d0bc0 tx=0x7f413c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d845208be3bb820 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:31.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.373+0000 7f4145130640 1 -- 172.21.15.76:0/2808503396 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c036070 con 0x7f414006de20 2024-04-25T19:19:31.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.373+0000 7f4145130640 1 -- 172.21.15.76:0/2808503396 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f413c031600 con 0x7f414006de20 2024-04-25T19:19:31.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.373+0000 7f4145130640 1 -- 172.21.15.76:0/2808503396 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c0319d0 con 0x7f414006de20 2024-04-25T19:19:31.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.374+0000 7f4148396640 1 -- 172.21.15.76:0/2808503396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f414006de20 msgr2=0x7f414006d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:31.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.374+0000 7f4148396640 1 --2- 172.21.15.76:0/2808503396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f414006de20 0x7f414006d6d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f41400d0bc0 tx=0x7f413c02a660 comp rx=0 tx=0).stop 2024-04-25T19:19:31.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.374+0000 7f4148396640 1 -- 172.21.15.76:0/2808503396 shutdown_connections 2024-04-25T19:19:31.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.374+0000 7f4148396640 1 --2- 172.21.15.76:0/2808503396 >> v2:172.21.15.105:3300/0 conn(0x7f41400cc9e0 0x7f4140156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.374+0000 7f4148396640 1 --2- 172.21.15.76:0/2808503396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f414006de20 0x7f414006d6d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.374+0000 7f4148396640 1 -- 172.21.15.76:0/2808503396 >> 172.21.15.76:0/2808503396 conn(0x7f41400684e0 msgr2=0x7f41400688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:31.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.374+0000 7f4148396640 1 -- 172.21.15.76:0/2808503396 shutdown_connections 2024-04-25T19:19:31.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.374+0000 7f4148396640 1 -- 172.21.15.76:0/2808503396 wait complete. 2024-04-25T19:19:31.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.375+0000 7f4148396640 1 Processor -- start 2024-04-25T19:19:31.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.376+0000 7f4148396640 1 -- start start 2024-04-25T19:19:31.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.377+0000 7f4148396640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f414006de20 0x7f41401f6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:31.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.377+0000 7f4148396640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41400cc9e0 0x7f4140155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:31.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.377+0000 7f4148396640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f414015d780 0x7f41401f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:31.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.377+0000 7f4148396640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f414015cf80 con 0x7f414006de20 2024-04-25T19:19:31.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.377+0000 7f4148396640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f41400cb980 con 0x7f41400cc9e0 2024-04-25T19:19:31.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.377+0000 7f4148396640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f41400cbda0 con 0x7f414015d780 2024-04-25T19:19:31.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.377+0000 7f4146132640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f414006de20 0x7f41401f6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:31.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.377+0000 7f4146132640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f414006de20 0x7f41401f6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50464/0 (socket says 172.21.15.76:50464) 2024-04-25T19:19:31.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.377+0000 7f4146132640 1 -- 172.21.15.76:0/1904373600 learned_addr learned my addr 172.21.15.76:0/1904373600 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:31.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.377+0000 7f4145931640 1 --2- 172.21.15.76:0/1904373600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41400cc9e0 0x7f4140155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:31.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.377+0000 7f4146933640 1 --2- 172.21.15.76:0/1904373600 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f414015d780 0x7f41401f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:31.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.377+0000 7f4146132640 1 -- 172.21.15.76:0/1904373600 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f414015d780 msgr2=0x7f41401f59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:31.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.377+0000 7f4146132640 1 --2- 172.21.15.76:0/1904373600 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f414015d780 0x7f41401f59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.377+0000 7f4146132640 1 -- 172.21.15.76:0/1904373600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41400cc9e0 msgr2=0x7f4140155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:31.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.377+0000 7f4146132640 1 --2- 172.21.15.76:0/1904373600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41400cc9e0 0x7f4140155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.377+0000 7f4146132640 1 -- 172.21.15.76:0/1904373600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41401f4360 con 0x7f414006de20 2024-04-25T19:19:31.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.377+0000 7f4146132640 1 --2- 172.21.15.76:0/1904373600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f414006de20 0x7f41401f6de0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f413c009f40 tx=0x7f413c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:31.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.377+0000 7f4136ffd640 1 -- 172.21.15.76:0/1904373600 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c036050 con 0x7f414006de20 2024-04-25T19:19:31.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.377+0000 7f4136ffd640 1 -- 172.21.15.76:0/1904373600 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f413c042070 con 0x7f414006de20 2024-04-25T19:19:31.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.377+0000 7f4148396640 1 -- 172.21.15.76:0/1904373600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41401f7320 con 0x7f414006de20 2024-04-25T19:19:31.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.378+0000 7f4136ffd640 1 -- 172.21.15.76:0/1904373600 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c009510 con 0x7f414006de20 2024-04-25T19:19:31.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.378+0000 7f4148396640 1 -- 172.21.15.76:0/1904373600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4140160b50 con 0x7f414006de20 2024-04-25T19:19:31.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.379+0000 7f4148396640 1 -- 172.21.15.76:0/1904373600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41401f4530 con 0x7f414006de20 2024-04-25T19:19:31.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.379+0000 7f4136ffd640 1 -- 172.21.15.76:0/1904373600 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f413c007c50 con 0x7f414006de20 2024-04-25T19:19:31.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.380+0000 7f4136ffd640 1 --2- 172.21.15.76:0/1904373600 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f41240744b0 0x7f41240769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:31.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.380+0000 7f4136ffd640 1 -- 172.21.15.76:0/1904373600 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f413c0b60f0 con 0x7f414006de20 2024-04-25T19:19:31.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.380+0000 7f4145931640 1 --2- 172.21.15.76:0/1904373600 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f41240744b0 0x7f41240769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:31.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.380+0000 7f4145931640 1 --2- 172.21.15.76:0/1904373600 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f41240744b0 0x7f41240769c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4130006fd0 tx=0x7f4130008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:31.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.385+0000 7f4136ffd640 1 -- 172.21.15.76:0/1904373600 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f413c080920 con 0x7f414006de20 2024-04-25T19:19:31.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.573+0000 7f4148396640 1 -- 172.21.15.76:0/1904373600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f41401571f0 con 0x7f414006de20 2024-04-25T19:19:31.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.573+0000 7f4136ffd640 1 -- 172.21.15.76:0/1904373600 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f41401571f0 con 0x7f414006de20 2024-04-25T19:19:31.574 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2024-04-25T19:19:31.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.575+0000 7f4148396640 1 -- 172.21.15.76:0/1904373600 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f41240744b0 msgr2=0x7f41240769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:31.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.575+0000 7f4148396640 1 --2- 172.21.15.76:0/1904373600 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f41240744b0 0x7f41240769c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4130006fd0 tx=0x7f4130008040 comp rx=0 tx=0).stop 2024-04-25T19:19:31.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.575+0000 7f4148396640 1 -- 172.21.15.76:0/1904373600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f414006de20 msgr2=0x7f41401f6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:31.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.575+0000 7f4148396640 1 --2- 172.21.15.76:0/1904373600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f414006de20 0x7f41401f6de0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f413c009f40 tx=0x7f413c004a40 comp rx=0 tx=0).stop 2024-04-25T19:19:31.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.575+0000 7f4148396640 1 -- 172.21.15.76:0/1904373600 shutdown_connections 2024-04-25T19:19:31.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.575+0000 7f4148396640 1 --2- 172.21.15.76:0/1904373600 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f41240744b0 0x7f41240769c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.575+0000 7f4148396640 1 --2- 172.21.15.76:0/1904373600 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f414015d780 0x7f41401f59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.575+0000 7f4148396640 1 --2- 172.21.15.76:0/1904373600 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f41400cc9e0 0x7f4140155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.575+0000 7f4148396640 1 --2- 172.21.15.76:0/1904373600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f414006de20 0x7f41401f6de0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.575+0000 7f4148396640 1 -- 172.21.15.76:0/1904373600 >> 172.21.15.76:0/1904373600 conn(0x7f41400684e0 msgr2=0x7f4140075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:31.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.575+0000 7f4148396640 1 -- 172.21.15.76:0/1904373600 shutdown_connections 2024-04-25T19:19:31.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.575+0000 7f4148396640 1 -- 172.21.15.76:0/1904373600 wait complete. 2024-04-25T19:19:31.588 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-04-25T19:19:31.588 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:19:31.603 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:19:31.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.750+0000 7fae8070d640 1 Processor -- start 2024-04-25T19:19:31.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.750+0000 7fae8070d640 1 -- start start 2024-04-25T19:19:31.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.751+0000 7fae8070d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae78068950 0x7fae780c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:31.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.751+0000 7fae8070d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fae780c96e0 0x7fae780c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:31.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.751+0000 7fae8070d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae780c9400 con 0x7fae7815a360 2024-04-25T19:19:31.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.751+0000 7fae8070d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fae78068750 con 0x7fae780c96e0 2024-04-25T19:19:31.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.751+0000 7fae8070d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fae780c8ed0 con 0x7fae78068950 2024-04-25T19:19:31.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.751+0000 7fae7dca8640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fae7815a360 0x7fae780c3d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35746/0 (socket says 172.21.15.76:35746) 2024-04-25T19:19:31.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.751+0000 7fae7dca8640 1 -- 172.21.15.76:0/3602983821 learned_addr learned my addr 172.21.15.76:0/3602983821 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:31.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.751+0000 7fae7e4a9640 1 --2- 172.21.15.76:0/3602983821 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae78068950 0x7fae780c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:31.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.752+0000 7fae7ecaa640 1 --2- 172.21.15.76:0/3602983821 >> v2:172.21.15.105:3300/0 conn(0x7fae780c96e0 0x7fae780c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:31.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.752+0000 7fae7d4a7640 1 -- 172.21.15.76:0/3602983821 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 894024264 0 0) 0x7fae68001110 con 0x7fae7815a360 2024-04-25T19:19:31.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.752+0000 7fae7d4a7640 1 -- 172.21.15.76:0/3602983821 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae64003520 con 0x7fae7815a360 2024-04-25T19:19:31.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.752+0000 7fae7d4a7640 1 -- 172.21.15.76:0/3602983821 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3359253623 0 0) 0x7fae64003520 con 0x7fae7815a360 2024-04-25T19:19:31.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.752+0000 7fae7d4a7640 1 -- 172.21.15.76:0/3602983821 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae78068950 msgr2=0x7fae780c4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:31.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.752+0000 7fae7d4a7640 1 --2- 172.21.15.76:0/3602983821 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae78068950 0x7fae780c4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.752+0000 7fae7d4a7640 1 -- 172.21.15.76:0/3602983821 >> v2:172.21.15.105:3300/0 conn(0x7fae780c96e0 msgr2=0x7fae780c9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:31.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.752+0000 7fae7d4a7640 1 --2- 172.21.15.76:0/3602983821 >> v2:172.21.15.105:3300/0 conn(0x7fae780c96e0 0x7fae780c9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.752+0000 7fae7d4a7640 1 -- 172.21.15.76:0/3602983821 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7815a770 con 0x7fae7815a360 2024-04-25T19:19:31.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.752+0000 7fae7d4a7640 1 -- 172.21.15.76:0/3602983821 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7fae68001840 con 0x7fae7815a360 2024-04-25T19:19:31.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.752+0000 7fae7ecaa640 1 --2- 172.21.15.76:0/3602983821 >> v2:172.21.15.105:3300/0 conn(0x7fae780c96e0 0x7fae780c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:19:31.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.752+0000 7fae7d4a7640 1 -- 172.21.15.76:0/3602983821 >> v1:172.21.15.76:6789/0 conn(0x7fae7815a360 legacy=0x7fae780c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:31.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.752+0000 7fae7d4a7640 1 --2- 172.21.15.76:0/3602983821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae640039b0 0x7fae64003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:31.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.752+0000 7fae7d4a7640 1 --2- 172.21.15.76:0/3602983821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae64004800 0x7fae64004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:31.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.752+0000 7fae7d4a7640 1 --2- 172.21.15.76:0/3602983821 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae64005150 0x7fae6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:31.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.752+0000 7fae7d4a7640 1 -- 172.21.15.76:0/3602983821 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fae780c8ed0 con 0x7fae64004800 2024-04-25T19:19:31.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.752+0000 7fae7d4a7640 1 -- 172.21.15.76:0/3602983821 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fae64005640 con 0x7fae640039b0 2024-04-25T19:19:31.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.752+0000 7fae7d4a7640 1 -- 172.21.15.76:0/3602983821 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fae6400bf30 con 0x7fae64005150 2024-04-25T19:19:31.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.752+0000 7fae7dca8640 1 --2- 172.21.15.76:0/3602983821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae64004800 0x7fae64004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:31.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.752+0000 7fae7e4a9640 1 --2- 172.21.15.76:0/3602983821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae640039b0 0x7fae64003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:31.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.752+0000 7fae7ecaa640 1 --2- 172.21.15.76:0/3602983821 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae64005150 0x7fae6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:31.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.752+0000 7fae7dca8640 1 -- 172.21.15.76:0/3602983821 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae64005150 msgr2=0x7fae6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:31.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.752+0000 7fae7dca8640 1 --2- 172.21.15.76:0/3602983821 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae64005150 0x7fae6400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.752+0000 7fae7dca8640 1 -- 172.21.15.76:0/3602983821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae640039b0 msgr2=0x7fae64003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:31.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.752+0000 7fae7dca8640 1 --2- 172.21.15.76:0/3602983821 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae640039b0 0x7fae64003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.752+0000 7fae7dca8640 1 -- 172.21.15.76:0/3602983821 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae64003520 con 0x7fae64004800 2024-04-25T19:19:31.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.753+0000 7fae7dca8640 1 --2- 172.21.15.76:0/3602983821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae64004800 0x7fae64004c10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fae7806fd10 tx=0x7fae64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e815e6464b40dad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:31.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.753+0000 7fae7d4a7640 1 -- 172.21.15.76:0/3602983821 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6800c0e0 con 0x7fae64004800 2024-04-25T19:19:31.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.753+0000 7fae7d4a7640 1 -- 172.21.15.76:0/3602983821 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae68034070 con 0x7fae64004800 2024-04-25T19:19:31.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.753+0000 7fae7d4a7640 1 -- 172.21.15.76:0/3602983821 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae68005d30 con 0x7fae64004800 2024-04-25T19:19:31.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.754+0000 7fae8070d640 1 -- 172.21.15.76:0/3602983821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae64004800 msgr2=0x7fae64004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:31.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.754+0000 7fae8070d640 1 --2- 172.21.15.76:0/3602983821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae64004800 0x7fae64004c10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fae7806fd10 tx=0x7fae64005830 comp rx=0 tx=0).stop 2024-04-25T19:19:31.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.754+0000 7fae7ecaa640 1 -- 172.21.15.76:0/3602983821 reap_dead start 2024-04-25T19:19:31.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.754+0000 7fae8070d640 1 -- 172.21.15.76:0/3602983821 shutdown_connections 2024-04-25T19:19:31.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.754+0000 7fae8070d640 1 -- 172.21.15.76:0/3602983821 >> 172.21.15.76:0/3602983821 conn(0x7fae7806f720 msgr2=0x7fae78070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:31.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.754+0000 7fae8070d640 1 -- 172.21.15.76:0/3602983821 shutdown_connections 2024-04-25T19:19:31.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.754+0000 7fae8070d640 1 -- 172.21.15.76:0/3602983821 wait complete. 2024-04-25T19:19:31.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.756+0000 7fae8070d640 1 Processor -- start 2024-04-25T19:19:31.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.756+0000 7fae8070d640 1 -- start start 2024-04-25T19:19:31.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.757+0000 7fae8070d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae64004800 0x7fae781f6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:31.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.757+0000 7fae8070d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae7814e8c0 0x7fae7814ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:31.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.757+0000 7fae8070d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae780cad00 0x7fae780cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:31.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.757+0000 7fae8070d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fae780ca030 con 0x7fae780cad00 2024-04-25T19:19:31.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.757+0000 7fae8070d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fae78074440 con 0x7fae64004800 2024-04-25T19:19:31.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.757+0000 7fae8070d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fae780c9120 con 0x7fae7814e8c0 2024-04-25T19:19:31.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.757+0000 7fae7ecaa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae780cad00 0x7fae780cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:31.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.757+0000 7fae7ecaa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae780cad00 0x7fae780cb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50488/0 (socket says 172.21.15.76:50488) 2024-04-25T19:19:31.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.757+0000 7fae7ecaa640 1 -- 172.21.15.76:0/3427635363 learned_addr learned my addr 172.21.15.76:0/3427635363 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:31.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.757+0000 7fae7e4a9640 1 --2- 172.21.15.76:0/3427635363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae64004800 0x7fae781f6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:31.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.757+0000 7fae7dca8640 1 --2- 172.21.15.76:0/3427635363 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae7814e8c0 0x7fae7814ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:31.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.758+0000 7fae7ecaa640 1 -- 172.21.15.76:0/3427635363 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae7814e8c0 msgr2=0x7fae7814ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:31.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.758+0000 7fae7ecaa640 1 --2- 172.21.15.76:0/3427635363 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae7814e8c0 0x7fae7814ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.758+0000 7fae7ecaa640 1 -- 172.21.15.76:0/3427635363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae64004800 msgr2=0x7fae781f6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:31.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.758+0000 7fae7ecaa640 1 --2- 172.21.15.76:0/3427635363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae64004800 0x7fae781f6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.758+0000 7fae7ecaa640 1 -- 172.21.15.76:0/3427635363 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7815b840 con 0x7fae780cad00 2024-04-25T19:19:31.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.758+0000 7fae7dca8640 1 --2- 172.21.15.76:0/3427635363 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae7814e8c0 0x7fae7814ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:19:31.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.758+0000 7fae7ecaa640 1 --2- 172.21.15.76:0/3427635363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae780cad00 0x7fae780cb110 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fae7815dd30 tx=0x7fae7000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:31.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.758+0000 7fae6effd640 1 -- 172.21.15.76:0/3427635363 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7000fd90 con 0x7fae780cad00 2024-04-25T19:19:31.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.758+0000 7fae6effd640 1 -- 172.21.15.76:0/3427635363 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae70003990 con 0x7fae780cad00 2024-04-25T19:19:31.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.758+0000 7fae8070d640 1 -- 172.21.15.76:0/3427635363 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae7815ba30 con 0x7fae780cad00 2024-04-25T19:19:31.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.758+0000 7fae6effd640 1 -- 172.21.15.76:0/3427635363 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae70003d80 con 0x7fae780cad00 2024-04-25T19:19:31.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.758+0000 7fae8070d640 1 -- 172.21.15.76:0/3427635363 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae7815b560 con 0x7fae780cad00 2024-04-25T19:19:31.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.759+0000 7fae8070d640 1 -- 172.21.15.76:0/3427635363 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae7806a390 con 0x7fae780cad00 2024-04-25T19:19:31.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.760+0000 7fae6effd640 1 -- 172.21.15.76:0/3427635363 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fae7001f020 con 0x7fae780cad00 2024-04-25T19:19:31.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.760+0000 7fae6effd640 1 --2- 172.21.15.76:0/3427635363 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fae5c0744b0 0x7fae5c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:31.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.760+0000 7fae6effd640 1 -- 172.21.15.76:0/3427635363 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fae70095760 con 0x7fae780cad00 2024-04-25T19:19:31.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.761+0000 7fae7e4a9640 1 --2- 172.21.15.76:0/3427635363 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fae5c0744b0 0x7fae5c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:31.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.761+0000 7fae7e4a9640 1 --2- 172.21.15.76:0/3427635363 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fae5c0744b0 0x7fae5c0769c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fae78072690 tx=0x7fae74009190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:31.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.765+0000 7fae6effd640 1 -- 172.21.15.76:0/3427635363 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fae70010040 con 0x7fae780cad00 2024-04-25T19:19:31.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.943+0000 7fae8070d640 1 -- 172.21.15.76:0/3427635363 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae780ca1e0 con 0x7fae780cad00 2024-04-25T19:19:31.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.944+0000 7fae6effd640 1 -- 172.21.15.76:0/3427635363 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16855 (secure 0 0 0) 0x7fae7005fe70 con 0x7fae780cad00 2024-04-25T19:19:31.945 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:19:31.946 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":23,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","created":"2024-04-25T19:15:01.693137+0000","modified":"2024-04-25T19:16:17.545036+0000","last_up_change":"2024-04-25T19:15:11.891681+0000","last_in_change":"2024-04-25T19:15:03.898065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:15:13.846514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:15:16.606533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:15:17.719197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72a16c02-48f9-45aa-9447-fcf755ebf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409574998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409574998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409574998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409574998}]},"public_addr":"172.21.15.76:6817/1409574998","cluster_addr":"172.21.15.76:6819/1409574998","heartbeat_back_addr":"172.21.15.76:6823/1409574998","heartbeat_front_addr":"172.21.15.76:6821/1409574998","state":["exists","up"]},{"osd":1,"uuid":"818d50f0-76c6-45f8-bf79-5e304f14c5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6801","nonce":1302663821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6803","nonce":1302663821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6807","nonce":1302663821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6805","nonce":1302663821}]},"public_addr":"172.21.15.76:6801/1302663821","cluster_addr":"172.21.15.76:6803/1302663821","heartbeat_back_addr":"172.21.15.76:6807/1302663821","heartbeat_front_addr":"172.21.15.76:6805/1302663821","state":["exists","up"]},{"osd":2,"uuid":"5714c8f5-ae23-473c-8cf1-db73811acded","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6809","nonce":1992938181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6811","nonce":1992938181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6815","nonce":1992938181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6813","nonce":1992938181}]},"public_addr":"172.21.15.76:6809/1992938181","cluster_addr":"172.21.15.76:6811/1992938181","heartbeat_back_addr":"172.21.15.76:6815/1992938181","heartbeat_front_addr":"172.21.15.76:6813/1992938181","state":["exists","up"]},{"osd":3,"uuid":"13204cdb-d8e6-4f97-bc6e-9561f442decf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6825","nonce":3418257163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6827","nonce":3418257163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6831","nonce":3418257163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6829","nonce":3418257163}]},"public_addr":"172.21.15.76:6825/3418257163","cluster_addr":"172.21.15.76:6827/3418257163","heartbeat_back_addr":"172.21.15.76:6831/3418257163","heartbeat_front_addr":"172.21.15.76:6829/3418257163","state":["exists","up"]},{"osd":4,"uuid":"5bea5166-1986-4d56-b5f5-ecc8cbe7f3ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6801","nonce":2125002468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6803","nonce":2125002468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6807","nonce":2125002468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6805","nonce":2125002468}]},"public_addr":"172.21.15.105:6801/2125002468","cluster_addr":"172.21.15.105:6803/2125002468","heartbeat_back_addr":"172.21.15.105:6807/2125002468","heartbeat_front_addr":"172.21.15.105:6805/2125002468","state":["exists","up"]},{"osd":5,"uuid":"96f89dcc-270d-4421-b8e8-3b92ba5f9844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6819","nonce":3413103121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6821","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6823","nonce":3413103121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6831","nonce":3413103121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6828","nonce":3413103121}]},"public_addr":"172.21.15.105:6819/3413103121","cluster_addr":"172.21.15.105:6823/3413103121","heartbeat_back_addr":"172.21.15.105:6831/3413103121","heartbeat_front_addr":"172.21.15.105:6828/3413103121","state":["exists","up"]},{"osd":6,"uuid":"93eb087a-d9ba-4096-b85d-b74d96e819a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6809","nonce":2721832954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6811","nonce":2721832954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6815","nonce":2721832954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6813","nonce":2721832954}]},"public_addr":"172.21.15.105:6809/2721832954","cluster_addr":"172.21.15.105:6811/2721832954","heartbeat_back_addr":"172.21.15.105:6815/2721832954","heartbeat_front_addr":"172.21.15.105:6813/2721832954","state":["exists","up"]},{"osd":7,"uuid":"2f80f790-ad2d-460e-a37d-47bf1771ef43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6818","nonce":548350430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6822","nonce":548350430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6829","nonce":548350430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6825","nonce":548350430}]},"public_addr":"172.21.15.105:6818/548350430","cluster_addr":"172.21.15.105:6822/548350430","heartbeat_back_addr":"172.21.15.105:6829/548350430","heartbeat_front_addr":"172.21.15.105:6825/548350430","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.106901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.072819+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.126865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.053464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:08.982614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.104825+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.112714+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.107387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":1}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:19:31.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.946+0000 7fae8070d640 1 -- 172.21.15.76:0/3427635363 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fae5c0744b0 msgr2=0x7fae5c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:31.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.946+0000 7fae8070d640 1 --2- 172.21.15.76:0/3427635363 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fae5c0744b0 0x7fae5c0769c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fae78072690 tx=0x7fae74009190 comp rx=0 tx=0).stop 2024-04-25T19:19:31.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.946+0000 7fae8070d640 1 -- 172.21.15.76:0/3427635363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae780cad00 msgr2=0x7fae780cb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:31.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.946+0000 7fae8070d640 1 --2- 172.21.15.76:0/3427635363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae780cad00 0x7fae780cb110 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fae7815dd30 tx=0x7fae7000f580 comp rx=0 tx=0).stop 2024-04-25T19:19:31.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.946+0000 7fae8070d640 1 -- 172.21.15.76:0/3427635363 shutdown_connections 2024-04-25T19:19:31.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.946+0000 7fae8070d640 1 --2- 172.21.15.76:0/3427635363 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fae5c0744b0 0x7fae5c0769c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.946+0000 7fae8070d640 1 --2- 172.21.15.76:0/3427635363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae780cad00 0x7fae780cb110 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.946+0000 7fae8070d640 1 --2- 172.21.15.76:0/3427635363 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fae7814e8c0 0x7fae7814ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.946+0000 7fae8070d640 1 --2- 172.21.15.76:0/3427635363 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fae64004800 0x7fae781f6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:31.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.946+0000 7fae8070d640 1 -- 172.21.15.76:0/3427635363 >> 172.21.15.76:0/3427635363 conn(0x7fae7806f720 msgr2=0x7fae780c4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:31.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.946+0000 7fae8070d640 1 -- 172.21.15.76:0/3427635363 shutdown_connections 2024-04-25T19:19:31.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:31.946+0000 7fae8070d640 1 -- 172.21.15.76:0/3427635363 wait complete. 2024-04-25T19:19:31.959 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T19:15:13.846514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T19:15:16.606533+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T19:15:17.719197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T19:19:31.960 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:19:32.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.068+0000 7f10dddf3640 1 Processor -- start 2024-04-25T19:19:32.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.068+0000 7f10dddf3640 1 -- start start 2024-04-25T19:19:32.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.069+0000 7f10dddf3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10d80c9730 0x7f10d80c7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:32.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.069+0000 7f10dddf3640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f10d8156bf0 0x7f10d80c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:32.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.069+0000 7f10dddf3640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10d806d160 con 0x7f10d80714c0 2024-04-25T19:19:32.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.069+0000 7f10dddf3640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f10d815d660 con 0x7f10d8156bf0 2024-04-25T19:19:32.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.069+0000 7f10dddf3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f10d8071fa0 con 0x7f10d80c9730 2024-04-25T19:19:32.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.070+0000 7f10d77fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f10d80714c0 0x7f10d806c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35748/0 (socket says 172.21.15.76:35748) 2024-04-25T19:19:32.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.070+0000 7f10d77fe640 1 -- 172.21.15.76:0/2619993997 learned_addr learned my addr 172.21.15.76:0/2619993997 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:32.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.070+0000 7f10d7fff640 1 --2- 172.21.15.76:0/2619993997 >> v2:172.21.15.105:3300/0 conn(0x7f10d8156bf0 0x7f10d80c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:32.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.070+0000 7f10d6ffd640 1 --2- 172.21.15.76:0/2619993997 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10d80c9730 0x7f10d80c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:32.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.070+0000 7f10d67fc640 1 -- 172.21.15.76:0/2619993997 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2622832416 0 0) 0x7f10d806d160 con 0x7f10d80714c0 2024-04-25T19:19:32.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.070+0000 7f10d67fc640 1 -- 172.21.15.76:0/2619993997 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10c0003520 con 0x7f10d80714c0 2024-04-25T19:19:32.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.070+0000 7f10d67fc640 1 -- 172.21.15.76:0/2619993997 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2058567464 0 0) 0x7f10c0003520 con 0x7f10d80714c0 2024-04-25T19:19:32.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.070+0000 7f10d67fc640 1 -- 172.21.15.76:0/2619993997 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10d80c9730 msgr2=0x7f10d80c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:32.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.070+0000 7f10d67fc640 1 --2- 172.21.15.76:0/2619993997 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10d80c9730 0x7f10d80c7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.070+0000 7f10d67fc640 1 -- 172.21.15.76:0/2619993997 >> v2:172.21.15.105:3300/0 conn(0x7f10d8156bf0 msgr2=0x7f10d80c7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:32.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.070+0000 7f10d67fc640 1 --2- 172.21.15.76:0/2619993997 >> v2:172.21.15.105:3300/0 conn(0x7f10d8156bf0 0x7f10d80c7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.070+0000 7f10d67fc640 1 -- 172.21.15.76:0/2619993997 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d815d940 con 0x7f10d80714c0 2024-04-25T19:19:32.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.070+0000 7f10d67fc640 1 -- 172.21.15.76:0/2619993997 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f10c8001670 con 0x7f10d80714c0 2024-04-25T19:19:32.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.070+0000 7f10d67fc640 1 -- 172.21.15.76:0/2619993997 >> v1:172.21.15.76:6789/0 conn(0x7f10d80714c0 legacy=0x7f10d806c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:32.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.070+0000 7f10d67fc640 1 --2- 172.21.15.76:0/2619993997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10c00039b0 0x7f10c0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:32.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.070+0000 7f10d67fc640 1 --2- 172.21.15.76:0/2619993997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10c0004800 0x7f10c0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:32.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.070+0000 7f10d67fc640 1 --2- 172.21.15.76:0/2619993997 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10d80c9730 0x7f10c0005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:32.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.070+0000 7f10d67fc640 1 -- 172.21.15.76:0/2619993997 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f10d815d660 con 0x7f10c00039b0 2024-04-25T19:19:32.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.070+0000 7f10d67fc640 1 -- 172.21.15.76:0/2619993997 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f10d8071fa0 con 0x7f10c0004800 2024-04-25T19:19:32.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.070+0000 7f10d67fc640 1 -- 172.21.15.76:0/2619993997 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f10c000b9c0 con 0x7f10d80c9730 2024-04-25T19:19:32.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.070+0000 7f10d77fe640 1 --2- 172.21.15.76:0/2619993997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10c00039b0 0x7f10c0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:32.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.070+0000 7f10d6ffd640 1 --2- 172.21.15.76:0/2619993997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10c0004800 0x7f10c0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:32.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.070+0000 7f10d7fff640 1 --2- 172.21.15.76:0/2619993997 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10d80c9730 0x7f10c0005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:32.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.071+0000 7f10d77fe640 1 -- 172.21.15.76:0/2619993997 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10d80c9730 msgr2=0x7f10c0005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:32.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.071+0000 7f10d77fe640 1 --2- 172.21.15.76:0/2619993997 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10d80c9730 0x7f10c0005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.071+0000 7f10d77fe640 1 -- 172.21.15.76:0/2619993997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10c0004800 msgr2=0x7f10c0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:32.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.071+0000 7f10d77fe640 1 --2- 172.21.15.76:0/2619993997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10c0004800 0x7f10c0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.071+0000 7f10d77fe640 1 -- 172.21.15.76:0/2619993997 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c0003520 con 0x7f10c00039b0 2024-04-25T19:19:32.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.071+0000 7f10d77fe640 1 --2- 172.21.15.76:0/2619993997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10c00039b0 0x7f10c0003e40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f10c8001050 tx=0x7f10c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e325f4f01555a76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:32.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.071+0000 7f10d67fc640 1 -- 172.21.15.76:0/2619993997 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8007d70 con 0x7f10c00039b0 2024-04-25T19:19:32.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.071+0000 7f10d67fc640 1 -- 172.21.15.76:0/2619993997 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c8005af0 con 0x7f10c00039b0 2024-04-25T19:19:32.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.071+0000 7f10d67fc640 1 -- 172.21.15.76:0/2619993997 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8007070 con 0x7f10c00039b0 2024-04-25T19:19:32.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.072+0000 7f10dddf3640 1 -- 172.21.15.76:0/2619993997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10c00039b0 msgr2=0x7f10c0003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:32.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.072+0000 7f10dddf3640 1 --2- 172.21.15.76:0/2619993997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10c00039b0 0x7f10c0003e40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f10c8001050 tx=0x7f10c0005830 comp rx=0 tx=0).stop 2024-04-25T19:19:32.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.072+0000 7f10d7fff640 1 -- 172.21.15.76:0/2619993997 reap_dead start 2024-04-25T19:19:32.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.072+0000 7f10dddf3640 1 -- 172.21.15.76:0/2619993997 shutdown_connections 2024-04-25T19:19:32.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.072+0000 7f10dddf3640 1 -- 172.21.15.76:0/2619993997 >> 172.21.15.76:0/2619993997 conn(0x7f10d806d6d0 msgr2=0x7f10d806dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:32.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.072+0000 7f10dddf3640 1 -- 172.21.15.76:0/2619993997 shutdown_connections 2024-04-25T19:19:32.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.072+0000 7f10dddf3640 1 -- 172.21.15.76:0/2619993997 wait complete. 2024-04-25T19:19:32.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.074+0000 7f10dddf3640 1 Processor -- start 2024-04-25T19:19:32.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.074+0000 7f10dddf3640 1 -- start start 2024-04-25T19:19:32.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.075+0000 7f10dddf3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10c00039b0 0x7f10d8150660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:32.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.075+0000 7f10dddf3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10d8159060 0x7f10d81f2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:32.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.075+0000 7f10dddf3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10d8159ac0 0x7f10d81495f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:32.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.075+0000 7f10dddf3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f10d80cc6f0 con 0x7f10d8159060 2024-04-25T19:19:32.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.075+0000 7f10dddf3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f10d80cc570 con 0x7f10d8159ac0 2024-04-25T19:19:32.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.075+0000 7f10dddf3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f10d806ccd0 con 0x7f10c00039b0 2024-04-25T19:19:32.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.075+0000 7f10d6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10d8159060 0x7f10d81f2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:32.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.075+0000 7f10d6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10d8159060 0x7f10d81f2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50502/0 (socket says 172.21.15.76:50502) 2024-04-25T19:19:32.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.075+0000 7f10d6ffd640 1 -- 172.21.15.76:0/2592861753 learned_addr learned my addr 172.21.15.76:0/2592861753 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:32.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.076+0000 7f10d7fff640 1 --2- 172.21.15.76:0/2592861753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10d8159ac0 0x7f10d81495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:32.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.076+0000 7f10d77fe640 1 --2- 172.21.15.76:0/2592861753 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10c00039b0 0x7f10d8150660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:32.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.076+0000 7f10d6ffd640 1 -- 172.21.15.76:0/2592861753 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10c00039b0 msgr2=0x7f10d8150660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:32.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.076+0000 7f10d6ffd640 1 --2- 172.21.15.76:0/2592861753 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10c00039b0 0x7f10d8150660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.076+0000 7f10d6ffd640 1 -- 172.21.15.76:0/2592861753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10d8159ac0 msgr2=0x7f10d81495f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:32.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.076+0000 7f10d6ffd640 1 --2- 172.21.15.76:0/2592861753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10d8159ac0 0x7f10d81495f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.076+0000 7f10d6ffd640 1 -- 172.21.15.76:0/2592861753 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d815e310 con 0x7f10d8159060 2024-04-25T19:19:32.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.076+0000 7f10d6ffd640 1 --2- 172.21.15.76:0/2592861753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10d8159060 0x7f10d81f2d50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0118e0 tx=0x7f10cc011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:32.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.076+0000 7f10bffff640 1 -- 172.21.15.76:0/2592861753 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc00faf0 con 0x7f10d8159060 2024-04-25T19:19:32.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.076+0000 7f10bffff640 1 -- 172.21.15.76:0/2592861753 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10cc00fc90 con 0x7f10d8159060 2024-04-25T19:19:32.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.076+0000 7f10dddf3640 1 -- 172.21.15.76:0/2592861753 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d81f3290 con 0x7f10d8159060 2024-04-25T19:19:32.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.076+0000 7f10bffff640 1 -- 172.21.15.76:0/2592861753 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc014440 con 0x7f10d8159060 2024-04-25T19:19:32.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.076+0000 7f10dddf3640 1 -- 172.21.15.76:0/2592861753 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d815e030 con 0x7f10d8159060 2024-04-25T19:19:32.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.077+0000 7f10dddf3640 1 -- 172.21.15.76:0/2592861753 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d80709f0 con 0x7f10d8159060 2024-04-25T19:19:32.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.078+0000 7f10bffff640 1 -- 172.21.15.76:0/2592861753 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f10cc012040 con 0x7f10d8159060 2024-04-25T19:19:32.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.078+0000 7f10bffff640 1 --2- 172.21.15.76:0/2592861753 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f10c40744b0 0x7f10c40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:32.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.078+0000 7f10bffff640 1 -- 172.21.15.76:0/2592861753 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f10cc095a30 con 0x7f10d8159060 2024-04-25T19:19:32.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.079+0000 7f10d77fe640 1 --2- 172.21.15.76:0/2592861753 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f10c40744b0 0x7f10c40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:32.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.079+0000 7f10d77fe640 1 --2- 172.21.15.76:0/2592861753 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f10c40744b0 0x7f10c40769c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f10c8033000 tx=0x7f10c8009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:32.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.083+0000 7f10bffff640 1 -- 172.21.15.76:0/2592861753 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10d80709f0 con 0x7f10d8159060 2024-04-25T19:19:32.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.256+0000 7f10dddf3640 1 -- 172.21.15.76:0/2592861753 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f10d81f6770 con 0x7f10d8159060 2024-04-25T19:19:32.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.257+0000 7f10bffff640 1 -- 172.21.15.76:0/2592861753 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f10d81f6770 con 0x7f10d8159060 2024-04-25T19:19:32.259 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-04-25T19:19:32.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.258+0000 7f10dddf3640 1 -- 172.21.15.76:0/2592861753 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f10c40744b0 msgr2=0x7f10c40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:32.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.258+0000 7f10dddf3640 1 --2- 172.21.15.76:0/2592861753 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f10c40744b0 0x7f10c40769c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f10c8033000 tx=0x7f10c8009030 comp rx=0 tx=0).stop 2024-04-25T19:19:32.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.258+0000 7f10dddf3640 1 -- 172.21.15.76:0/2592861753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10d8159060 msgr2=0x7f10d81f2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:32.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.258+0000 7f10dddf3640 1 --2- 172.21.15.76:0/2592861753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10d8159060 0x7f10d81f2d50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0118e0 tx=0x7f10cc011da0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.259+0000 7f10dddf3640 1 -- 172.21.15.76:0/2592861753 shutdown_connections 2024-04-25T19:19:32.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.259+0000 7f10dddf3640 1 --2- 172.21.15.76:0/2592861753 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f10c40744b0 0x7f10c40769c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.259+0000 7f10dddf3640 1 --2- 172.21.15.76:0/2592861753 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10d8159ac0 0x7f10d81495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.259+0000 7f10dddf3640 1 --2- 172.21.15.76:0/2592861753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10d8159060 0x7f10d81f2d50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.259+0000 7f10dddf3640 1 --2- 172.21.15.76:0/2592861753 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f10c00039b0 0x7f10d8150660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.259+0000 7f10dddf3640 1 -- 172.21.15.76:0/2592861753 >> 172.21.15.76:0/2592861753 conn(0x7f10d806d6d0 msgr2=0x7f10d8151c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:32.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.259+0000 7f10dddf3640 1 -- 172.21.15.76:0/2592861753 shutdown_connections 2024-04-25T19:19:32.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.259+0000 7f10dddf3640 1 -- 172.21.15.76:0/2592861753 wait complete. 2024-04-25T19:19:32.272 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T19:19:32.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.381+0000 7fbadd627640 1 Processor -- start 2024-04-25T19:19:32.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.381+0000 7fbadd627640 1 -- start start 2024-04-25T19:19:32.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.383+0000 7fbadd627640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbad80d0810 0x7fbad806f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:32.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.383+0000 7fbadd627640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbad806e870 0x7fbad80cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:32.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.383+0000 7fbadd627640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbad815ce80 con 0x7fbad80cea70 2024-04-25T19:19:32.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.383+0000 7fbadd627640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fbad815d160 con 0x7fbad806e870 2024-04-25T19:19:32.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.383+0000 7fbadd627640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fbad815d440 con 0x7fbad80d0810 2024-04-25T19:19:32.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.383+0000 7fbad77fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbad806e870 0x7fbad80cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:32.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.383+0000 7fbad77fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbad806e870 0x7fbad80cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50512/0 (socket says 172.21.15.76:50512) 2024-04-25T19:19:32.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.383+0000 7fbad77fe640 1 -- 172.21.15.76:0/3644695777 learned_addr learned my addr 172.21.15.76:0/3644695777 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:32.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.383+0000 7fbad6ffd640 1 --2- 172.21.15.76:0/3644695777 >> v2:172.21.15.105:3300/0 conn(0x7fbad80d0810 0x7fbad806f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:32.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.383+0000 7fbad77fe640 1 -- 172.21.15.76:0/3644695777 >> v2:172.21.15.105:3300/0 conn(0x7fbad80d0810 msgr2=0x7fbad806f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:32.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.383+0000 7fbad77fe640 1 --2- 172.21.15.76:0/3644695777 >> v2:172.21.15.105:3300/0 conn(0x7fbad80d0810 0x7fbad806f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.383+0000 7fbad77fe640 1 -- 172.21.15.76:0/3644695777 >> v1:172.21.15.105:6789/0 conn(0x7fbad80cea70 legacy=0x7fbad806e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:32.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.383+0000 7fbad77fe640 1 -- 172.21.15.76:0/3644695777 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad815d720 con 0x7fbad806e870 2024-04-25T19:19:32.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.384+0000 7fbad77fe640 1 --2- 172.21.15.76:0/3644695777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbad806e870 0x7fbad80cee80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbad808fb10 tx=0x7fbad002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe0f26c73144d46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:32.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.384+0000 7fbad67fc640 1 -- 172.21.15.76:0/3644695777 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad0036070 con 0x7fbad806e870 2024-04-25T19:19:32.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.384+0000 7fbad67fc640 1 -- 172.21.15.76:0/3644695777 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbad0031600 con 0x7fbad806e870 2024-04-25T19:19:32.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.384+0000 7fbad67fc640 1 -- 172.21.15.76:0/3644695777 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad00319d0 con 0x7fbad806e870 2024-04-25T19:19:32.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.385+0000 7fbadd627640 1 -- 172.21.15.76:0/3644695777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbad806e870 msgr2=0x7fbad80cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:32.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.385+0000 7fbadd627640 1 --2- 172.21.15.76:0/3644695777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbad806e870 0x7fbad80cee80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbad808fb10 tx=0x7fbad002a730 comp rx=0 tx=0).stop 2024-04-25T19:19:32.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.385+0000 7fbadd627640 1 -- 172.21.15.76:0/3644695777 shutdown_connections 2024-04-25T19:19:32.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.385+0000 7fbadd627640 1 --2- 172.21.15.76:0/3644695777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbad806e870 0x7fbad80cee80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.385+0000 7fbadd627640 1 --2- 172.21.15.76:0/3644695777 >> v2:172.21.15.105:3300/0 conn(0x7fbad80d0810 0x7fbad806f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.385+0000 7fbadd627640 1 -- 172.21.15.76:0/3644695777 >> 172.21.15.76:0/3644695777 conn(0x7fbad80b3c00 msgr2=0x7fbad80b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:32.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.385+0000 7fbadd627640 1 -- 172.21.15.76:0/3644695777 shutdown_connections 2024-04-25T19:19:32.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.385+0000 7fbadd627640 1 -- 172.21.15.76:0/3644695777 wait complete. 2024-04-25T19:19:32.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.387+0000 7fbadd627640 1 Processor -- start 2024-04-25T19:19:32.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.387+0000 7fbadd627640 1 -- start start 2024-04-25T19:19:32.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.388+0000 7fbadd627640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbad806e870 0x7fbad81f3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:32.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.388+0000 7fbadd627640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbad80cea70 0x7fbad8155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:32.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.388+0000 7fbadd627640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbad80d0810 0x7fbad815e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:32.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.388+0000 7fbadd627640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbad815cb70 con 0x7fbad806e870 2024-04-25T19:19:32.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.388+0000 7fbadd627640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbad806dc40 con 0x7fbad80cea70 2024-04-25T19:19:32.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.388+0000 7fbadd627640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fbad806f540 con 0x7fbad80d0810 2024-04-25T19:19:32.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.388+0000 7fbad6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbad806e870 0x7fbad81f3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:32.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.388+0000 7fbad6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbad806e870 0x7fbad81f3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50516/0 (socket says 172.21.15.76:50516) 2024-04-25T19:19:32.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.388+0000 7fbad6ffd640 1 -- 172.21.15.76:0/3422497782 learned_addr learned my addr 172.21.15.76:0/3422497782 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:32.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.388+0000 7fbacffff640 1 --2- 172.21.15.76:0/3422497782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbad80cea70 0x7fbad8155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:32.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.388+0000 7fbad77fe640 1 --2- 172.21.15.76:0/3422497782 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbad80d0810 0x7fbad815e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:32.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.388+0000 7fbad6ffd640 1 -- 172.21.15.76:0/3422497782 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbad80d0810 msgr2=0x7fbad815e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:32.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.388+0000 7fbad6ffd640 1 --2- 172.21.15.76:0/3422497782 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbad80d0810 0x7fbad815e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.388+0000 7fbad6ffd640 1 -- 172.21.15.76:0/3422497782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbad80cea70 msgr2=0x7fbad8155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:32.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.388+0000 7fbad6ffd640 1 --2- 172.21.15.76:0/3422497782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbad80cea70 0x7fbad8155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.388+0000 7fbad6ffd640 1 -- 172.21.15.76:0/3422497782 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad81f3550 con 0x7fbad806e870 2024-04-25T19:19:32.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.389+0000 7fbad6ffd640 1 --2- 172.21.15.76:0/3422497782 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbad806e870 0x7fbad81f3010 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbac800d620 tx=0x7fbac800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:32.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.389+0000 7fbacf7fe640 1 -- 172.21.15.76:0/3422497782 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac800b620 con 0x7fbad806e870 2024-04-25T19:19:32.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.389+0000 7fbacf7fe640 1 -- 172.21.15.76:0/3422497782 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbac8019070 con 0x7fbad806e870 2024-04-25T19:19:32.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.389+0000 7fbacf7fe640 1 -- 172.21.15.76:0/3422497782 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac800ba00 con 0x7fbad806e870 2024-04-25T19:19:32.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.389+0000 7fbadd627640 1 -- 172.21.15.76:0/3422497782 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad8151460 con 0x7fbad806e870 2024-04-25T19:19:32.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.389+0000 7fbadd627640 1 -- 172.21.15.76:0/3422497782 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad8151a20 con 0x7fbad806e870 2024-04-25T19:19:32.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.390+0000 7fbadd627640 1 -- 172.21.15.76:0/3422497782 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad806a390 con 0x7fbad806e870 2024-04-25T19:19:32.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.390+0000 7fbacf7fe640 1 -- 172.21.15.76:0/3422497782 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbac800bba0 con 0x7fbad806e870 2024-04-25T19:19:32.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.391+0000 7fbacf7fe640 1 --2- 172.21.15.76:0/3422497782 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbab80744e0 0x7fbab80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:32.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.391+0000 7fbacf7fe640 1 -- 172.21.15.76:0/3422497782 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fbac8095690 con 0x7fbad806e870 2024-04-25T19:19:32.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.391+0000 7fbacffff640 1 --2- 172.21.15.76:0/3422497782 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbab80744e0 0x7fbab80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:32.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.392+0000 7fbacffff640 1 --2- 172.21.15.76:0/3422497782 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbab80744e0 0x7fbab80769f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbac0007fd0 tx=0x7fbac0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:32.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.396+0000 7fbacf7fe640 1 -- 172.21.15.76:0/3422497782 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbac805fdb0 con 0x7fbad806e870 2024-04-25T19:19:32.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.573+0000 7fbadd627640 1 -- 172.21.15.76:0/3422497782 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbad815a8c0 con 0x7fbad806e870 2024-04-25T19:19:32.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.574+0000 7fbacf7fe640 1 -- 172.21.15.76:0/3422497782 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7fbac805f6b0 con 0x7fbad806e870 2024-04-25T19:19:32.575 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-04-25T19:19:32.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.576+0000 7fbadd627640 1 -- 172.21.15.76:0/3422497782 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbab80744e0 msgr2=0x7fbab80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:32.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.576+0000 7fbadd627640 1 --2- 172.21.15.76:0/3422497782 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbab80744e0 0x7fbab80769f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbac0007fd0 tx=0x7fbac0009040 comp rx=0 tx=0).stop 2024-04-25T19:19:32.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.576+0000 7fbadd627640 1 -- 172.21.15.76:0/3422497782 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbad806e870 msgr2=0x7fbad81f3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:32.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.576+0000 7fbadd627640 1 --2- 172.21.15.76:0/3422497782 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbad806e870 0x7fbad81f3010 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbac800d620 tx=0x7fbac800dae0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.576+0000 7fbadd627640 1 -- 172.21.15.76:0/3422497782 shutdown_connections 2024-04-25T19:19:32.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.576+0000 7fbadd627640 1 --2- 172.21.15.76:0/3422497782 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbab80744e0 0x7fbab80769f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.576+0000 7fbadd627640 1 --2- 172.21.15.76:0/3422497782 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbad80d0810 0x7fbad815e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.576+0000 7fbadd627640 1 --2- 172.21.15.76:0/3422497782 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbad80cea70 0x7fbad8155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.576+0000 7fbadd627640 1 --2- 172.21.15.76:0/3422497782 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbad806e870 0x7fbad81f3010 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.576+0000 7fbadd627640 1 -- 172.21.15.76:0/3422497782 >> 172.21.15.76:0/3422497782 conn(0x7fbad80b3c00 msgr2=0x7fbad80d1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:32.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.576+0000 7fbadd627640 1 -- 172.21.15.76:0/3422497782 shutdown_connections 2024-04-25T19:19:32.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.576+0000 7fbadd627640 1 -- 172.21.15.76:0/3422497782 wait complete. 2024-04-25T19:19:32.589 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T19:19:32.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.698+0000 7fcbcb91c640 1 Processor -- start 2024-04-25T19:19:32.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.699+0000 7fcbcb91c640 1 -- start start 2024-04-25T19:19:32.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.700+0000 7fcbcb91c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcbc40ca750 0x7fcbc40cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:32.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.700+0000 7fcbcb91c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcbc40cc6f0 0x7fcbc40ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:32.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.700+0000 7fcbcb91c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbc40cd040 con 0x7fcbc41600b0 2024-04-25T19:19:32.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.700+0000 7fcbcb91c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fcbc41604c0 con 0x7fcbc40ca750 2024-04-25T19:19:32.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.700+0000 7fcbcb91c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fcbc415dae0 con 0x7fcbc40cc6f0 2024-04-25T19:19:32.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.700+0000 7fcbc8eb7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcbc40ca750 0x7fcbc40cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:32.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.700+0000 7fcbc8eb7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcbc40ca750 0x7fcbc40cab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50520/0 (socket says 172.21.15.76:50520) 2024-04-25T19:19:32.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.700+0000 7fcbc8eb7640 1 -- 172.21.15.76:0/3802934156 learned_addr learned my addr 172.21.15.76:0/3802934156 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:32.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.700+0000 7fcbc9eb9640 1 --2- 172.21.15.76:0/3802934156 >> v2:172.21.15.105:3300/0 conn(0x7fcbc40cc6f0 0x7fcbc40ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:32.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.700+0000 7fcbc8eb7640 1 -- 172.21.15.76:0/3802934156 >> v2:172.21.15.105:3300/0 conn(0x7fcbc40cc6f0 msgr2=0x7fcbc40ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:32.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.700+0000 7fcbc8eb7640 1 --2- 172.21.15.76:0/3802934156 >> v2:172.21.15.105:3300/0 conn(0x7fcbc40cc6f0 0x7fcbc40ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.700+0000 7fcbc8eb7640 1 -- 172.21.15.76:0/3802934156 >> v1:172.21.15.105:6789/0 conn(0x7fcbc41600b0 legacy=0x7fcbc415d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:32.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.701+0000 7fcbc8eb7640 1 -- 172.21.15.76:0/3802934156 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc41606f0 con 0x7fcbc40ca750 2024-04-25T19:19:32.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.701+0000 7fcbc8eb7640 1 --2- 172.21.15.76:0/3802934156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbc40ca750 0x7fcbc40cab60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40cdbe0 tx=0x7fcbb402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e270def9b6bddb07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:32.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.701+0000 7fcbbbfff640 1 -- 172.21.15.76:0/3802934156 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4037070 con 0x7fcbc40ca750 2024-04-25T19:19:32.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.701+0000 7fcbbbfff640 1 -- 172.21.15.76:0/3802934156 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb402ae50 con 0x7fcbc40ca750 2024-04-25T19:19:32.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.701+0000 7fcbbbfff640 1 -- 172.21.15.76:0/3802934156 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4032860 con 0x7fcbc40ca750 2024-04-25T19:19:32.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.702+0000 7fcbcb91c640 1 -- 172.21.15.76:0/3802934156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbc40ca750 msgr2=0x7fcbc40cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:32.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.702+0000 7fcbcb91c640 1 --2- 172.21.15.76:0/3802934156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbc40ca750 0x7fcbc40cab60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40cdbe0 tx=0x7fcbb402a300 comp rx=0 tx=0).stop 2024-04-25T19:19:32.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.702+0000 7fcbcb91c640 1 -- 172.21.15.76:0/3802934156 shutdown_connections 2024-04-25T19:19:32.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.702+0000 7fcbcb91c640 1 --2- 172.21.15.76:0/3802934156 >> v2:172.21.15.105:3300/0 conn(0x7fcbc40cc6f0 0x7fcbc40ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.702+0000 7fcbcb91c640 1 --2- 172.21.15.76:0/3802934156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbc40ca750 0x7fcbc40cab60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.702+0000 7fcbcb91c640 1 -- 172.21.15.76:0/3802934156 >> 172.21.15.76:0/3802934156 conn(0x7fcbc4068b10 msgr2=0x7fcbc4072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:32.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.702+0000 7fcbcb91c640 1 -- 172.21.15.76:0/3802934156 shutdown_connections 2024-04-25T19:19:32.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.702+0000 7fcbcb91c640 1 -- 172.21.15.76:0/3802934156 wait complete. 2024-04-25T19:19:32.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.704+0000 7fcbcb91c640 1 Processor -- start 2024-04-25T19:19:32.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.704+0000 7fcbcb91c640 1 -- start start 2024-04-25T19:19:32.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.705+0000 7fcbcb91c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbc40ca750 0x7fcbc41e8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:32.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.705+0000 7fcbcb91c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbc40cc6f0 0x7fcbc41538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:32.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.705+0000 7fcbcb91c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcbc41600b0 0x7fcbc4153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:32.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.705+0000 7fcbcb91c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc415cbd0 con 0x7fcbc40cc6f0 2024-04-25T19:19:32.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.705+0000 7fcbcb91c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc4160cb0 con 0x7fcbc40ca750 2024-04-25T19:19:32.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.705+0000 7fcbcb91c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fcbc415ceb0 con 0x7fcbc41600b0 2024-04-25T19:19:32.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.705+0000 7fcbc8eb7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbc40cc6f0 0x7fcbc41538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:32.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.705+0000 7fcbc8eb7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbc40cc6f0 0x7fcbc41538f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50532/0 (socket says 172.21.15.76:50532) 2024-04-25T19:19:32.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.705+0000 7fcbc8eb7640 1 -- 172.21.15.76:0/1974006083 learned_addr learned my addr 172.21.15.76:0/1974006083 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:32.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.705+0000 7fcbc9eb9640 1 --2- 172.21.15.76:0/1974006083 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcbc41600b0 0x7fcbc4153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:32.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.705+0000 7fcbc96b8640 1 --2- 172.21.15.76:0/1974006083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbc40ca750 0x7fcbc41e8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:32.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.706+0000 7fcbc8eb7640 1 -- 172.21.15.76:0/1974006083 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcbc41600b0 msgr2=0x7fcbc4153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:32.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.706+0000 7fcbc8eb7640 1 --2- 172.21.15.76:0/1974006083 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcbc41600b0 0x7fcbc4153e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.706+0000 7fcbc8eb7640 1 -- 172.21.15.76:0/1974006083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbc40ca750 msgr2=0x7fcbc41e8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:32.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.706+0000 7fcbc8eb7640 1 --2- 172.21.15.76:0/1974006083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbc40ca750 0x7fcbc41e8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.706+0000 7fcbc8eb7640 1 -- 172.21.15.76:0/1974006083 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc41e92a0 con 0x7fcbc40cc6f0 2024-04-25T19:19:32.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.706+0000 7fcbc96b8640 1 --2- 172.21.15.76:0/1974006083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbc40ca750 0x7fcbc41e8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:19:32.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.706+0000 7fcbc8eb7640 1 --2- 172.21.15.76:0/1974006083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbc40cc6f0 0x7fcbc41538f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcbb402a340 tx=0x7fcbb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:32.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.706+0000 7fcbb9ffb640 1 -- 172.21.15.76:0/1974006083 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4041020 con 0x7fcbc40cc6f0 2024-04-25T19:19:32.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.706+0000 7fcbb9ffb640 1 -- 172.21.15.76:0/1974006083 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb403c670 con 0x7fcbc40cc6f0 2024-04-25T19:19:32.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.706+0000 7fcbcb91c640 1 -- 172.21.15.76:0/1974006083 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc4156550 con 0x7fcbc40cc6f0 2024-04-25T19:19:32.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.706+0000 7fcbb9ffb640 1 -- 172.21.15.76:0/1974006083 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4037040 con 0x7fcbc40cc6f0 2024-04-25T19:19:32.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.706+0000 7fcbcb91c640 1 -- 172.21.15.76:0/1974006083 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc41519c0 con 0x7fcbc40cc6f0 2024-04-25T19:19:32.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.707+0000 7fcbcb91c640 1 -- 172.21.15.76:0/1974006083 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbc406a390 con 0x7fcbc40cc6f0 2024-04-25T19:19:32.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.708+0000 7fcbb9ffb640 1 -- 172.21.15.76:0/1974006083 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcbb4044020 con 0x7fcbc40cc6f0 2024-04-25T19:19:32.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.708+0000 7fcbb9ffb640 1 --2- 172.21.15.76:0/1974006083 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fcba80744b0 0x7fcba80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:32.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.708+0000 7fcbb9ffb640 1 -- 172.21.15.76:0/1974006083 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fcbb40b58f0 con 0x7fcbc40cc6f0 2024-04-25T19:19:32.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.708+0000 7fcbc96b8640 1 --2- 172.21.15.76:0/1974006083 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fcba80744b0 0x7fcba80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:32.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.709+0000 7fcbc96b8640 1 --2- 172.21.15.76:0/1974006083 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fcba80744b0 0x7fcba80769c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0001cd0 tx=0x7fcbc0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:32.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.713+0000 7fcbb9ffb640 1 -- 172.21.15.76:0/1974006083 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcbb404b030 con 0x7fcbc40cc6f0 2024-04-25T19:19:32.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.890+0000 7fcbcb91c640 1 -- 172.21.15.76:0/1974006083 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcbc415a4d0 con 0x7fcbc40cc6f0 2024-04-25T19:19:32.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.890+0000 7fcbb9ffb640 1 -- 172.21.15.76:0/1974006083 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fcbb4080090 con 0x7fcbc40cc6f0 2024-04-25T19:19:32.892 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-04-25T19:19:32.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.892+0000 7fcbcb91c640 1 -- 172.21.15.76:0/1974006083 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fcba80744b0 msgr2=0x7fcba80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:32.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.892+0000 7fcbcb91c640 1 --2- 172.21.15.76:0/1974006083 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fcba80744b0 0x7fcba80769c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0001cd0 tx=0x7fcbc0009040 comp rx=0 tx=0).stop 2024-04-25T19:19:32.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.892+0000 7fcbcb91c640 1 -- 172.21.15.76:0/1974006083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbc40cc6f0 msgr2=0x7fcbc41538f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:32.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.892+0000 7fcbcb91c640 1 --2- 172.21.15.76:0/1974006083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbc40cc6f0 0x7fcbc41538f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcbb402a340 tx=0x7fcbb40046c0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.892+0000 7fcbcb91c640 1 -- 172.21.15.76:0/1974006083 shutdown_connections 2024-04-25T19:19:32.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.892+0000 7fcbcb91c640 1 --2- 172.21.15.76:0/1974006083 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fcba80744b0 0x7fcba80769c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.892+0000 7fcbcb91c640 1 --2- 172.21.15.76:0/1974006083 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcbc41600b0 0x7fcbc4153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.892+0000 7fcbcb91c640 1 --2- 172.21.15.76:0/1974006083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcbc40cc6f0 0x7fcbc41538f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.892+0000 7fcbcb91c640 1 --2- 172.21.15.76:0/1974006083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcbc40ca750 0x7fcbc41e8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:32.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.892+0000 7fcbcb91c640 1 -- 172.21.15.76:0/1974006083 >> 172.21.15.76:0/1974006083 conn(0x7fcbc4068b10 msgr2=0x7fcbc406d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:32.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.892+0000 7fcbcb91c640 1 -- 172.21.15.76:0/1974006083 shutdown_connections 2024-04-25T19:19:32.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:32.892+0000 7fcbcb91c640 1 -- 172.21.15.76:0/1974006083 wait complete. 2024-04-25T19:19:32.905 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T19:19:33.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.014+0000 7ff102a61640 1 Processor -- start 2024-04-25T19:19:33.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.014+0000 7ff102a61640 1 -- start start 2024-04-25T19:19:33.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.015+0000 7ff102a61640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0fc0cb800 0x7ff0fc0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:33.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.015+0000 7ff102a61640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff0fc0d0510 0x7ff0fc1653b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:33.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.015+0000 7ff102a61640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0fc0c6850 con 0x7ff0fc0c62c0 2024-04-25T19:19:33.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.015+0000 7ff102a61640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff0fc15af90 con 0x7ff0fc0d0510 2024-04-25T19:19:33.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.015+0000 7ff102a61640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff0fc0c70f0 con 0x7ff0fc0cb800 2024-04-25T19:19:33.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.015+0000 7ff100ffe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff0fc0d0510 0x7ff0fc1653b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:33.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.015+0000 7ff0fbfff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff0fc0c62c0 0x7ff0fc15b420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35760/0 (socket says 172.21.15.76:35760) 2024-04-25T19:19:33.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.015+0000 7ff0fbfff640 1 -- 172.21.15.76:0/722944807 learned_addr learned my addr 172.21.15.76:0/722944807 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:33.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.015+0000 7ff0fb7fe640 1 --2- 172.21.15.76:0/722944807 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0fc0cb800 0x7ff0fc0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:33.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.015+0000 7ff100ffe640 1 -- 172.21.15.76:0/722944807 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0fc0cb800 msgr2=0x7ff0fc0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:33.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.015+0000 7ff100ffe640 1 --2- 172.21.15.76:0/722944807 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0fc0cb800 0x7ff0fc0cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:33.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.015+0000 7ff100ffe640 1 -- 172.21.15.76:0/722944807 >> v1:172.21.15.76:6789/0 conn(0x7ff0fc0c62c0 legacy=0x7ff0fc15b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:33.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.015+0000 7ff100ffe640 1 -- 172.21.15.76:0/722944807 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0fc0c6b30 con 0x7ff0fc0d0510 2024-04-25T19:19:33.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.015+0000 7ff0faffd640 1 -- 172.21.15.76:0/722944807 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 496148804 0 0) 0x7ff0fc0c6850 con 0x7ff0fc0c62c0 2024-04-25T19:19:33.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.016+0000 7ff100ffe640 1 --2- 172.21.15.76:0/722944807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0fc0d0510 0x7ff0fc1653b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009d90 tx=0x7ff0f402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb9e33e8307066f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:33.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.016+0000 7ff0faffd640 1 -- 172.21.15.76:0/722944807 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4007740 con 0x7ff0fc0d0510 2024-04-25T19:19:33.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.016+0000 7ff0faffd640 1 -- 172.21.15.76:0/722944807 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0f40078e0 con 0x7ff0fc0d0510 2024-04-25T19:19:33.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.016+0000 7ff0faffd640 1 -- 172.21.15.76:0/722944807 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4007cb0 con 0x7ff0fc0d0510 2024-04-25T19:19:33.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.017+0000 7ff102a61640 1 -- 172.21.15.76:0/722944807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0fc0d0510 msgr2=0x7ff0fc1653b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:33.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.017+0000 7ff102a61640 1 --2- 172.21.15.76:0/722944807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0fc0d0510 0x7ff0fc1653b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009d90 tx=0x7ff0f402a660 comp rx=0 tx=0).stop 2024-04-25T19:19:33.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.017+0000 7ff102a61640 1 -- 172.21.15.76:0/722944807 shutdown_connections 2024-04-25T19:19:33.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.017+0000 7ff102a61640 1 --2- 172.21.15.76:0/722944807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0fc0d0510 0x7ff0fc1653b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:33.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.017+0000 7ff102a61640 1 --2- 172.21.15.76:0/722944807 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0fc0cb800 0x7ff0fc0cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:33.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.017+0000 7ff102a61640 1 -- 172.21.15.76:0/722944807 >> 172.21.15.76:0/722944807 conn(0x7ff0fc0684e0 msgr2=0x7ff0fc0688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:33.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.017+0000 7ff102a61640 1 -- 172.21.15.76:0/722944807 shutdown_connections 2024-04-25T19:19:33.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.017+0000 7ff102a61640 1 -- 172.21.15.76:0/722944807 wait complete. 2024-04-25T19:19:33.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.019+0000 7ff102a61640 1 Processor -- start 2024-04-25T19:19:33.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.019+0000 7ff102a61640 1 -- start start 2024-04-25T19:19:33.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.020+0000 7ff102a61640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0fc0c62c0 0x7ff0fc1f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:33.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.020+0000 7ff102a61640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0fc0cb800 0x7ff0fc1e3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:33.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.020+0000 7ff102a61640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0fc0d0510 0x7ff0fc0cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:33.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.020+0000 7ff102a61640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc0cbe10 con 0x7ff0fc0cb800 2024-04-25T19:19:33.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.020+0000 7ff102a61640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc15ac20 con 0x7ff0fc0d0510 2024-04-25T19:19:33.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.020+0000 7ff102a61640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff0fc154ff0 con 0x7ff0fc0c62c0 2024-04-25T19:19:33.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.020+0000 7ff0fb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0fc0cb800 0x7ff0fc1e3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:33.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.020+0000 7ff0fb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0fc0cb800 0x7ff0fc1e3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50560/0 (socket says 172.21.15.76:50560) 2024-04-25T19:19:33.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.020+0000 7ff0fb7fe640 1 -- 172.21.15.76:0/620255954 learned_addr learned my addr 172.21.15.76:0/620255954 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:33.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.020+0000 7ff100ffe640 1 --2- 172.21.15.76:0/620255954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0fc0d0510 0x7ff0fc0cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:33.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.020+0000 7ff0fbfff640 1 --2- 172.21.15.76:0/620255954 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0fc0c62c0 0x7ff0fc1f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:33.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.021+0000 7ff0fb7fe640 1 -- 172.21.15.76:0/620255954 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0fc0c62c0 msgr2=0x7ff0fc1f4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:33.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.021+0000 7ff0fb7fe640 1 --2- 172.21.15.76:0/620255954 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0fc0c62c0 0x7ff0fc1f4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:33.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.021+0000 7ff0fb7fe640 1 -- 172.21.15.76:0/620255954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0fc0d0510 msgr2=0x7ff0fc0cca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:33.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.021+0000 7ff0fb7fe640 1 --2- 172.21.15.76:0/620255954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0fc0d0510 0x7ff0fc0cca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:33.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.021+0000 7ff0fb7fe640 1 -- 172.21.15.76:0/620255954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0fc1f0cb0 con 0x7ff0fc0cb800 2024-04-25T19:19:33.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.021+0000 7ff0fb7fe640 1 --2- 172.21.15.76:0/620255954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0fc0cb800 0x7ff0fc1e3e60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff0f000d5d0 tx=0x7ff0f000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:33.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.021+0000 7ff0f8ff9640 1 -- 172.21.15.76:0/620255954 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0004240 con 0x7ff0fc0cb800 2024-04-25T19:19:33.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.021+0000 7ff0f8ff9640 1 -- 172.21.15.76:0/620255954 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0f00043e0 con 0x7ff0fc0cb800 2024-04-25T19:19:33.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.021+0000 7ff102a61640 1 -- 172.21.15.76:0/620255954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0fc1f0f90 con 0x7ff0fc0cb800 2024-04-25T19:19:33.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.021+0000 7ff0f8ff9640 1 -- 172.21.15.76:0/620255954 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0003830 con 0x7ff0fc0cb800 2024-04-25T19:19:33.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.021+0000 7ff102a61640 1 -- 172.21.15.76:0/620255954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0fc1f1550 con 0x7ff0fc0cb800 2024-04-25T19:19:33.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.022+0000 7ff102a61640 1 -- 172.21.15.76:0/620255954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0fc06a390 con 0x7ff0fc0cb800 2024-04-25T19:19:33.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.022+0000 7ff0f8ff9640 1 -- 172.21.15.76:0/620255954 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff0f00039d0 con 0x7ff0fc0cb800 2024-04-25T19:19:33.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.023+0000 7ff0f8ff9640 1 --2- 172.21.15.76:0/620255954 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff0e80744e0 0x7ff0e80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:33.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.023+0000 7ff0f8ff9640 1 -- 172.21.15.76:0/620255954 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7ff0f0095460 con 0x7ff0fc0cb800 2024-04-25T19:19:33.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.023+0000 7ff0fbfff640 1 --2- 172.21.15.76:0/620255954 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff0e80744e0 0x7ff0e80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:33.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.023+0000 7ff0fbfff640 1 --2- 172.21.15.76:0/620255954 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff0e80744e0 0x7ff0e80769f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec000f60 tx=0x7ff0ec009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:33.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.027+0000 7ff0f8ff9640 1 -- 172.21.15.76:0/620255954 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff0f005f520 con 0x7ff0fc0cb800 2024-04-25T19:19:33.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.215+0000 7ff102a61640 1 -- 172.21.15.76:0/620255954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff0fc0cc0b0 con 0x7ff0fc0cb800 2024-04-25T19:19:33.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.216+0000 7ff0f8ff9640 1 -- 172.21.15.76:0/620255954 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7ff0f0063820 con 0x7ff0fc0cb800 2024-04-25T19:19:33.217 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2024-04-25T19:19:33.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.218+0000 7ff102a61640 1 -- 172.21.15.76:0/620255954 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff0e80744e0 msgr2=0x7ff0e80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:33.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.218+0000 7ff102a61640 1 --2- 172.21.15.76:0/620255954 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff0e80744e0 0x7ff0e80769f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec000f60 tx=0x7ff0ec009040 comp rx=0 tx=0).stop 2024-04-25T19:19:33.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.218+0000 7ff102a61640 1 -- 172.21.15.76:0/620255954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0fc0cb800 msgr2=0x7ff0fc1e3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:33.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.218+0000 7ff102a61640 1 --2- 172.21.15.76:0/620255954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0fc0cb800 0x7ff0fc1e3e60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff0f000d5d0 tx=0x7ff0f000da90 comp rx=0 tx=0).stop 2024-04-25T19:19:33.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.218+0000 7ff102a61640 1 -- 172.21.15.76:0/620255954 shutdown_connections 2024-04-25T19:19:33.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.218+0000 7ff102a61640 1 --2- 172.21.15.76:0/620255954 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff0e80744e0 0x7ff0e80769f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:33.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.218+0000 7ff102a61640 1 --2- 172.21.15.76:0/620255954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0fc0d0510 0x7ff0fc0cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:33.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.218+0000 7ff102a61640 1 --2- 172.21.15.76:0/620255954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0fc0cb800 0x7ff0fc1e3e60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:33.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.218+0000 7ff102a61640 1 --2- 172.21.15.76:0/620255954 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff0fc0c62c0 0x7ff0fc1f4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:33.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.218+0000 7ff102a61640 1 -- 172.21.15.76:0/620255954 >> 172.21.15.76:0/620255954 conn(0x7ff0fc0684e0 msgr2=0x7ff0fc0688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:33.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.218+0000 7ff102a61640 1 -- 172.21.15.76:0/620255954 shutdown_connections 2024-04-25T19:19:33.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.218+0000 7ff102a61640 1 -- 172.21.15.76:0/620255954 wait complete. 2024-04-25T19:19:33.231 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi076.front.sepia.ceph.com... 2024-04-25T19:19:33.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:19:33.231 DEBUG:teuthology.orchestra.run.smithi076:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T19:19:33.255 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:19:33.255 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[42247]: fuse finished with error 0 and tester_r 0 2024-04-25T19:19:33.342 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-04-25T19:19:33.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:19:33.343 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T19:19:33.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:19:33.370 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-04-25T19:19:33.425 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T19:19:33.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:19:33.425 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T19:19:33.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:19:33.489 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link delete ceph-brx 2024-04-25T19:19:33.582 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T19:19:33.592 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T19:19:33.592 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T19:19:33.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.700+0000 7f63ad5ab640 1 Processor -- start 2024-04-25T19:19:33.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.700+0000 7f63ad5ab640 1 -- start start 2024-04-25T19:19:33.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.701+0000 7f63ad5ab640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63a8159520 0x7f63a8159930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:33.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.701+0000 7f63ad5ab640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63a815c820 con 0x7f63a81530b0 2024-04-25T19:19:33.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.701+0000 7f63ad5ab640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63a815cb00 con 0x7f63a8073dd0 2024-04-25T19:19:33.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.701+0000 7f63ad5ab640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f63a815d3a0 con 0x7f63a8159520 2024-04-25T19:19:33.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.702+0000 7f63a6ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f63a81530b0 0x7f63a80c5020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35762/0 (socket says 172.21.15.76:35762) 2024-04-25T19:19:33.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.702+0000 7f63a6ffd640 1 -- 172.21.15.76:0/1349192679 learned_addr learned my addr 172.21.15.76:0/1349192679 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:33.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.702+0000 7f63a5ffb640 1 -- 172.21.15.76:0/1349192679 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2270163797 0 0) 0x7f63a815c820 con 0x7f63a81530b0 2024-04-25T19:19:33.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.702+0000 7f63a5ffb640 1 -- 172.21.15.76:0/1349192679 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6390004630 con 0x7f63a81530b0 2024-04-25T19:19:33.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.702+0000 7f63a67fc640 1 --2- 172.21.15.76:0/1349192679 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63a8159520 0x7f63a8159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:33.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.702+0000 7f63a5ffb640 1 -- 172.21.15.76:0/1349192679 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1151509540 0 0) 0x7f6390004630 con 0x7f63a81530b0 2024-04-25T19:19:33.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.702+0000 7f63a5ffb640 1 -- 172.21.15.76:0/1349192679 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63a8159520 msgr2=0x7f63a8159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:33.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.702+0000 7f63a5ffb640 1 --2- 172.21.15.76:0/1349192679 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63a8159520 0x7f63a8159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:33.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.702+0000 7f63a5ffb640 1 -- 172.21.15.76:0/1349192679 >> v1:172.21.15.105:6789/0 conn(0x7f63a8073dd0 legacy=0x7f63a80741e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:33.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.702+0000 7f63a5ffb640 1 -- 172.21.15.76:0/1349192679 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a815cde0 con 0x7f63a81530b0 2024-04-25T19:19:33.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.702+0000 7f63a5ffb640 1 -- 172.21.15.76:0/1349192679 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f63980019f0 con 0x7f63a81530b0 2024-04-25T19:19:33.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.702+0000 7f63a5ffb640 1 -- 172.21.15.76:0/1349192679 >> v1:172.21.15.76:6789/0 conn(0x7f63a81530b0 legacy=0x7f63a80c5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:33.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.702+0000 7f63a5ffb640 1 --2- 172.21.15.76:0/1349192679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6390004af0 0x7f6390004f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:33.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.702+0000 7f63a5ffb640 1 --2- 172.21.15.76:0/1349192679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6390005910 0x7f6390005d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:33.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.702+0000 7f63a5ffb640 1 --2- 172.21.15.76:0/1349192679 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63a8159520 0x7f6390006310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:33.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.702+0000 7f63a5ffb640 1 -- 172.21.15.76:0/1349192679 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f63a815d3a0 con 0x7f6390004af0 2024-04-25T19:19:33.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.702+0000 7f63a5ffb640 1 -- 172.21.15.76:0/1349192679 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f639000cc90 con 0x7f6390005910 2024-04-25T19:19:33.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.702+0000 7f63a5ffb640 1 -- 172.21.15.76:0/1349192679 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f639000ce90 con 0x7f63a8159520 2024-04-25T19:19:33.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.702+0000 7f63a6ffd640 1 --2- 172.21.15.76:0/1349192679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6390004af0 0x7f6390004f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:33.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.702+0000 7f63a67fc640 1 --2- 172.21.15.76:0/1349192679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6390005910 0x7f6390005d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:33.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.702+0000 7f63a77fe640 1 --2- 172.21.15.76:0/1349192679 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63a8159520 0x7f6390006310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:33.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.703+0000 7f63a6ffd640 1 -- 172.21.15.76:0/1349192679 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63a8159520 msgr2=0x7f6390006310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:33.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.703+0000 7f63a6ffd640 1 --2- 172.21.15.76:0/1349192679 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63a8159520 0x7f6390006310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:33.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.703+0000 7f63a6ffd640 1 -- 172.21.15.76:0/1349192679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6390005910 msgr2=0x7f6390005d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:33.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.703+0000 7f63a6ffd640 1 --2- 172.21.15.76:0/1349192679 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6390005910 0x7f6390005d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:33.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.703+0000 7f63a6ffd640 1 -- 172.21.15.76:0/1349192679 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6390004630 con 0x7f6390004af0 2024-04-25T19:19:33.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.703+0000 7f63a6ffd640 1 --2- 172.21.15.76:0/1349192679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6390004af0 0x7f6390004f60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f639800a830 tx=0x7f6390006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df68658b0047c3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:33.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.703+0000 7f63a5ffb640 1 -- 172.21.15.76:0/1349192679 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639800a480 con 0x7f6390004af0 2024-04-25T19:19:33.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.704+0000 7f63a5ffb640 1 -- 172.21.15.76:0/1349192679 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f639802f070 con 0x7f6390004af0 2024-04-25T19:19:33.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.704+0000 7f63a5ffb640 1 -- 172.21.15.76:0/1349192679 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639802bcd0 con 0x7f6390004af0 2024-04-25T19:19:33.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.704+0000 7f63ad5ab640 1 -- 172.21.15.76:0/1349192679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6390004af0 msgr2=0x7f6390004f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:33.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.704+0000 7f63ad5ab640 1 --2- 172.21.15.76:0/1349192679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6390004af0 0x7f6390004f60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f639800a830 tx=0x7f6390006940 comp rx=0 tx=0).stop 2024-04-25T19:19:33.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.704+0000 7f63a77fe640 1 -- 172.21.15.76:0/1349192679 reap_dead start 2024-04-25T19:19:33.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.704+0000 7f63ad5ab640 1 -- 172.21.15.76:0/1349192679 shutdown_connections 2024-04-25T19:19:33.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.704+0000 7f63ad5ab640 1 -- 172.21.15.76:0/1349192679 >> 172.21.15.76:0/1349192679 conn(0x7f63a8070800 msgr2=0x7f63a8070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:33.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.704+0000 7f63ad5ab640 1 -- 172.21.15.76:0/1349192679 shutdown_connections 2024-04-25T19:19:33.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.704+0000 7f63ad5ab640 1 -- 172.21.15.76:0/1349192679 wait complete. 2024-04-25T19:19:33.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.706+0000 7f63ad5ab640 1 Processor -- start 2024-04-25T19:19:33.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.706+0000 7f63ad5ab640 1 -- start start 2024-04-25T19:19:33.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.707+0000 7f63ad5ab640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6390004af0 0x7f63a814b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:33.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.707+0000 7f63ad5ab640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63a814c740 0x7f63a81f3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:33.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.707+0000 7f63ad5ab640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63a815b250 0x7f63a80c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:33.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.707+0000 7f63ad5ab640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f63a81534c0 con 0x7f63a815b250 2024-04-25T19:19:33.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.707+0000 7f63ad5ab640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f63a8159000 con 0x7f6390004af0 2024-04-25T19:19:33.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.707+0000 7f63ad5ab640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f63a815c450 con 0x7f63a814c740 2024-04-25T19:19:33.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.708+0000 7f63a77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63a815b250 0x7f63a80c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:33.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.708+0000 7f63a77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63a815b250 0x7f63a80c88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50574/0 (socket says 172.21.15.76:50574) 2024-04-25T19:19:33.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.708+0000 7f63a77fe640 1 -- 172.21.15.76:0/3403536547 learned_addr learned my addr 172.21.15.76:0/3403536547 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:33.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.708+0000 7f63a6ffd640 1 --2- 172.21.15.76:0/3403536547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6390004af0 0x7f63a814b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:33.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.708+0000 7f63a67fc640 1 --2- 172.21.15.76:0/3403536547 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63a814c740 0x7f63a81f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:33.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.708+0000 7f63a77fe640 1 -- 172.21.15.76:0/3403536547 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63a814c740 msgr2=0x7f63a81f3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:33.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.708+0000 7f63a77fe640 1 --2- 172.21.15.76:0/3403536547 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63a814c740 0x7f63a81f3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:33.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.708+0000 7f63a77fe640 1 -- 172.21.15.76:0/3403536547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6390004af0 msgr2=0x7f63a814b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:33.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.708+0000 7f63a77fe640 1 --2- 172.21.15.76:0/3403536547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6390004af0 0x7f63a814b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:33.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.708+0000 7f63a77fe640 1 -- 172.21.15.76:0/3403536547 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a81f3ea0 con 0x7f63a815b250 2024-04-25T19:19:33.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.708+0000 7f63a77fe640 1 --2- 172.21.15.76:0/3403536547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63a815b250 0x7f63a80c88b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f63a80729e0 tx=0x7f63a000f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:33.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.708+0000 7f638f7fe640 1 -- 172.21.15.76:0/3403536547 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a00053a0 con 0x7f63a815b250 2024-04-25T19:19:33.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.708+0000 7f638f7fe640 1 -- 172.21.15.76:0/3403536547 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63a0018070 con 0x7f63a815b250 2024-04-25T19:19:33.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.708+0000 7f638f7fe640 1 -- 172.21.15.76:0/3403536547 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0003830 con 0x7f63a815b250 2024-04-25T19:19:33.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.708+0000 7f63ad5ab640 1 -- 172.21.15.76:0/3403536547 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a814a070 con 0x7f63a815b250 2024-04-25T19:19:33.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.708+0000 7f63ad5ab640 1 -- 172.21.15.76:0/3403536547 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a814a630 con 0x7f63a815b250 2024-04-25T19:19:33.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.709+0000 7f63ad5ab640 1 -- 172.21.15.76:0/3403536547 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63a81f4070 con 0x7f63a815b250 2024-04-25T19:19:33.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.710+0000 7f638f7fe640 1 -- 172.21.15.76:0/3403536547 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f63a00039d0 con 0x7f63a815b250 2024-04-25T19:19:33.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.710+0000 7f638f7fe640 1 --2- 172.21.15.76:0/3403536547 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f63940744b0 0x7f63940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:33.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.710+0000 7f638f7fe640 1 -- 172.21.15.76:0/3403536547 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f63a0094d30 con 0x7f63a815b250 2024-04-25T19:19:33.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.711+0000 7f63a6ffd640 1 --2- 172.21.15.76:0/3403536547 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f63940744b0 0x7f63940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:33.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.711+0000 7f63a6ffd640 1 --2- 172.21.15.76:0/3403536547 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f63940744b0 0x7f63940769c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6398033000 tx=0x7f6398009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:33.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.715+0000 7f638f7fe640 1 -- 172.21.15.76:0/3403536547 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f63a81f4070 con 0x7f63a815b250 2024-04-25T19:19:33.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.888+0000 7f63ad5ab640 1 -- 172.21.15.76:0/3403536547 --> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f63a806a390 con 0x7f63940744b0 2024-04-25T19:19:33.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.896+0000 7f638f7fe640 1 -- 172.21.15.76:0/3403536547 <== mgr.4109 v2:172.21.15.105:6832/2585775531 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+253023 (secure 0 0 0) 0x7f63a806a390 con 0x7f63940744b0 2024-04-25T19:19:33.900 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:19:33.902 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-04-25T19:19:33.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.899+0000 7f63ad5ab640 1 -- 172.21.15.76:0/3403536547 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f63940744b0 msgr2=0x7f63940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:33.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.899+0000 7f63ad5ab640 1 --2- 172.21.15.76:0/3403536547 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f63940744b0 0x7f63940769c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6398033000 tx=0x7f6398009030 comp rx=0 tx=0).stop 2024-04-25T19:19:33.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.899+0000 7f63ad5ab640 1 -- 172.21.15.76:0/3403536547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63a815b250 msgr2=0x7f63a80c88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:33.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.899+0000 7f63ad5ab640 1 --2- 172.21.15.76:0/3403536547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63a815b250 0x7f63a80c88b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f63a80729e0 tx=0x7f63a000f920 comp rx=0 tx=0).stop 2024-04-25T19:19:33.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.899+0000 7f63ad5ab640 1 -- 172.21.15.76:0/3403536547 shutdown_connections 2024-04-25T19:19:33.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.899+0000 7f63ad5ab640 1 --2- 172.21.15.76:0/3403536547 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f63940744b0 0x7f63940769c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:33.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.899+0000 7f63ad5ab640 1 --2- 172.21.15.76:0/3403536547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63a815b250 0x7f63a80c88b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:33.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.899+0000 7f63ad5ab640 1 --2- 172.21.15.76:0/3403536547 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f63a814c740 0x7f63a81f3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:33.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.899+0000 7f63ad5ab640 1 --2- 172.21.15.76:0/3403536547 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6390004af0 0x7f63a814b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:33.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.899+0000 7f63ad5ab640 1 -- 172.21.15.76:0/3403536547 >> 172.21.15.76:0/3403536547 conn(0x7f63a8070800 msgr2=0x7f63a8075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:33.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.899+0000 7f63ad5ab640 1 -- 172.21.15.76:0/3403536547 shutdown_connections 2024-04-25T19:19:33.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:33.899+0000 7f63ad5ab640 1 -- 172.21.15.76:0/3403536547 wait complete. 2024-04-25T19:19:33.912 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2024-04-25T19:19:31.895597+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":815163,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":70,"num_read_kb":59,"num_write":9661,"num_write_kb":25167785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9595,"ondisk_log_size":9595,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":306184,"kb_used_data":2696,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754668536,"statfs":{"total":773094113280,"available":772780580864,"internally_reserved":0,"allocated":2760704,"data_stored":2332686,"data_compressed":64838,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":310693424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1413477513,"num_objects":-337,"num_object_clones":0,"num_object_copies":-674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":340,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003521"},"pg_stats":[{"pgid":"3.3f","version":"23'138","reported_seq":155,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.040345+0000","last_change":"2024-04-25T19:15:19.433569+0000","last_active":"2024-04-25T19:19:18.040345+0000","last_peered":"2024-04-25T19:19:18.040345+0000","last_clean":"2024-04-25T19:19:18.040345+0000","last_became_active":"2024-04-25T19:15:19.431332+0000","last_became_peered":"2024-04-25T19:15:19.431332+0000","last_unstale":"2024-04-25T19:19:18.040345+0000","last_undegraded":"2024-04-25T19:19:18.040345+0000","last_fullsized":"2024-04-25T19:19:18.040345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:57:01.213834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'126","reported_seq":143,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.032165+0000","last_change":"2024-04-25T19:15:19.418695+0000","last_active":"2024-04-25T19:19:18.032165+0000","last_peered":"2024-04-25T19:19:18.032165+0000","last_clean":"2024-04-25T19:19:18.032165+0000","last_became_active":"2024-04-25T19:15:19.418235+0000","last_became_peered":"2024-04-25T19:15:19.418235+0000","last_unstale":"2024-04-25T19:19:18.032165+0000","last_undegraded":"2024-04-25T19:19:18.032165+0000","last_fullsized":"2024-04-25T19:19:18.032165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:30.516472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'144","reported_seq":161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.020533+0000","last_change":"2024-04-25T19:15:19.416983+0000","last_active":"2024-04-25T19:19:18.020533+0000","last_peered":"2024-04-25T19:19:18.020533+0000","last_clean":"2024-04-25T19:19:18.020533+0000","last_became_active":"2024-04-25T19:15:19.416443+0000","last_became_peered":"2024-04-25T19:15:19.416443+0000","last_unstale":"2024-04-25T19:19:18.020533+0000","last_undegraded":"2024-04-25T19:19:18.020533+0000","last_fullsized":"2024-04-25T19:19:18.020533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:02.794164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'165","reported_seq":183,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.098994+0000","last_change":"2024-04-25T19:15:19.429058+0000","last_active":"2024-04-25T19:19:18.098994+0000","last_peered":"2024-04-25T19:19:18.098994+0000","last_clean":"2024-04-25T19:19:18.098994+0000","last_became_active":"2024-04-25T19:15:19.428419+0000","last_became_peered":"2024-04-25T19:15:19.428419+0000","last_unstale":"2024-04-25T19:19:18.098994+0000","last_undegraded":"2024-04-25T19:19:18.098994+0000","last_fullsized":"2024-04-25T19:19:18.098994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:23:56.817616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'144","reported_seq":161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.073061+0000","last_change":"2024-04-25T19:15:19.427615+0000","last_active":"2024-04-25T19:19:18.073061+0000","last_peered":"2024-04-25T19:19:18.073061+0000","last_clean":"2024-04-25T19:19:18.073061+0000","last_became_active":"2024-04-25T19:15:19.426968+0000","last_became_peered":"2024-04-25T19:15:19.426968+0000","last_unstale":"2024-04-25T19:19:18.073061+0000","last_undegraded":"2024-04-25T19:19:18.073061+0000","last_fullsized":"2024-04-25T19:19:18.073061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:42:00.686712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'129","reported_seq":146,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.070490+0000","last_change":"2024-04-25T19:15:19.417602+0000","last_active":"2024-04-25T19:19:18.070490+0000","last_peered":"2024-04-25T19:19:18.070490+0000","last_clean":"2024-04-25T19:19:18.070490+0000","last_became_active":"2024-04-25T19:15:19.417292+0000","last_became_peered":"2024-04-25T19:15:19.417292+0000","last_unstale":"2024-04-25T19:19:18.070490+0000","last_undegraded":"2024-04-25T19:19:18.070490+0000","last_fullsized":"2024-04-25T19:19:18.070490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:37:43.583809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'159","reported_seq":176,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.041488+0000","last_change":"2024-04-25T19:15:19.428180+0000","last_active":"2024-04-25T19:19:18.041488+0000","last_peered":"2024-04-25T19:19:18.041488+0000","last_clean":"2024-04-25T19:19:18.041488+0000","last_became_active":"2024-04-25T19:15:19.427852+0000","last_became_peered":"2024-04-25T19:15:19.427852+0000","last_unstale":"2024-04-25T19:19:18.041488+0000","last_undegraded":"2024-04-25T19:19:18.041488+0000","last_fullsized":"2024-04-25T19:19:18.041488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:51.300880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'177","reported_seq":194,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.083993+0000","last_change":"2024-04-25T19:15:19.418253+0000","last_active":"2024-04-25T19:19:18.083993+0000","last_peered":"2024-04-25T19:19:18.083993+0000","last_clean":"2024-04-25T19:19:18.083993+0000","last_became_active":"2024-04-25T19:15:19.417183+0000","last_became_peered":"2024-04-25T19:15:19.417183+0000","last_unstale":"2024-04-25T19:19:18.083993+0000","last_undegraded":"2024-04-25T19:19:18.083993+0000","last_fullsized":"2024-04-25T19:19:18.083993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:05:13.181646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'141","reported_seq":158,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.050350+0000","last_change":"2024-04-25T19:15:19.431671+0000","last_active":"2024-04-25T19:19:18.050350+0000","last_peered":"2024-04-25T19:19:18.050350+0000","last_clean":"2024-04-25T19:19:18.050350+0000","last_became_active":"2024-04-25T19:15:19.431240+0000","last_became_peered":"2024-04-25T19:15:19.431240+0000","last_unstale":"2024-04-25T19:19:18.050350+0000","last_undegraded":"2024-04-25T19:19:18.050350+0000","last_fullsized":"2024-04-25T19:19:18.050350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:22.759462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'141","reported_seq":163,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.058501+0000","last_change":"2024-04-25T19:16:17.555086+0000","last_active":"2024-04-25T19:19:18.058501+0000","last_peered":"2024-04-25T19:19:18.058501+0000","last_clean":"2024-04-25T19:19:18.058501+0000","last_became_active":"2024-04-25T19:16:17.554771+0000","last_became_peered":"2024-04-25T19:16:17.554771+0000","last_unstale":"2024-04-25T19:19:18.058501+0000","last_undegraded":"2024-04-25T19:19:18.058501+0000","last_fullsized":"2024-04-25T19:19:18.058501+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:22:35.638155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'183","reported_seq":200,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.036612+0000","last_change":"2024-04-25T19:15:19.417596+0000","last_active":"2024-04-25T19:19:18.036612+0000","last_peered":"2024-04-25T19:19:18.036612+0000","last_clean":"2024-04-25T19:19:18.036612+0000","last_became_active":"2024-04-25T19:15:19.417230+0000","last_became_peered":"2024-04-25T19:15:19.417230+0000","last_unstale":"2024-04-25T19:19:18.036612+0000","last_undegraded":"2024-04-25T19:19:18.036612+0000","last_fullsized":"2024-04-25T19:19:18.036612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:28:12.387651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'135","reported_seq":152,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.052108+0000","last_change":"2024-04-25T19:15:19.420564+0000","last_active":"2024-04-25T19:19:18.052108+0000","last_peered":"2024-04-25T19:19:18.052108+0000","last_clean":"2024-04-25T19:19:18.052108+0000","last_became_active":"2024-04-25T19:15:19.420044+0000","last_became_peered":"2024-04-25T19:15:19.420044+0000","last_unstale":"2024-04-25T19:19:18.052108+0000","last_undegraded":"2024-04-25T19:19:18.052108+0000","last_fullsized":"2024-04-25T19:19:18.052108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:04.066030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'141","reported_seq":158,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.030803+0000","last_change":"2024-04-25T19:15:19.415500+0000","last_active":"2024-04-25T19:19:18.030803+0000","last_peered":"2024-04-25T19:19:18.030803+0000","last_clean":"2024-04-25T19:19:18.030803+0000","last_became_active":"2024-04-25T19:15:19.414007+0000","last_became_peered":"2024-04-25T19:15:19.414007+0000","last_unstale":"2024-04-25T19:19:18.030803+0000","last_undegraded":"2024-04-25T19:19:18.030803+0000","last_fullsized":"2024-04-25T19:19:18.030803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:56:12.320555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'141","reported_seq":158,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.052414+0000","last_change":"2024-04-25T19:15:19.423884+0000","last_active":"2024-04-25T19:19:18.052414+0000","last_peered":"2024-04-25T19:19:18.052414+0000","last_clean":"2024-04-25T19:19:18.052414+0000","last_became_active":"2024-04-25T19:15:19.422575+0000","last_became_peered":"2024-04-25T19:15:19.422575+0000","last_unstale":"2024-04-25T19:19:18.052414+0000","last_undegraded":"2024-04-25T19:19:18.052414+0000","last_fullsized":"2024-04-25T19:19:18.052414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:30.988950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'144","reported_seq":161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.064766+0000","last_change":"2024-04-25T19:15:19.424267+0000","last_active":"2024-04-25T19:19:18.064766+0000","last_peered":"2024-04-25T19:19:18.064766+0000","last_clean":"2024-04-25T19:19:18.064766+0000","last_became_active":"2024-04-25T19:15:19.422953+0000","last_became_peered":"2024-04-25T19:15:19.422953+0000","last_unstale":"2024-04-25T19:19:18.064766+0000","last_undegraded":"2024-04-25T19:19:18.064766+0000","last_fullsized":"2024-04-25T19:19:18.064766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:42.823233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'138","reported_seq":149,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.018289+0000","last_change":"2024-04-25T19:16:17.801933+0000","last_active":"2024-04-25T19:19:18.018289+0000","last_peered":"2024-04-25T19:19:18.018289+0000","last_clean":"2024-04-25T19:19:18.018289+0000","last_became_active":"2024-04-25T19:16:17.801500+0000","last_became_peered":"2024-04-25T19:16:17.801500+0000","last_unstale":"2024-04-25T19:19:18.018289+0000","last_undegraded":"2024-04-25T19:19:18.018289+0000","last_fullsized":"2024-04-25T19:19:18.018289+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:41.633057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"23'138","reported_seq":155,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.038730+0000","last_change":"2024-04-25T19:15:19.434476+0000","last_active":"2024-04-25T19:19:18.038730+0000","last_peered":"2024-04-25T19:19:18.038730+0000","last_clean":"2024-04-25T19:19:18.038730+0000","last_became_active":"2024-04-25T19:15:19.433836+0000","last_became_peered":"2024-04-25T19:15:19.433836+0000","last_unstale":"2024-04-25T19:19:18.038730+0000","last_undegraded":"2024-04-25T19:19:18.038730+0000","last_fullsized":"2024-04-25T19:19:18.038730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:13.234829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"23'147","reported_seq":164,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.181499+0000","last_change":"2024-04-25T19:15:19.415521+0000","last_active":"2024-04-25T19:19:18.181499+0000","last_peered":"2024-04-25T19:19:18.181499+0000","last_clean":"2024-04-25T19:19:18.181499+0000","last_became_active":"2024-04-25T19:15:19.415177+0000","last_became_peered":"2024-04-25T19:15:19.415177+0000","last_unstale":"2024-04-25T19:19:18.181499+0000","last_undegraded":"2024-04-25T19:19:18.181499+0000","last_fullsized":"2024-04-25T19:19:18.181499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:24.355619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'165","reported_seq":182,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.085552+0000","last_change":"2024-04-25T19:15:19.430291+0000","last_active":"2024-04-25T19:19:18.085552+0000","last_peered":"2024-04-25T19:19:18.085552+0000","last_clean":"2024-04-25T19:19:18.085552+0000","last_became_active":"2024-04-25T19:15:19.427226+0000","last_became_peered":"2024-04-25T19:15:19.427226+0000","last_unstale":"2024-04-25T19:19:18.085552+0000","last_undegraded":"2024-04-25T19:19:18.085552+0000","last_fullsized":"2024-04-25T19:19:18.085552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:38.464211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"23'153","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.064081+0000","last_change":"2024-04-25T19:15:19.421696+0000","last_active":"2024-04-25T19:19:18.064081+0000","last_peered":"2024-04-25T19:19:18.064081+0000","last_clean":"2024-04-25T19:19:18.064081+0000","last_became_active":"2024-04-25T19:15:19.421300+0000","last_became_peered":"2024-04-25T19:15:19.421300+0000","last_unstale":"2024-04-25T19:19:18.064081+0000","last_undegraded":"2024-04-25T19:19:18.064081+0000","last_fullsized":"2024-04-25T19:19:18.064081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:15.226398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'126","reported_seq":143,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.090958+0000","last_change":"2024-04-25T19:15:19.427880+0000","last_active":"2024-04-25T19:19:18.090958+0000","last_peered":"2024-04-25T19:19:18.090958+0000","last_clean":"2024-04-25T19:19:18.090958+0000","last_became_active":"2024-04-25T19:15:19.427579+0000","last_became_peered":"2024-04-25T19:15:19.427579+0000","last_unstale":"2024-04-25T19:19:18.090958+0000","last_undegraded":"2024-04-25T19:19:18.090958+0000","last_fullsized":"2024-04-25T19:19:18.090958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:47:10.657715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'156","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.083315+0000","last_change":"2024-04-25T19:15:19.430409+0000","last_active":"2024-04-25T19:19:18.083315+0000","last_peered":"2024-04-25T19:19:18.083315+0000","last_clean":"2024-04-25T19:19:18.083315+0000","last_became_active":"2024-04-25T19:15:19.428305+0000","last_became_peered":"2024-04-25T19:15:19.428305+0000","last_unstale":"2024-04-25T19:19:18.083315+0000","last_undegraded":"2024-04-25T19:19:18.083315+0000","last_fullsized":"2024-04-25T19:19:18.083315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:10:01.268963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"23'126","reported_seq":143,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.827387+0000","last_change":"2024-04-25T19:15:19.424435+0000","last_active":"2024-04-25T19:19:17.827387+0000","last_peered":"2024-04-25T19:19:17.827387+0000","last_clean":"2024-04-25T19:19:17.827387+0000","last_became_active":"2024-04-25T19:15:19.422763+0000","last_became_peered":"2024-04-25T19:15:19.422763+0000","last_unstale":"2024-04-25T19:19:17.827387+0000","last_undegraded":"2024-04-25T19:19:17.827387+0000","last_fullsized":"2024-04-25T19:19:17.827387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:58.092412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'174","reported_seq":191,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.937851+0000","last_change":"2024-04-25T19:15:19.420275+0000","last_active":"2024-04-25T19:19:17.937851+0000","last_peered":"2024-04-25T19:19:17.937851+0000","last_clean":"2024-04-25T19:19:17.937851+0000","last_became_active":"2024-04-25T19:15:19.419936+0000","last_became_peered":"2024-04-25T19:15:19.419936+0000","last_unstale":"2024-04-25T19:19:17.937851+0000","last_undegraded":"2024-04-25T19:19:17.937851+0000","last_fullsized":"2024-04-25T19:19:17.937851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:55.710620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'132","reported_seq":149,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.061024+0000","last_change":"2024-04-25T19:15:19.423408+0000","last_active":"2024-04-25T19:19:18.061024+0000","last_peered":"2024-04-25T19:19:18.061024+0000","last_clean":"2024-04-25T19:19:18.061024+0000","last_became_active":"2024-04-25T19:15:19.423116+0000","last_became_peered":"2024-04-25T19:15:19.423116+0000","last_unstale":"2024-04-25T19:19:18.061024+0000","last_undegraded":"2024-04-25T19:19:18.061024+0000","last_fullsized":"2024-04-25T19:19:18.061024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:18:57.787013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'183","reported_seq":200,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.071336+0000","last_change":"2024-04-25T19:15:19.428826+0000","last_active":"2024-04-25T19:19:18.071336+0000","last_peered":"2024-04-25T19:19:18.071336+0000","last_clean":"2024-04-25T19:19:18.071336+0000","last_became_active":"2024-04-25T19:15:19.428474+0000","last_became_peered":"2024-04-25T19:15:19.428474+0000","last_unstale":"2024-04-25T19:19:18.071336+0000","last_undegraded":"2024-04-25T19:19:18.071336+0000","last_fullsized":"2024-04-25T19:19:18.071336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:19.974851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'138","reported_seq":155,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.059590+0000","last_change":"2024-04-25T19:15:19.423353+0000","last_active":"2024-04-25T19:19:18.059590+0000","last_peered":"2024-04-25T19:19:18.059590+0000","last_clean":"2024-04-25T19:19:18.059590+0000","last_became_active":"2024-04-25T19:15:19.422795+0000","last_became_peered":"2024-04-25T19:15:19.422795+0000","last_unstale":"2024-04-25T19:19:18.059590+0000","last_undegraded":"2024-04-25T19:19:18.059590+0000","last_fullsized":"2024-04-25T19:19:18.059590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:34.119976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'129","reported_seq":146,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.066581+0000","last_change":"2024-04-25T19:15:19.421809+0000","last_active":"2024-04-25T19:19:18.066581+0000","last_peered":"2024-04-25T19:19:18.066581+0000","last_clean":"2024-04-25T19:19:18.066581+0000","last_became_active":"2024-04-25T19:15:19.421515+0000","last_became_peered":"2024-04-25T19:15:19.421515+0000","last_unstale":"2024-04-25T19:19:18.066581+0000","last_undegraded":"2024-04-25T19:19:18.066581+0000","last_fullsized":"2024-04-25T19:19:18.066581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:22.603291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'111","reported_seq":128,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.917934+0000","last_change":"2024-04-25T19:15:19.410183+0000","last_active":"2024-04-25T19:19:17.917934+0000","last_peered":"2024-04-25T19:19:17.917934+0000","last_clean":"2024-04-25T19:19:17.917934+0000","last_became_active":"2024-04-25T19:15:19.409857+0000","last_became_peered":"2024-04-25T19:15:19.409857+0000","last_unstale":"2024-04-25T19:19:17.917934+0000","last_undegraded":"2024-04-25T19:19:17.917934+0000","last_fullsized":"2024-04-25T19:19:17.917934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:30.180786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'123","reported_seq":140,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.994763+0000","last_change":"2024-04-25T19:15:19.425524+0000","last_active":"2024-04-25T19:19:17.994763+0000","last_peered":"2024-04-25T19:19:17.994763+0000","last_clean":"2024-04-25T19:19:17.994763+0000","last_became_active":"2024-04-25T19:15:19.424578+0000","last_became_peered":"2024-04-25T19:15:19.424578+0000","last_unstale":"2024-04-25T19:19:17.994763+0000","last_undegraded":"2024-04-25T19:19:17.994763+0000","last_fullsized":"2024-04-25T19:19:17.994763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:02.103879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"23'123","reported_seq":140,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.093496+0000","last_change":"2024-04-25T19:15:19.426720+0000","last_active":"2024-04-25T19:19:18.093496+0000","last_peered":"2024-04-25T19:19:18.093496+0000","last_clean":"2024-04-25T19:19:18.093496+0000","last_became_active":"2024-04-25T19:15:19.426139+0000","last_became_peered":"2024-04-25T19:15:19.426139+0000","last_unstale":"2024-04-25T19:19:18.093496+0000","last_undegraded":"2024-04-25T19:19:18.093496+0000","last_fullsized":"2024-04-25T19:19:18.093496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:31.608993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'141","reported_seq":158,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.017033+0000","last_change":"2024-04-25T19:15:19.413805+0000","last_active":"2024-04-25T19:19:18.017033+0000","last_peered":"2024-04-25T19:19:18.017033+0000","last_clean":"2024-04-25T19:19:18.017033+0000","last_became_active":"2024-04-25T19:15:19.413455+0000","last_became_peered":"2024-04-25T19:15:19.413455+0000","last_unstale":"2024-04-25T19:19:18.017033+0000","last_undegraded":"2024-04-25T19:19:18.017033+0000","last_fullsized":"2024-04-25T19:19:18.017033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:05:45.008076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'123","reported_seq":140,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.921522+0000","last_change":"2024-04-25T19:15:19.423770+0000","last_active":"2024-04-25T19:19:17.921522+0000","last_peered":"2024-04-25T19:19:17.921522+0000","last_clean":"2024-04-25T19:19:17.921522+0000","last_became_active":"2024-04-25T19:15:19.423406+0000","last_became_peered":"2024-04-25T19:15:19.423406+0000","last_unstale":"2024-04-25T19:19:17.921522+0000","last_undegraded":"2024-04-25T19:19:17.921522+0000","last_fullsized":"2024-04-25T19:19:17.921522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:41:23.669832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'6","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.660321+0000","last_change":"2024-04-25T19:15:18.422035+0000","last_active":"2024-04-25T19:16:17.660321+0000","last_peered":"2024-04-25T19:16:17.660321+0000","last_clean":"2024-04-25T19:16:17.660321+0000","last_became_active":"2024-04-25T19:15:18.421658+0000","last_became_peered":"2024-04-25T19:15:18.421658+0000","last_unstale":"2024-04-25T19:16:17.660321+0000","last_undegraded":"2024-04-25T19:16:17.660321+0000","last_fullsized":"2024-04-25T19:16:17.660321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:23.349126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1063,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'129","reported_seq":146,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.994924+0000","last_change":"2024-04-25T19:15:19.421805+0000","last_active":"2024-04-25T19:19:17.994924+0000","last_peered":"2024-04-25T19:19:17.994924+0000","last_clean":"2024-04-25T19:19:17.994924+0000","last_became_active":"2024-04-25T19:15:19.421497+0000","last_became_peered":"2024-04-25T19:15:19.421497+0000","last_unstale":"2024-04-25T19:19:17.994924+0000","last_undegraded":"2024-04-25T19:19:17.994924+0000","last_fullsized":"2024-04-25T19:19:17.994924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:29:29.949508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'7","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.553158+0000","last_change":"2024-04-25T19:15:18.416894+0000","last_active":"2024-04-25T19:16:17.553158+0000","last_peered":"2024-04-25T19:16:17.553158+0000","last_clean":"2024-04-25T19:16:17.553158+0000","last_became_active":"2024-04-25T19:15:18.416539+0000","last_became_peered":"2024-04-25T19:15:18.416539+0000","last_unstale":"2024-04-25T19:16:17.553158+0000","last_undegraded":"2024-04-25T19:16:17.553158+0000","last_fullsized":"2024-04-25T19:16:17.553158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:30.388606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"21'3","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.659993+0000","last_change":"2024-04-25T19:15:18.420129+0000","last_active":"2024-04-25T19:16:17.659993+0000","last_peered":"2024-04-25T19:16:17.659993+0000","last_clean":"2024-04-25T19:16:17.659993+0000","last_became_active":"2024-04-25T19:15:18.419732+0000","last_became_peered":"2024-04-25T19:15:18.419732+0000","last_unstale":"2024-04-25T19:16:17.659993+0000","last_undegraded":"2024-04-25T19:16:17.659993+0000","last_fullsized":"2024-04-25T19:16:17.659993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:41:48.673989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'162","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.034836+0000","last_change":"2024-04-25T19:16:17.801934+0000","last_active":"2024-04-25T19:19:18.034836+0000","last_peered":"2024-04-25T19:19:18.034836+0000","last_clean":"2024-04-25T19:19:18.034836+0000","last_became_active":"2024-04-25T19:16:17.801451+0000","last_became_peered":"2024-04-25T19:16:17.801451+0000","last_unstale":"2024-04-25T19:19:18.034836+0000","last_undegraded":"2024-04-25T19:19:18.034836+0000","last_fullsized":"2024-04-25T19:19:18.034836+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:41:52.528120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"21'3","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.550543+0000","last_change":"2024-04-25T19:15:18.420711+0000","last_active":"2024-04-25T19:16:17.550543+0000","last_peered":"2024-04-25T19:16:17.550543+0000","last_clean":"2024-04-25T19:16:17.550543+0000","last_became_active":"2024-04-25T19:15:18.420362+0000","last_became_peered":"2024-04-25T19:15:18.420362+0000","last_unstale":"2024-04-25T19:16:17.550543+0000","last_undegraded":"2024-04-25T19:16:17.550543+0000","last_fullsized":"2024-04-25T19:16:17.550543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:48.245193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'147","reported_seq":164,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.906678+0000","last_change":"2024-04-25T19:15:19.424356+0000","last_active":"2024-04-25T19:19:17.906678+0000","last_peered":"2024-04-25T19:19:17.906678+0000","last_clean":"2024-04-25T19:19:17.906678+0000","last_became_active":"2024-04-25T19:15:19.423556+0000","last_became_peered":"2024-04-25T19:15:19.423556+0000","last_unstale":"2024-04-25T19:19:17.906678+0000","last_undegraded":"2024-04-25T19:19:17.906678+0000","last_fullsized":"2024-04-25T19:19:17.906678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:19:35.427261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'7","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:42.864151+0000","last_change":"2024-04-25T19:15:18.419250+0000","last_active":"2024-04-25T19:17:42.864151+0000","last_peered":"2024-04-25T19:17:42.864151+0000","last_clean":"2024-04-25T19:17:42.864151+0000","last_became_active":"2024-04-25T19:15:18.418927+0000","last_became_peered":"2024-04-25T19:15:18.418927+0000","last_unstale":"2024-04-25T19:17:42.864151+0000","last_undegraded":"2024-04-25T19:17:42.864151+0000","last_fullsized":"2024-04-25T19:17:42.864151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:26.012965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'219","reported_seq":236,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.335276+0000","last_change":"2024-04-25T19:15:19.430117+0000","last_active":"2024-04-25T19:19:18.335276+0000","last_peered":"2024-04-25T19:19:18.335276+0000","last_clean":"2024-04-25T19:19:18.335276+0000","last_became_active":"2024-04-25T19:15:19.428985+0000","last_became_peered":"2024-04-25T19:15:19.428985+0000","last_unstale":"2024-04-25T19:19:18.335276+0000","last_undegraded":"2024-04-25T19:19:18.335276+0000","last_fullsized":"2024-04-25T19:19:18.335276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:36:29.069541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"21'4","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.550765+0000","last_change":"2024-04-25T19:15:18.422931+0000","last_active":"2024-04-25T19:16:17.550765+0000","last_peered":"2024-04-25T19:16:17.550765+0000","last_clean":"2024-04-25T19:16:17.550765+0000","last_became_active":"2024-04-25T19:15:18.422610+0000","last_became_peered":"2024-04-25T19:15:18.422610+0000","last_unstale":"2024-04-25T19:16:17.550765+0000","last_undegraded":"2024-04-25T19:16:17.550765+0000","last_fullsized":"2024-04-25T19:16:17.550765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:18.489484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'132","reported_seq":149,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.080062+0000","last_change":"2024-04-25T19:15:19.416109+0000","last_active":"2024-04-25T19:19:18.080062+0000","last_peered":"2024-04-25T19:19:18.080062+0000","last_clean":"2024-04-25T19:19:18.080062+0000","last_became_active":"2024-04-25T19:15:19.415734+0000","last_became_peered":"2024-04-25T19:15:19.415734+0000","last_unstale":"2024-04-25T19:19:18.080062+0000","last_undegraded":"2024-04-25T19:19:18.080062+0000","last_fullsized":"2024-04-25T19:19:18.080062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:57:31.406303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'10","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:16.549699+0000","last_change":"2024-04-25T19:15:18.428677+0000","last_active":"2024-04-25T19:19:16.549699+0000","last_peered":"2024-04-25T19:19:16.549699+0000","last_clean":"2024-04-25T19:19:16.549699+0000","last_became_active":"2024-04-25T19:15:18.428432+0000","last_became_peered":"2024-04-25T19:15:18.428432+0000","last_unstale":"2024-04-25T19:19:16.549699+0000","last_undegraded":"2024-04-25T19:19:16.549699+0000","last_fullsized":"2024-04-25T19:19:16.549699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:38.478595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'138","reported_seq":155,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.036477+0000","last_change":"2024-04-25T19:15:19.428245+0000","last_active":"2024-04-25T19:19:18.036477+0000","last_peered":"2024-04-25T19:19:18.036477+0000","last_clean":"2024-04-25T19:19:18.036477+0000","last_became_active":"2024-04-25T19:15:19.427941+0000","last_became_peered":"2024-04-25T19:15:19.427941+0000","last_unstale":"2024-04-25T19:19:18.036477+0000","last_undegraded":"2024-04-25T19:19:18.036477+0000","last_fullsized":"2024-04-25T19:19:18.036477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:09.947923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'99","reported_seq":118,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:19.439568+0000","last_change":"2024-04-25T19:15:18.420881+0000","last_active":"2024-04-25T19:19:19.439568+0000","last_peered":"2024-04-25T19:19:19.439568+0000","last_clean":"2024-04-25T19:19:19.439568+0000","last_became_active":"2024-04-25T19:15:18.420303+0000","last_became_peered":"2024-04-25T19:15:18.420303+0000","last_unstale":"2024-04-25T19:19:19.439568+0000","last_undegraded":"2024-04-25T19:19:19.439568+0000","last_fullsized":"2024-04-25T19:19:19.439568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:39:24.356298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":216545,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'144","reported_seq":155,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.986213+0000","last_change":"2024-04-25T19:16:17.661342+0000","last_active":"2024-04-25T19:19:17.986213+0000","last_peered":"2024-04-25T19:19:17.986213+0000","last_clean":"2024-04-25T19:19:17.986213+0000","last_became_active":"2024-04-25T19:16:17.661023+0000","last_became_peered":"2024-04-25T19:16:17.661023+0000","last_unstale":"2024-04-25T19:19:17.986213+0000","last_undegraded":"2024-04-25T19:19:17.986213+0000","last_fullsized":"2024-04-25T19:19:17.986213+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:31:11.190902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.13","version":"21'9","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.800767+0000","last_change":"2024-04-25T19:15:18.421008+0000","last_active":"2024-04-25T19:16:17.800767+0000","last_peered":"2024-04-25T19:16:17.800767+0000","last_clean":"2024-04-25T19:16:17.800767+0000","last_became_active":"2024-04-25T19:15:18.420402+0000","last_became_peered":"2024-04-25T19:15:18.420402+0000","last_unstale":"2024-04-25T19:16:17.800767+0000","last_undegraded":"2024-04-25T19:16:17.800767+0000","last_fullsized":"2024-04-25T19:16:17.800767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:05:28.723958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'148","reported_seq":165,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.054624+0000","last_change":"2024-04-25T19:15:19.424805+0000","last_active":"2024-04-25T19:19:18.054624+0000","last_peered":"2024-04-25T19:19:18.054624+0000","last_clean":"2024-04-25T19:19:18.054624+0000","last_became_active":"2024-04-25T19:15:19.424512+0000","last_became_peered":"2024-04-25T19:15:19.424512+0000","last_unstale":"2024-04-25T19:19:18.054624+0000","last_undegraded":"2024-04-25T19:19:18.054624+0000","last_fullsized":"2024-04-25T19:19:18.054624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:50.423186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.550627+0000","last_change":"2024-04-25T19:15:18.426805+0000","last_active":"2024-04-25T19:16:17.550627+0000","last_peered":"2024-04-25T19:16:17.550627+0000","last_clean":"2024-04-25T19:16:17.550627+0000","last_became_active":"2024-04-25T19:15:18.426472+0000","last_became_peered":"2024-04-25T19:15:18.426472+0000","last_unstale":"2024-04-25T19:16:17.550627+0000","last_undegraded":"2024-04-25T19:16:17.550627+0000","last_fullsized":"2024-04-25T19:16:17.550627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:29.373363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'156","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.047258+0000","last_change":"2024-04-25T19:15:19.428780+0000","last_active":"2024-04-25T19:19:18.047258+0000","last_peered":"2024-04-25T19:19:18.047258+0000","last_clean":"2024-04-25T19:19:18.047258+0000","last_became_active":"2024-04-25T19:15:19.428450+0000","last_became_peered":"2024-04-25T19:15:19.428450+0000","last_unstale":"2024-04-25T19:19:18.047258+0000","last_undegraded":"2024-04-25T19:19:18.047258+0000","last_fullsized":"2024-04-25T19:19:18.047258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:01.203880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"21'6","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.552957+0000","last_change":"2024-04-25T19:15:18.417740+0000","last_active":"2024-04-25T19:16:17.552957+0000","last_peered":"2024-04-25T19:16:17.552957+0000","last_clean":"2024-04-25T19:16:17.552957+0000","last_became_active":"2024-04-25T19:15:18.417404+0000","last_became_peered":"2024-04-25T19:15:18.417404+0000","last_unstale":"2024-04-25T19:16:17.552957+0000","last_undegraded":"2024-04-25T19:16:17.552957+0000","last_fullsized":"2024-04-25T19:16:17.552957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:21.737283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1029,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'159","reported_seq":176,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.060341+0000","last_change":"2024-04-25T19:15:19.411214+0000","last_active":"2024-04-25T19:19:18.060341+0000","last_peered":"2024-04-25T19:19:18.060341+0000","last_clean":"2024-04-25T19:19:18.060341+0000","last_became_active":"2024-04-25T19:15:19.410814+0000","last_became_peered":"2024-04-25T19:15:19.410814+0000","last_unstale":"2024-04-25T19:19:18.060341+0000","last_undegraded":"2024-04-25T19:19:18.060341+0000","last_fullsized":"2024-04-25T19:19:18.060341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:42.430277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'18","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:37.867718+0000","last_change":"2024-04-25T19:15:18.422226+0000","last_active":"2024-04-25T19:17:37.867718+0000","last_peered":"2024-04-25T19:17:37.867718+0000","last_clean":"2024-04-25T19:17:37.867718+0000","last_became_active":"2024-04-25T19:15:18.421900+0000","last_became_peered":"2024-04-25T19:15:18.421900+0000","last_unstale":"2024-04-25T19:17:37.867718+0000","last_undegraded":"2024-04-25T19:17:37.867718+0000","last_fullsized":"2024-04-25T19:17:37.867718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:18.040114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'183","reported_seq":200,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.048026+0000","last_change":"2024-04-25T19:15:19.428885+0000","last_active":"2024-04-25T19:19:18.048026+0000","last_peered":"2024-04-25T19:19:18.048026+0000","last_clean":"2024-04-25T19:19:18.048026+0000","last_became_active":"2024-04-25T19:15:19.428563+0000","last_became_peered":"2024-04-25T19:15:19.428563+0000","last_unstale":"2024-04-25T19:19:18.048026+0000","last_undegraded":"2024-04-25T19:19:18.048026+0000","last_fullsized":"2024-04-25T19:19:18.048026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:43.469189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'2","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.800374+0000","last_change":"2024-04-25T19:15:18.422125+0000","last_active":"2024-04-25T19:16:17.800374+0000","last_peered":"2024-04-25T19:16:17.800374+0000","last_clean":"2024-04-25T19:16:17.800374+0000","last_became_active":"2024-04-25T19:15:18.421777+0000","last_became_peered":"2024-04-25T19:15:18.421777+0000","last_unstale":"2024-04-25T19:16:17.800374+0000","last_undegraded":"2024-04-25T19:16:17.800374+0000","last_fullsized":"2024-04-25T19:16:17.800374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:24.289010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'141","reported_seq":158,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.052159+0000","last_change":"2024-04-25T19:15:19.429675+0000","last_active":"2024-04-25T19:19:18.052159+0000","last_peered":"2024-04-25T19:19:18.052159+0000","last_clean":"2024-04-25T19:19:18.052159+0000","last_became_active":"2024-04-25T19:15:19.429316+0000","last_became_peered":"2024-04-25T19:15:19.429316+0000","last_unstale":"2024-04-25T19:19:18.052159+0000","last_undegraded":"2024-04-25T19:19:18.052159+0000","last_fullsized":"2024-04-25T19:19:18.052159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:09:55.521944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.552135+0000","last_change":"2024-04-25T19:15:18.418661+0000","last_active":"2024-04-25T19:16:17.552135+0000","last_peered":"2024-04-25T19:16:17.552135+0000","last_clean":"2024-04-25T19:16:17.552135+0000","last_became_active":"2024-04-25T19:15:18.418359+0000","last_became_peered":"2024-04-25T19:15:18.418359+0000","last_unstale":"2024-04-25T19:16:17.552135+0000","last_undegraded":"2024-04-25T19:16:17.552135+0000","last_fullsized":"2024-04-25T19:16:17.552135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:49:22.023652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'129","reported_seq":146,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.027304+0000","last_change":"2024-04-25T19:15:19.434853+0000","last_active":"2024-04-25T19:19:18.027304+0000","last_peered":"2024-04-25T19:19:18.027304+0000","last_clean":"2024-04-25T19:19:18.027304+0000","last_became_active":"2024-04-25T19:15:19.434508+0000","last_became_peered":"2024-04-25T19:15:19.434508+0000","last_unstale":"2024-04-25T19:19:18.027304+0000","last_undegraded":"2024-04-25T19:19:18.027304+0000","last_fullsized":"2024-04-25T19:19:18.027304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:52.965864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'3","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.552800+0000","last_change":"2024-04-25T19:15:18.428626+0000","last_active":"2024-04-25T19:16:17.552800+0000","last_peered":"2024-04-25T19:16:17.552800+0000","last_clean":"2024-04-25T19:16:17.552800+0000","last_became_active":"2024-04-25T19:15:18.428332+0000","last_became_peered":"2024-04-25T19:15:18.428332+0000","last_unstale":"2024-04-25T19:16:17.552800+0000","last_undegraded":"2024-04-25T19:16:17.552800+0000","last_fullsized":"2024-04-25T19:16:17.552800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:15.595305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'138","reported_seq":155,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.016419+0000","last_change":"2024-04-25T19:15:19.428754+0000","last_active":"2024-04-25T19:19:18.016419+0000","last_peered":"2024-04-25T19:19:18.016419+0000","last_clean":"2024-04-25T19:19:18.016419+0000","last_became_active":"2024-04-25T19:15:19.427992+0000","last_became_peered":"2024-04-25T19:15:19.427992+0000","last_unstale":"2024-04-25T19:19:18.016419+0000","last_undegraded":"2024-04-25T19:19:18.016419+0000","last_fullsized":"2024-04-25T19:19:18.016419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:43:48.443923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"21'6","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.551964+0000","last_change":"2024-04-25T19:15:18.419645+0000","last_active":"2024-04-25T19:16:17.551964+0000","last_peered":"2024-04-25T19:16:17.551964+0000","last_clean":"2024-04-25T19:16:17.551964+0000","last_became_active":"2024-04-25T19:15:18.419286+0000","last_became_peered":"2024-04-25T19:15:18.419286+0000","last_unstale":"2024-04-25T19:16:17.551964+0000","last_undegraded":"2024-04-25T19:16:17.551964+0000","last_fullsized":"2024-04-25T19:16:17.551964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:08.295208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1611,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'144","reported_seq":161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.964325+0000","last_change":"2024-04-25T19:15:19.432395+0000","last_active":"2024-04-25T19:19:17.964325+0000","last_peered":"2024-04-25T19:19:17.964325+0000","last_clean":"2024-04-25T19:19:17.964325+0000","last_became_active":"2024-04-25T19:15:19.431739+0000","last_became_peered":"2024-04-25T19:15:19.431739+0000","last_unstale":"2024-04-25T19:19:17.964325+0000","last_undegraded":"2024-04-25T19:19:17.964325+0000","last_fullsized":"2024-04-25T19:19:17.964325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:50:03.964030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":116,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:17.871195+0000","last_change":"2024-04-25T19:15:15.604825+0000","last_active":"2024-04-25T19:17:17.871195+0000","last_peered":"2024-04-25T19:17:17.871195+0000","last_clean":"2024-04-25T19:17:17.871195+0000","last_became_active":"2024-04-25T19:15:15.603730+0000","last_became_peered":"2024-04-25T19:15:15.603730+0000","last_unstale":"2024-04-25T19:17:17.871195+0000","last_undegraded":"2024-04-25T19:17:17.871195+0000","last_fullsized":"2024-04-25T19:17:17.871195+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:14.367890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:14.367890+0000","last_clean_scrub_stamp":"2024-04-25T19:15:14.367890+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:02.411899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'6","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:22.863164+0000","last_change":"2024-04-25T19:15:18.426331+0000","last_active":"2024-04-25T19:17:22.863164+0000","last_peered":"2024-04-25T19:17:22.863164+0000","last_clean":"2024-04-25T19:17:22.863164+0000","last_became_active":"2024-04-25T19:15:18.425993+0000","last_became_peered":"2024-04-25T19:15:18.425993+0000","last_unstale":"2024-04-25T19:17:22.863164+0000","last_undegraded":"2024-04-25T19:17:22.863164+0000","last_fullsized":"2024-04-25T19:17:22.863164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:17.440484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'150","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.078219+0000","last_change":"2024-04-25T19:15:19.417337+0000","last_active":"2024-04-25T19:19:18.078219+0000","last_peered":"2024-04-25T19:19:18.078219+0000","last_clean":"2024-04-25T19:19:18.078219+0000","last_became_active":"2024-04-25T19:15:19.416808+0000","last_became_peered":"2024-04-25T19:15:19.416808+0000","last_unstale":"2024-04-25T19:19:18.078219+0000","last_undegraded":"2024-04-25T19:19:18.078219+0000","last_fullsized":"2024-04-25T19:19:18.078219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:57.797904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"21'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:17.871167+0000","last_change":"2024-04-25T19:15:18.426358+0000","last_active":"2024-04-25T19:17:17.871167+0000","last_peered":"2024-04-25T19:17:17.871167+0000","last_clean":"2024-04-25T19:17:17.871167+0000","last_became_active":"2024-04-25T19:15:18.426037+0000","last_became_peered":"2024-04-25T19:15:18.426037+0000","last_unstale":"2024-04-25T19:17:17.871167+0000","last_undegraded":"2024-04-25T19:17:17.871167+0000","last_fullsized":"2024-04-25T19:17:17.871167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:26:28.945667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'126","reported_seq":143,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.082392+0000","last_change":"2024-04-25T19:15:19.426219+0000","last_active":"2024-04-25T19:19:18.082392+0000","last_peered":"2024-04-25T19:19:18.082392+0000","last_clean":"2024-04-25T19:19:18.082392+0000","last_became_active":"2024-04-25T19:15:19.425626+0000","last_became_peered":"2024-04-25T19:15:19.425626+0000","last_unstale":"2024-04-25T19:19:18.082392+0000","last_undegraded":"2024-04-25T19:19:18.082392+0000","last_fullsized":"2024-04-25T19:19:18.082392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:58.372454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'11","reported_seq":35,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:16.540140+0000","last_change":"2024-04-25T19:15:18.424560+0000","last_active":"2024-04-25T19:19:16.540140+0000","last_peered":"2024-04-25T19:19:16.540140+0000","last_clean":"2024-04-25T19:19:16.540140+0000","last_became_active":"2024-04-25T19:15:18.424179+0000","last_became_peered":"2024-04-25T19:15:18.424179+0000","last_unstale":"2024-04-25T19:19:16.540140+0000","last_undegraded":"2024-04-25T19:19:16.540140+0000","last_fullsized":"2024-04-25T19:19:16.540140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:06.184399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'123","reported_seq":141,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.076512+0000","last_change":"2024-04-25T19:15:19.415219+0000","last_active":"2024-04-25T19:19:18.076512+0000","last_peered":"2024-04-25T19:19:18.076512+0000","last_clean":"2024-04-25T19:19:18.076512+0000","last_became_active":"2024-04-25T19:15:19.414877+0000","last_became_peered":"2024-04-25T19:15:19.414877+0000","last_unstale":"2024-04-25T19:19:18.076512+0000","last_undegraded":"2024-04-25T19:19:18.076512+0000","last_fullsized":"2024-04-25T19:19:18.076512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:15.063419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'5","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:22.863125+0000","last_change":"2024-04-25T19:15:18.423402+0000","last_active":"2024-04-25T19:17:22.863125+0000","last_peered":"2024-04-25T19:17:22.863125+0000","last_clean":"2024-04-25T19:17:22.863125+0000","last_became_active":"2024-04-25T19:15:18.422330+0000","last_became_peered":"2024-04-25T19:15:18.422330+0000","last_unstale":"2024-04-25T19:17:22.863125+0000","last_undegraded":"2024-04-25T19:17:22.863125+0000","last_fullsized":"2024-04-25T19:17:22.863125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:39:17.512502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'138","reported_seq":155,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.037118+0000","last_change":"2024-04-25T19:15:19.421096+0000","last_active":"2024-04-25T19:19:18.037118+0000","last_peered":"2024-04-25T19:19:18.037118+0000","last_clean":"2024-04-25T19:19:18.037118+0000","last_became_active":"2024-04-25T19:15:19.420598+0000","last_became_peered":"2024-04-25T19:15:19.420598+0000","last_unstale":"2024-04-25T19:19:18.037118+0000","last_undegraded":"2024-04-25T19:19:18.037118+0000","last_fullsized":"2024-04-25T19:19:18.037118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:20.733888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:22.862361+0000","last_change":"2024-04-25T19:15:18.422979+0000","last_active":"2024-04-25T19:17:22.862361+0000","last_peered":"2024-04-25T19:17:22.862361+0000","last_clean":"2024-04-25T19:17:22.862361+0000","last_became_active":"2024-04-25T19:15:18.422621+0000","last_became_peered":"2024-04-25T19:15:18.422621+0000","last_unstale":"2024-04-25T19:17:22.862361+0000","last_undegraded":"2024-04-25T19:17:22.862361+0000","last_fullsized":"2024-04-25T19:17:22.862361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:47:56.160615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'138","reported_seq":155,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.064130+0000","last_change":"2024-04-25T19:15:19.426379+0000","last_active":"2024-04-25T19:19:18.064130+0000","last_peered":"2024-04-25T19:19:18.064130+0000","last_clean":"2024-04-25T19:19:18.064130+0000","last_became_active":"2024-04-25T19:15:19.425997+0000","last_became_peered":"2024-04-25T19:15:19.425997+0000","last_unstale":"2024-04-25T19:19:18.064130+0000","last_undegraded":"2024-04-25T19:19:18.064130+0000","last_fullsized":"2024-04-25T19:19:18.064130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:32.972274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"21'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:22.871729+0000","last_change":"2024-04-25T19:15:18.427748+0000","last_active":"2024-04-25T19:17:22.871729+0000","last_peered":"2024-04-25T19:17:22.871729+0000","last_clean":"2024-04-25T19:17:22.871729+0000","last_became_active":"2024-04-25T19:15:18.427374+0000","last_became_peered":"2024-04-25T19:15:18.427374+0000","last_unstale":"2024-04-25T19:17:22.871729+0000","last_undegraded":"2024-04-25T19:17:22.871729+0000","last_fullsized":"2024-04-25T19:17:22.871729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:47:41.469558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'135","reported_seq":152,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.078511+0000","last_change":"2024-04-25T19:15:19.422196+0000","last_active":"2024-04-25T19:19:18.078511+0000","last_peered":"2024-04-25T19:19:18.078511+0000","last_clean":"2024-04-25T19:19:18.078511+0000","last_became_active":"2024-04-25T19:15:19.421835+0000","last_became_peered":"2024-04-25T19:15:19.421835+0000","last_unstale":"2024-04-25T19:19:18.078511+0000","last_undegraded":"2024-04-25T19:19:18.078511+0000","last_fullsized":"2024-04-25T19:19:18.078511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:38.500369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'24","reported_seq":43,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:14.465958+0000","last_change":"2024-04-25T19:15:18.419086+0000","last_active":"2024-04-25T19:19:14.465958+0000","last_peered":"2024-04-25T19:19:14.465958+0000","last_clean":"2024-04-25T19:19:14.465958+0000","last_became_active":"2024-04-25T19:15:18.418775+0000","last_became_peered":"2024-04-25T19:15:18.418775+0000","last_unstale":"2024-04-25T19:19:14.465958+0000","last_undegraded":"2024-04-25T19:19:14.465958+0000","last_fullsized":"2024-04-25T19:19:14.465958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:20.074753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'138","reported_seq":155,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.072440+0000","last_change":"2024-04-25T19:15:19.432155+0000","last_active":"2024-04-25T19:19:18.072440+0000","last_peered":"2024-04-25T19:19:18.072440+0000","last_clean":"2024-04-25T19:19:18.072440+0000","last_became_active":"2024-04-25T19:15:19.431905+0000","last_became_peered":"2024-04-25T19:15:19.431905+0000","last_unstale":"2024-04-25T19:19:18.072440+0000","last_undegraded":"2024-04-25T19:19:18.072440+0000","last_fullsized":"2024-04-25T19:19:18.072440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:52.752469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'8","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:27.862195+0000","last_change":"2024-04-25T19:15:18.423144+0000","last_active":"2024-04-25T19:17:27.862195+0000","last_peered":"2024-04-25T19:17:27.862195+0000","last_clean":"2024-04-25T19:17:27.862195+0000","last_became_active":"2024-04-25T19:15:18.422838+0000","last_became_peered":"2024-04-25T19:15:18.422838+0000","last_unstale":"2024-04-25T19:17:27.862195+0000","last_undegraded":"2024-04-25T19:17:27.862195+0000","last_fullsized":"2024-04-25T19:17:27.862195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:28.026912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'156","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.014340+0000","last_change":"2024-04-25T19:15:19.420582+0000","last_active":"2024-04-25T19:19:18.014340+0000","last_peered":"2024-04-25T19:19:18.014340+0000","last_clean":"2024-04-25T19:19:18.014340+0000","last_became_active":"2024-04-25T19:15:19.420270+0000","last_became_peered":"2024-04-25T19:15:19.420270+0000","last_unstale":"2024-04-25T19:19:18.014340+0000","last_undegraded":"2024-04-25T19:19:18.014340+0000","last_fullsized":"2024-04-25T19:19:18.014340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:06.827819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"21'2","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.549438+0000","last_change":"2024-04-25T19:15:18.425469+0000","last_active":"2024-04-25T19:16:17.549438+0000","last_peered":"2024-04-25T19:16:17.549438+0000","last_clean":"2024-04-25T19:16:17.549438+0000","last_became_active":"2024-04-25T19:15:18.425084+0000","last_became_peered":"2024-04-25T19:15:18.425084+0000","last_unstale":"2024-04-25T19:16:17.549438+0000","last_undegraded":"2024-04-25T19:16:17.549438+0000","last_fullsized":"2024-04-25T19:16:17.549438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:14.363158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'132","reported_seq":149,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.814179+0000","last_change":"2024-04-25T19:15:19.425159+0000","last_active":"2024-04-25T19:19:17.814179+0000","last_peered":"2024-04-25T19:19:17.814179+0000","last_clean":"2024-04-25T19:19:17.814179+0000","last_became_active":"2024-04-25T19:15:19.424847+0000","last_became_peered":"2024-04-25T19:15:19.424847+0000","last_unstale":"2024-04-25T19:19:17.814179+0000","last_undegraded":"2024-04-25T19:19:17.814179+0000","last_fullsized":"2024-04-25T19:19:17.814179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:28.715286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"21'7","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.552420+0000","last_change":"2024-04-25T19:15:18.424645+0000","last_active":"2024-04-25T19:16:17.552420+0000","last_peered":"2024-04-25T19:16:17.552420+0000","last_clean":"2024-04-25T19:16:17.552420+0000","last_became_active":"2024-04-25T19:15:18.424241+0000","last_became_peered":"2024-04-25T19:15:18.424241+0000","last_unstale":"2024-04-25T19:16:17.552420+0000","last_undegraded":"2024-04-25T19:16:17.552420+0000","last_fullsized":"2024-04-25T19:16:17.552420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:29.373363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'144","reported_seq":161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.074330+0000","last_change":"2024-04-25T19:15:19.421095+0000","last_active":"2024-04-25T19:19:18.074330+0000","last_peered":"2024-04-25T19:19:18.074330+0000","last_clean":"2024-04-25T19:19:18.074330+0000","last_became_active":"2024-04-25T19:15:19.420795+0000","last_became_peered":"2024-04-25T19:15:19.420795+0000","last_unstale":"2024-04-25T19:19:18.074330+0000","last_undegraded":"2024-04-25T19:19:18.074330+0000","last_fullsized":"2024-04-25T19:19:18.074330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:13.061506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'3","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.552520+0000","last_change":"2024-04-25T19:15:18.423841+0000","last_active":"2024-04-25T19:16:17.552520+0000","last_peered":"2024-04-25T19:16:17.552520+0000","last_clean":"2024-04-25T19:16:17.552520+0000","last_became_active":"2024-04-25T19:15:18.423500+0000","last_became_peered":"2024-04-25T19:15:18.423500+0000","last_unstale":"2024-04-25T19:16:17.552520+0000","last_undegraded":"2024-04-25T19:16:17.552520+0000","last_fullsized":"2024-04-25T19:16:17.552520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:58.933085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'135","reported_seq":152,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.018424+0000","last_change":"2024-04-25T19:15:19.416232+0000","last_active":"2024-04-25T19:19:18.018424+0000","last_peered":"2024-04-25T19:19:18.018424+0000","last_clean":"2024-04-25T19:19:18.018424+0000","last_became_active":"2024-04-25T19:15:19.415828+0000","last_became_peered":"2024-04-25T19:15:19.415828+0000","last_unstale":"2024-04-25T19:19:18.018424+0000","last_undegraded":"2024-04-25T19:19:18.018424+0000","last_fullsized":"2024-04-25T19:19:18.018424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:59.751840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:32.872674+0000","last_change":"2024-04-25T19:15:18.423918+0000","last_active":"2024-04-25T19:17:32.872674+0000","last_peered":"2024-04-25T19:17:32.872674+0000","last_clean":"2024-04-25T19:17:32.872674+0000","last_became_active":"2024-04-25T19:15:18.423588+0000","last_became_peered":"2024-04-25T19:15:18.423588+0000","last_unstale":"2024-04-25T19:17:32.872674+0000","last_undegraded":"2024-04-25T19:17:32.872674+0000","last_fullsized":"2024-04-25T19:17:32.872674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:16.378285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'123","reported_seq":140,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.998130+0000","last_change":"2024-04-25T19:15:19.422984+0000","last_active":"2024-04-25T19:19:17.998130+0000","last_peered":"2024-04-25T19:19:17.998130+0000","last_clean":"2024-04-25T19:19:17.998130+0000","last_became_active":"2024-04-25T19:15:19.422651+0000","last_became_peered":"2024-04-25T19:15:19.422651+0000","last_unstale":"2024-04-25T19:19:17.998130+0000","last_undegraded":"2024-04-25T19:19:17.998130+0000","last_fullsized":"2024-04-25T19:19:17.998130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:54.984914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'168","reported_seq":185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.068520+0000","last_change":"2024-04-25T19:15:19.423982+0000","last_active":"2024-04-25T19:19:18.068520+0000","last_peered":"2024-04-25T19:19:18.068520+0000","last_clean":"2024-04-25T19:19:18.068520+0000","last_became_active":"2024-04-25T19:15:19.423341+0000","last_became_peered":"2024-04-25T19:15:19.423341+0000","last_unstale":"2024-04-25T19:19:18.068520+0000","last_undegraded":"2024-04-25T19:19:18.068520+0000","last_fullsized":"2024-04-25T19:19:18.068520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:51.808352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.549697+0000","last_change":"2024-04-25T19:15:18.424455+0000","last_active":"2024-04-25T19:16:17.549697+0000","last_peered":"2024-04-25T19:16:17.549697+0000","last_clean":"2024-04-25T19:16:17.549697+0000","last_became_active":"2024-04-25T19:15:18.424119+0000","last_became_peered":"2024-04-25T19:15:18.424119+0000","last_unstale":"2024-04-25T19:16:17.549697+0000","last_undegraded":"2024-04-25T19:16:17.549697+0000","last_fullsized":"2024-04-25T19:16:17.549697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:43.135567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'132","reported_seq":149,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.023065+0000","last_change":"2024-04-25T19:15:19.424064+0000","last_active":"2024-04-25T19:19:18.023065+0000","last_peered":"2024-04-25T19:19:18.023065+0000","last_clean":"2024-04-25T19:19:18.023065+0000","last_became_active":"2024-04-25T19:15:19.423187+0000","last_became_peered":"2024-04-25T19:15:19.423187+0000","last_unstale":"2024-04-25T19:19:18.023065+0000","last_undegraded":"2024-04-25T19:19:18.023065+0000","last_fullsized":"2024-04-25T19:19:18.023065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:43:31.590700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"23'13","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:47.874568+0000","last_change":"2024-04-25T19:15:18.423151+0000","last_active":"2024-04-25T19:17:47.874568+0000","last_peered":"2024-04-25T19:17:47.874568+0000","last_clean":"2024-04-25T19:17:47.874568+0000","last_became_active":"2024-04-25T19:15:18.422831+0000","last_became_peered":"2024-04-25T19:15:18.422831+0000","last_unstale":"2024-04-25T19:17:47.874568+0000","last_undegraded":"2024-04-25T19:17:47.874568+0000","last_fullsized":"2024-04-25T19:17:47.874568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:43.244963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'129","reported_seq":146,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.087772+0000","last_change":"2024-04-25T19:15:19.416717+0000","last_active":"2024-04-25T19:19:18.087772+0000","last_peered":"2024-04-25T19:19:18.087772+0000","last_clean":"2024-04-25T19:19:18.087772+0000","last_became_active":"2024-04-25T19:15:19.416163+0000","last_became_peered":"2024-04-25T19:15:19.416163+0000","last_unstale":"2024-04-25T19:19:18.087772+0000","last_undegraded":"2024-04-25T19:19:18.087772+0000","last_fullsized":"2024-04-25T19:19:18.087772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:46.955159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:17.863896+0000","last_change":"2024-04-25T19:15:18.422875+0000","last_active":"2024-04-25T19:17:17.863896+0000","last_peered":"2024-04-25T19:17:17.863896+0000","last_clean":"2024-04-25T19:17:17.863896+0000","last_became_active":"2024-04-25T19:15:18.422489+0000","last_became_peered":"2024-04-25T19:15:18.422489+0000","last_unstale":"2024-04-25T19:17:17.863896+0000","last_undegraded":"2024-04-25T19:17:17.863896+0000","last_fullsized":"2024-04-25T19:17:17.863896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:30.602500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'147","reported_seq":165,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.080832+0000","last_change":"2024-04-25T19:15:19.416987+0000","last_active":"2024-04-25T19:19:18.080832+0000","last_peered":"2024-04-25T19:19:18.080832+0000","last_clean":"2024-04-25T19:19:18.080832+0000","last_became_active":"2024-04-25T19:15:19.414173+0000","last_became_peered":"2024-04-25T19:15:19.414173+0000","last_unstale":"2024-04-25T19:19:18.080832+0000","last_undegraded":"2024-04-25T19:19:18.080832+0000","last_fullsized":"2024-04-25T19:19:18.080832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:32:43.188653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'13","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:22.862662+0000","last_change":"2024-04-25T19:15:18.423867+0000","last_active":"2024-04-25T19:17:22.862662+0000","last_peered":"2024-04-25T19:17:22.862662+0000","last_clean":"2024-04-25T19:17:22.862662+0000","last_became_active":"2024-04-25T19:15:18.423532+0000","last_became_peered":"2024-04-25T19:15:18.423532+0000","last_unstale":"2024-04-25T19:17:22.862662+0000","last_undegraded":"2024-04-25T19:17:22.862662+0000","last_fullsized":"2024-04-25T19:17:22.862662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:04.531338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9217,"ondisk_log_size":9217,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":224795,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":344,"num_write_kb":333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":696320,"data_stored":449590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64838,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574902,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39832,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332008,"statfs":{"total":96636764160,"available":96595976192,"internally_reserved":0,"allocated":344064,"data_stored":680704,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35808,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336032,"statfs":{"total":96636764160,"available":96600096768,"internally_reserved":0,"allocated":483328,"data_stored":305571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574902,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39780,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332060,"statfs":{"total":96636764160,"available":96596029440,"internally_reserved":0,"allocated":290816,"data_stored":92168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574902,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39756,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332084,"statfs":{"total":96636764160,"available":96596054016,"internally_reserved":0,"allocated":266240,"data_stored":89566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39976,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331864,"statfs":{"total":96636764160,"available":96595828736,"internally_reserved":0,"allocated":491520,"data_stored":306110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574902,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35608,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336232,"statfs":{"total":96636764160,"available":96600301568,"internally_reserved":0,"allocated":278528,"data_stored":90203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35600,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336240,"statfs":{"total":96636764160,"available":96600309760,"internally_reserved":0,"allocated":270336,"data_stored":89327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39824,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332016,"statfs":{"total":96636764160,"available":96595984384,"internally_reserved":0,"allocated":335872,"data_stored":679037,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":249856,"data_stored":218315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":217776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T19:19:33.916 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T19:19:34.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.059+0000 7ffa72526640 1 Processor -- start 2024-04-25T19:19:34.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.059+0000 7ffa72526640 1 -- start start 2024-04-25T19:19:34.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.060+0000 7ffa72526640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffa6c06e870 0x7ffa6c0cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:34.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.060+0000 7ffa72526640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa6c15ce80 con 0x7ffa6c0d0810 2024-04-25T19:19:34.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.060+0000 7ffa72526640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa6c15d160 con 0x7ffa6c0cea70 2024-04-25T19:19:34.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.060+0000 7ffa72526640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ffa6c15d440 con 0x7ffa6c06e870 2024-04-25T19:19:34.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.060+0000 7ffa70ac3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffa6c06e870 0x7ffa6c0cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:34.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.060+0000 7ffa6bfff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ffa6c0d0810 0x7ffa6c06f000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35766/0 (socket says 172.21.15.76:35766) 2024-04-25T19:19:34.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.060+0000 7ffa6bfff640 1 -- 172.21.15.76:0/953773970 learned_addr learned my addr 172.21.15.76:0/953773970 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:34.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.061+0000 7ffa70ac3640 1 -- 172.21.15.76:0/953773970 >> v1:172.21.15.105:6789/0 conn(0x7ffa6c0cea70 legacy=0x7ffa6c06e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:34.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.061+0000 7ffa70ac3640 1 -- 172.21.15.76:0/953773970 >> v1:172.21.15.76:6789/0 conn(0x7ffa6c0d0810 legacy=0x7ffa6c06f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:34.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.061+0000 7ffa6affd640 1 -- 172.21.15.76:0/953773970 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1569935106 0 0) 0x7ffa6c15ce80 con 0x7ffa6c0d0810 2024-04-25T19:19:34.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.061+0000 7ffa70ac3640 1 -- 172.21.15.76:0/953773970 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6c15d720 con 0x7ffa6c06e870 2024-04-25T19:19:34.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.061+0000 7ffa70ac3640 1 --2- 172.21.15.76:0/953773970 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa6c06e870 0x7ffa6c0cee80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009d90 tx=0x7ffa6402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19129178bfcf5359 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:34.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.061+0000 7ffa6affd640 1 -- 172.21.15.76:0/953773970 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa64007740 con 0x7ffa6c06e870 2024-04-25T19:19:34.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.061+0000 7ffa6affd640 1 -- 172.21.15.76:0/953773970 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa640078e0 con 0x7ffa6c06e870 2024-04-25T19:19:34.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.061+0000 7ffa6affd640 1 -- 172.21.15.76:0/953773970 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa64007cb0 con 0x7ffa6c06e870 2024-04-25T19:19:34.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.062+0000 7ffa72526640 1 -- 172.21.15.76:0/953773970 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa6c06e870 msgr2=0x7ffa6c0cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:34.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.062+0000 7ffa72526640 1 --2- 172.21.15.76:0/953773970 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa6c06e870 0x7ffa6c0cee80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009d90 tx=0x7ffa6402a6a0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.062+0000 7ffa72526640 1 -- 172.21.15.76:0/953773970 shutdown_connections 2024-04-25T19:19:34.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.062+0000 7ffa72526640 1 --2- 172.21.15.76:0/953773970 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa6c06e870 0x7ffa6c0cee80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.062+0000 7ffa72526640 1 -- 172.21.15.76:0/953773970 >> 172.21.15.76:0/953773970 conn(0x7ffa6c0b3c00 msgr2=0x7ffa6c0b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:34.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.062+0000 7ffa72526640 1 -- 172.21.15.76:0/953773970 shutdown_connections 2024-04-25T19:19:34.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.062+0000 7ffa72526640 1 -- 172.21.15.76:0/953773970 wait complete. 2024-04-25T19:19:34.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.064+0000 7ffa72526640 1 Processor -- start 2024-04-25T19:19:34.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.064+0000 7ffa72526640 1 -- start start 2024-04-25T19:19:34.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.065+0000 7ffa72526640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa6c06e870 0x7ffa6c1534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:34.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.065+0000 7ffa72526640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa6c0cea70 0x7ffa6c1f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:34.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.065+0000 7ffa72526640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa6c0d0810 0x7ffa6c150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:34.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.065+0000 7ffa72526640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c15cb70 con 0x7ffa6c06e870 2024-04-25T19:19:34.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.065+0000 7ffa72526640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c06dc40 con 0x7ffa6c0cea70 2024-04-25T19:19:34.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.065+0000 7ffa72526640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ffa6c1570e0 con 0x7ffa6c0d0810 2024-04-25T19:19:34.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.065+0000 7ffa6bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa6c06e870 0x7ffa6c1534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:34.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.065+0000 7ffa6bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa6c06e870 0x7ffa6c1534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50586/0 (socket says 172.21.15.76:50586) 2024-04-25T19:19:34.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.065+0000 7ffa6bfff640 1 -- 172.21.15.76:0/745247134 learned_addr learned my addr 172.21.15.76:0/745247134 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:34.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.066+0000 7ffa70ac3640 1 --2- 172.21.15.76:0/745247134 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa6c0d0810 0x7ffa6c150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:34.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.066+0000 7ffa6b7fe640 1 --2- 172.21.15.76:0/745247134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa6c0cea70 0x7ffa6c1f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:34.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.066+0000 7ffa6bfff640 1 -- 172.21.15.76:0/745247134 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa6c0d0810 msgr2=0x7ffa6c150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:34.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.066+0000 7ffa6bfff640 1 --2- 172.21.15.76:0/745247134 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa6c0d0810 0x7ffa6c150bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.066+0000 7ffa6bfff640 1 -- 172.21.15.76:0/745247134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa6c0cea70 msgr2=0x7ffa6c1f35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:34.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.066+0000 7ffa6bfff640 1 --2- 172.21.15.76:0/745247134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa6c0cea70 0x7ffa6c1f35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.066+0000 7ffa6bfff640 1 -- 172.21.15.76:0/745247134 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6c1f3b30 con 0x7ffa6c06e870 2024-04-25T19:19:34.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.066+0000 7ffa70ac3640 1 --2- 172.21.15.76:0/745247134 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa6c0d0810 0x7ffa6c150bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:19:34.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.066+0000 7ffa6b7fe640 1 --2- 172.21.15.76:0/745247134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa6c0cea70 0x7ffa6c1f35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:19:34.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.066+0000 7ffa6bfff640 1 --2- 172.21.15.76:0/745247134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa6c06e870 0x7ffa6c1534e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00c460 tx=0x7ffa5c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:34.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.066+0000 7ffa68ff9640 1 -- 172.21.15.76:0/745247134 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c018070 con 0x7ffa6c06e870 2024-04-25T19:19:34.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.066+0000 7ffa68ff9640 1 -- 172.21.15.76:0/745247134 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa5c007500 con 0x7ffa6c06e870 2024-04-25T19:19:34.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.066+0000 7ffa68ff9640 1 -- 172.21.15.76:0/745247134 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c004a20 con 0x7ffa6c06e870 2024-04-25T19:19:34.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.066+0000 7ffa72526640 1 -- 172.21.15.76:0/745247134 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa6c151b50 con 0x7ffa6c06e870 2024-04-25T19:19:34.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.066+0000 7ffa72526640 1 -- 172.21.15.76:0/745247134 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa6c152110 con 0x7ffa6c06e870 2024-04-25T19:19:34.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.067+0000 7ffa72526640 1 -- 172.21.15.76:0/745247134 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa6c0d1510 con 0x7ffa6c06e870 2024-04-25T19:19:34.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.068+0000 7ffa68ff9640 1 -- 172.21.15.76:0/745247134 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ffa5c01b070 con 0x7ffa6c06e870 2024-04-25T19:19:34.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.068+0000 7ffa68ff9640 1 --2- 172.21.15.76:0/745247134 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ffa580744e0 0x7ffa580769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:34.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.068+0000 7ffa68ff9640 1 -- 172.21.15.76:0/745247134 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7ffa5c0944f0 con 0x7ffa6c06e870 2024-04-25T19:19:34.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.069+0000 7ffa6b7fe640 1 --2- 172.21.15.76:0/745247134 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ffa580744e0 0x7ffa580769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:34.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.069+0000 7ffa6b7fe640 1 --2- 172.21.15.76:0/745247134 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ffa580744e0 0x7ffa580769f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffa60006fd0 tx=0x7ffa60008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:34.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.073+0000 7ffa68ff9640 1 -- 172.21.15.76:0/745247134 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffa6c0d1510 con 0x7ffa6c06e870 2024-04-25T19:19:34.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.245+0000 7ffa72526640 1 -- 172.21.15.76:0/745247134 --> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffa6c0507e0 con 0x7ffa580744e0 2024-04-25T19:19:34.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.251+0000 7ffa68ff9640 1 -- 172.21.15.76:0/745247134 <== mgr.4109 v2:172.21.15.105:6832/2585775531 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+253023 (secure 0 0 0) 0x7ffa6c0507e0 con 0x7ffa580744e0 2024-04-25T19:19:34.255 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:19:34.256 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-04-25T19:19:34.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.254+0000 7ffa72526640 1 -- 172.21.15.76:0/745247134 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ffa580744e0 msgr2=0x7ffa580769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:34.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.254+0000 7ffa72526640 1 --2- 172.21.15.76:0/745247134 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ffa580744e0 0x7ffa580769f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffa60006fd0 tx=0x7ffa60008040 comp rx=0 tx=0).stop 2024-04-25T19:19:34.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.254+0000 7ffa72526640 1 -- 172.21.15.76:0/745247134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa6c06e870 msgr2=0x7ffa6c1534e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:34.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.254+0000 7ffa72526640 1 --2- 172.21.15.76:0/745247134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa6c06e870 0x7ffa6c1534e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00c460 tx=0x7ffa5c00c920 comp rx=0 tx=0).stop 2024-04-25T19:19:34.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.254+0000 7ffa72526640 1 -- 172.21.15.76:0/745247134 shutdown_connections 2024-04-25T19:19:34.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.254+0000 7ffa72526640 1 --2- 172.21.15.76:0/745247134 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ffa580744e0 0x7ffa580769f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.254+0000 7ffa72526640 1 --2- 172.21.15.76:0/745247134 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ffa6c0d0810 0x7ffa6c150bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.254+0000 7ffa72526640 1 --2- 172.21.15.76:0/745247134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa6c0cea70 0x7ffa6c1f35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.254+0000 7ffa72526640 1 --2- 172.21.15.76:0/745247134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa6c06e870 0x7ffa6c1534e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.254+0000 7ffa72526640 1 -- 172.21.15.76:0/745247134 >> 172.21.15.76:0/745247134 conn(0x7ffa6c0b3c00 msgr2=0x7ffa6c0ca220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:34.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.254+0000 7ffa72526640 1 -- 172.21.15.76:0/745247134 shutdown_connections 2024-04-25T19:19:34.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.254+0000 7ffa72526640 1 -- 172.21.15.76:0/745247134 wait complete. 2024-04-25T19:19:34.267 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-04-25T19:19:33.896521+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":815163,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":70,"num_read_kb":59,"num_write":9661,"num_write_kb":25167785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9595,"ondisk_log_size":9595,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":306184,"kb_used_data":2696,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754668536,"statfs":{"total":773094113280,"available":772780580864,"internally_reserved":0,"allocated":2760704,"data_stored":2332686,"data_compressed":64838,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":310693424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1413477513,"num_objects":-337,"num_object_clones":0,"num_object_copies":-674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":340,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004062"},"pg_stats":[{"pgid":"3.3f","version":"23'138","reported_seq":156,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.068380+0000","last_change":"2024-04-25T19:15:19.433569+0000","last_active":"2024-04-25T19:19:18.068380+0000","last_peered":"2024-04-25T19:19:18.068380+0000","last_clean":"2024-04-25T19:19:18.068380+0000","last_became_active":"2024-04-25T19:15:19.431332+0000","last_became_peered":"2024-04-25T19:15:19.431332+0000","last_unstale":"2024-04-25T19:19:18.068380+0000","last_undegraded":"2024-04-25T19:19:18.068380+0000","last_fullsized":"2024-04-25T19:19:18.068380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:57:01.213834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'126","reported_seq":144,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.090933+0000","last_change":"2024-04-25T19:15:19.418695+0000","last_active":"2024-04-25T19:19:18.090933+0000","last_peered":"2024-04-25T19:19:18.090933+0000","last_clean":"2024-04-25T19:19:18.090933+0000","last_became_active":"2024-04-25T19:15:19.418235+0000","last_became_peered":"2024-04-25T19:15:19.418235+0000","last_unstale":"2024-04-25T19:19:18.090933+0000","last_undegraded":"2024-04-25T19:19:18.090933+0000","last_fullsized":"2024-04-25T19:19:18.090933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:30.516472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'144","reported_seq":162,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.024866+0000","last_change":"2024-04-25T19:15:19.416983+0000","last_active":"2024-04-25T19:19:18.024866+0000","last_peered":"2024-04-25T19:19:18.024866+0000","last_clean":"2024-04-25T19:19:18.024866+0000","last_became_active":"2024-04-25T19:15:19.416443+0000","last_became_peered":"2024-04-25T19:15:19.416443+0000","last_unstale":"2024-04-25T19:19:18.024866+0000","last_undegraded":"2024-04-25T19:19:18.024866+0000","last_fullsized":"2024-04-25T19:19:18.024866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:02.794164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'165","reported_seq":183,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.098994+0000","last_change":"2024-04-25T19:15:19.429058+0000","last_active":"2024-04-25T19:19:18.098994+0000","last_peered":"2024-04-25T19:19:18.098994+0000","last_clean":"2024-04-25T19:19:18.098994+0000","last_became_active":"2024-04-25T19:15:19.428419+0000","last_became_peered":"2024-04-25T19:15:19.428419+0000","last_unstale":"2024-04-25T19:19:18.098994+0000","last_undegraded":"2024-04-25T19:19:18.098994+0000","last_fullsized":"2024-04-25T19:19:18.098994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:23:56.817616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'144","reported_seq":161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.073061+0000","last_change":"2024-04-25T19:15:19.427615+0000","last_active":"2024-04-25T19:19:18.073061+0000","last_peered":"2024-04-25T19:19:18.073061+0000","last_clean":"2024-04-25T19:19:18.073061+0000","last_became_active":"2024-04-25T19:15:19.426968+0000","last_became_peered":"2024-04-25T19:15:19.426968+0000","last_unstale":"2024-04-25T19:19:18.073061+0000","last_undegraded":"2024-04-25T19:19:18.073061+0000","last_fullsized":"2024-04-25T19:19:18.073061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:42:00.686712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'129","reported_seq":146,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.070490+0000","last_change":"2024-04-25T19:15:19.417602+0000","last_active":"2024-04-25T19:19:18.070490+0000","last_peered":"2024-04-25T19:19:18.070490+0000","last_clean":"2024-04-25T19:19:18.070490+0000","last_became_active":"2024-04-25T19:15:19.417292+0000","last_became_peered":"2024-04-25T19:15:19.417292+0000","last_unstale":"2024-04-25T19:19:18.070490+0000","last_undegraded":"2024-04-25T19:19:18.070490+0000","last_fullsized":"2024-04-25T19:19:18.070490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:37:43.583809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'159","reported_seq":176,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.041488+0000","last_change":"2024-04-25T19:15:19.428180+0000","last_active":"2024-04-25T19:19:18.041488+0000","last_peered":"2024-04-25T19:19:18.041488+0000","last_clean":"2024-04-25T19:19:18.041488+0000","last_became_active":"2024-04-25T19:15:19.427852+0000","last_became_peered":"2024-04-25T19:15:19.427852+0000","last_unstale":"2024-04-25T19:19:18.041488+0000","last_undegraded":"2024-04-25T19:19:18.041488+0000","last_fullsized":"2024-04-25T19:19:18.041488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:51.300880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'177","reported_seq":194,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.083993+0000","last_change":"2024-04-25T19:15:19.418253+0000","last_active":"2024-04-25T19:19:18.083993+0000","last_peered":"2024-04-25T19:19:18.083993+0000","last_clean":"2024-04-25T19:19:18.083993+0000","last_became_active":"2024-04-25T19:15:19.417183+0000","last_became_peered":"2024-04-25T19:15:19.417183+0000","last_unstale":"2024-04-25T19:19:18.083993+0000","last_undegraded":"2024-04-25T19:19:18.083993+0000","last_fullsized":"2024-04-25T19:19:18.083993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:05:13.181646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'141","reported_seq":158,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.050350+0000","last_change":"2024-04-25T19:15:19.431671+0000","last_active":"2024-04-25T19:19:18.050350+0000","last_peered":"2024-04-25T19:19:18.050350+0000","last_clean":"2024-04-25T19:19:18.050350+0000","last_became_active":"2024-04-25T19:15:19.431240+0000","last_became_peered":"2024-04-25T19:15:19.431240+0000","last_unstale":"2024-04-25T19:19:18.050350+0000","last_undegraded":"2024-04-25T19:19:18.050350+0000","last_fullsized":"2024-04-25T19:19:18.050350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:22.759462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'141","reported_seq":163,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.058501+0000","last_change":"2024-04-25T19:16:17.555086+0000","last_active":"2024-04-25T19:19:18.058501+0000","last_peered":"2024-04-25T19:19:18.058501+0000","last_clean":"2024-04-25T19:19:18.058501+0000","last_became_active":"2024-04-25T19:16:17.554771+0000","last_became_peered":"2024-04-25T19:16:17.554771+0000","last_unstale":"2024-04-25T19:19:18.058501+0000","last_undegraded":"2024-04-25T19:19:18.058501+0000","last_fullsized":"2024-04-25T19:19:18.058501+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:22:35.638155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'183","reported_seq":200,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.036612+0000","last_change":"2024-04-25T19:15:19.417596+0000","last_active":"2024-04-25T19:19:18.036612+0000","last_peered":"2024-04-25T19:19:18.036612+0000","last_clean":"2024-04-25T19:19:18.036612+0000","last_became_active":"2024-04-25T19:15:19.417230+0000","last_became_peered":"2024-04-25T19:15:19.417230+0000","last_unstale":"2024-04-25T19:19:18.036612+0000","last_undegraded":"2024-04-25T19:19:18.036612+0000","last_fullsized":"2024-04-25T19:19:18.036612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:28:12.387651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'135","reported_seq":152,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.052108+0000","last_change":"2024-04-25T19:15:19.420564+0000","last_active":"2024-04-25T19:19:18.052108+0000","last_peered":"2024-04-25T19:19:18.052108+0000","last_clean":"2024-04-25T19:19:18.052108+0000","last_became_active":"2024-04-25T19:15:19.420044+0000","last_became_peered":"2024-04-25T19:15:19.420044+0000","last_unstale":"2024-04-25T19:19:18.052108+0000","last_undegraded":"2024-04-25T19:19:18.052108+0000","last_fullsized":"2024-04-25T19:19:18.052108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:04.066030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'141","reported_seq":158,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.030803+0000","last_change":"2024-04-25T19:15:19.415500+0000","last_active":"2024-04-25T19:19:18.030803+0000","last_peered":"2024-04-25T19:19:18.030803+0000","last_clean":"2024-04-25T19:19:18.030803+0000","last_became_active":"2024-04-25T19:15:19.414007+0000","last_became_peered":"2024-04-25T19:15:19.414007+0000","last_unstale":"2024-04-25T19:19:18.030803+0000","last_undegraded":"2024-04-25T19:19:18.030803+0000","last_fullsized":"2024-04-25T19:19:18.030803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:56:12.320555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'141","reported_seq":158,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.052414+0000","last_change":"2024-04-25T19:15:19.423884+0000","last_active":"2024-04-25T19:19:18.052414+0000","last_peered":"2024-04-25T19:19:18.052414+0000","last_clean":"2024-04-25T19:19:18.052414+0000","last_became_active":"2024-04-25T19:15:19.422575+0000","last_became_peered":"2024-04-25T19:15:19.422575+0000","last_unstale":"2024-04-25T19:19:18.052414+0000","last_undegraded":"2024-04-25T19:19:18.052414+0000","last_fullsized":"2024-04-25T19:19:18.052414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:30.988950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'144","reported_seq":161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.064766+0000","last_change":"2024-04-25T19:15:19.424267+0000","last_active":"2024-04-25T19:19:18.064766+0000","last_peered":"2024-04-25T19:19:18.064766+0000","last_clean":"2024-04-25T19:19:18.064766+0000","last_became_active":"2024-04-25T19:15:19.422953+0000","last_became_peered":"2024-04-25T19:15:19.422953+0000","last_unstale":"2024-04-25T19:19:18.064766+0000","last_undegraded":"2024-04-25T19:19:18.064766+0000","last_fullsized":"2024-04-25T19:19:18.064766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:42.823233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'138","reported_seq":149,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.018289+0000","last_change":"2024-04-25T19:16:17.801933+0000","last_active":"2024-04-25T19:19:18.018289+0000","last_peered":"2024-04-25T19:19:18.018289+0000","last_clean":"2024-04-25T19:19:18.018289+0000","last_became_active":"2024-04-25T19:16:17.801500+0000","last_became_peered":"2024-04-25T19:16:17.801500+0000","last_unstale":"2024-04-25T19:19:18.018289+0000","last_undegraded":"2024-04-25T19:19:18.018289+0000","last_fullsized":"2024-04-25T19:19:18.018289+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:41.633057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"23'138","reported_seq":155,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.038730+0000","last_change":"2024-04-25T19:15:19.434476+0000","last_active":"2024-04-25T19:19:18.038730+0000","last_peered":"2024-04-25T19:19:18.038730+0000","last_clean":"2024-04-25T19:19:18.038730+0000","last_became_active":"2024-04-25T19:15:19.433836+0000","last_became_peered":"2024-04-25T19:15:19.433836+0000","last_unstale":"2024-04-25T19:19:18.038730+0000","last_undegraded":"2024-04-25T19:19:18.038730+0000","last_fullsized":"2024-04-25T19:19:18.038730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:13.234829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"23'147","reported_seq":164,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.181499+0000","last_change":"2024-04-25T19:15:19.415521+0000","last_active":"2024-04-25T19:19:18.181499+0000","last_peered":"2024-04-25T19:19:18.181499+0000","last_clean":"2024-04-25T19:19:18.181499+0000","last_became_active":"2024-04-25T19:15:19.415177+0000","last_became_peered":"2024-04-25T19:15:19.415177+0000","last_unstale":"2024-04-25T19:19:18.181499+0000","last_undegraded":"2024-04-25T19:19:18.181499+0000","last_fullsized":"2024-04-25T19:19:18.181499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:24.355619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'165","reported_seq":182,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.085552+0000","last_change":"2024-04-25T19:15:19.430291+0000","last_active":"2024-04-25T19:19:18.085552+0000","last_peered":"2024-04-25T19:19:18.085552+0000","last_clean":"2024-04-25T19:19:18.085552+0000","last_became_active":"2024-04-25T19:15:19.427226+0000","last_became_peered":"2024-04-25T19:15:19.427226+0000","last_unstale":"2024-04-25T19:19:18.085552+0000","last_undegraded":"2024-04-25T19:19:18.085552+0000","last_fullsized":"2024-04-25T19:19:18.085552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:38.464211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"23'153","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.064081+0000","last_change":"2024-04-25T19:15:19.421696+0000","last_active":"2024-04-25T19:19:18.064081+0000","last_peered":"2024-04-25T19:19:18.064081+0000","last_clean":"2024-04-25T19:19:18.064081+0000","last_became_active":"2024-04-25T19:15:19.421300+0000","last_became_peered":"2024-04-25T19:15:19.421300+0000","last_unstale":"2024-04-25T19:19:18.064081+0000","last_undegraded":"2024-04-25T19:19:18.064081+0000","last_fullsized":"2024-04-25T19:19:18.064081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:15.226398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'126","reported_seq":143,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.090958+0000","last_change":"2024-04-25T19:15:19.427880+0000","last_active":"2024-04-25T19:19:18.090958+0000","last_peered":"2024-04-25T19:19:18.090958+0000","last_clean":"2024-04-25T19:19:18.090958+0000","last_became_active":"2024-04-25T19:15:19.427579+0000","last_became_peered":"2024-04-25T19:15:19.427579+0000","last_unstale":"2024-04-25T19:19:18.090958+0000","last_undegraded":"2024-04-25T19:19:18.090958+0000","last_fullsized":"2024-04-25T19:19:18.090958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:47:10.657715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'156","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.083315+0000","last_change":"2024-04-25T19:15:19.430409+0000","last_active":"2024-04-25T19:19:18.083315+0000","last_peered":"2024-04-25T19:19:18.083315+0000","last_clean":"2024-04-25T19:19:18.083315+0000","last_became_active":"2024-04-25T19:15:19.428305+0000","last_became_peered":"2024-04-25T19:15:19.428305+0000","last_unstale":"2024-04-25T19:19:18.083315+0000","last_undegraded":"2024-04-25T19:19:18.083315+0000","last_fullsized":"2024-04-25T19:19:18.083315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:10:01.268963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"23'126","reported_seq":143,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.827387+0000","last_change":"2024-04-25T19:15:19.424435+0000","last_active":"2024-04-25T19:19:17.827387+0000","last_peered":"2024-04-25T19:19:17.827387+0000","last_clean":"2024-04-25T19:19:17.827387+0000","last_became_active":"2024-04-25T19:15:19.422763+0000","last_became_peered":"2024-04-25T19:15:19.422763+0000","last_unstale":"2024-04-25T19:19:17.827387+0000","last_undegraded":"2024-04-25T19:19:17.827387+0000","last_fullsized":"2024-04-25T19:19:17.827387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:58.092412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'174","reported_seq":191,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.937851+0000","last_change":"2024-04-25T19:15:19.420275+0000","last_active":"2024-04-25T19:19:17.937851+0000","last_peered":"2024-04-25T19:19:17.937851+0000","last_clean":"2024-04-25T19:19:17.937851+0000","last_became_active":"2024-04-25T19:15:19.419936+0000","last_became_peered":"2024-04-25T19:15:19.419936+0000","last_unstale":"2024-04-25T19:19:17.937851+0000","last_undegraded":"2024-04-25T19:19:17.937851+0000","last_fullsized":"2024-04-25T19:19:17.937851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:55.710620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'132","reported_seq":149,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.061024+0000","last_change":"2024-04-25T19:15:19.423408+0000","last_active":"2024-04-25T19:19:18.061024+0000","last_peered":"2024-04-25T19:19:18.061024+0000","last_clean":"2024-04-25T19:19:18.061024+0000","last_became_active":"2024-04-25T19:15:19.423116+0000","last_became_peered":"2024-04-25T19:15:19.423116+0000","last_unstale":"2024-04-25T19:19:18.061024+0000","last_undegraded":"2024-04-25T19:19:18.061024+0000","last_fullsized":"2024-04-25T19:19:18.061024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:18:57.787013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'183","reported_seq":200,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.071336+0000","last_change":"2024-04-25T19:15:19.428826+0000","last_active":"2024-04-25T19:19:18.071336+0000","last_peered":"2024-04-25T19:19:18.071336+0000","last_clean":"2024-04-25T19:19:18.071336+0000","last_became_active":"2024-04-25T19:15:19.428474+0000","last_became_peered":"2024-04-25T19:15:19.428474+0000","last_unstale":"2024-04-25T19:19:18.071336+0000","last_undegraded":"2024-04-25T19:19:18.071336+0000","last_fullsized":"2024-04-25T19:19:18.071336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:19.974851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'138","reported_seq":155,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.059590+0000","last_change":"2024-04-25T19:15:19.423353+0000","last_active":"2024-04-25T19:19:18.059590+0000","last_peered":"2024-04-25T19:19:18.059590+0000","last_clean":"2024-04-25T19:19:18.059590+0000","last_became_active":"2024-04-25T19:15:19.422795+0000","last_became_peered":"2024-04-25T19:15:19.422795+0000","last_unstale":"2024-04-25T19:19:18.059590+0000","last_undegraded":"2024-04-25T19:19:18.059590+0000","last_fullsized":"2024-04-25T19:19:18.059590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:34.119976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'129","reported_seq":146,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.066581+0000","last_change":"2024-04-25T19:15:19.421809+0000","last_active":"2024-04-25T19:19:18.066581+0000","last_peered":"2024-04-25T19:19:18.066581+0000","last_clean":"2024-04-25T19:19:18.066581+0000","last_became_active":"2024-04-25T19:15:19.421515+0000","last_became_peered":"2024-04-25T19:15:19.421515+0000","last_unstale":"2024-04-25T19:19:18.066581+0000","last_undegraded":"2024-04-25T19:19:18.066581+0000","last_fullsized":"2024-04-25T19:19:18.066581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:22.603291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'111","reported_seq":129,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.068349+0000","last_change":"2024-04-25T19:15:19.410183+0000","last_active":"2024-04-25T19:19:18.068349+0000","last_peered":"2024-04-25T19:19:18.068349+0000","last_clean":"2024-04-25T19:19:18.068349+0000","last_became_active":"2024-04-25T19:15:19.409857+0000","last_became_peered":"2024-04-25T19:15:19.409857+0000","last_unstale":"2024-04-25T19:19:18.068349+0000","last_undegraded":"2024-04-25T19:19:18.068349+0000","last_fullsized":"2024-04-25T19:19:18.068349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:30.180786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'123","reported_seq":141,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.068201+0000","last_change":"2024-04-25T19:15:19.425524+0000","last_active":"2024-04-25T19:19:18.068201+0000","last_peered":"2024-04-25T19:19:18.068201+0000","last_clean":"2024-04-25T19:19:18.068201+0000","last_became_active":"2024-04-25T19:15:19.424578+0000","last_became_peered":"2024-04-25T19:15:19.424578+0000","last_unstale":"2024-04-25T19:19:18.068201+0000","last_undegraded":"2024-04-25T19:19:18.068201+0000","last_fullsized":"2024-04-25T19:19:18.068201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:02.103879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"23'123","reported_seq":141,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.093807+0000","last_change":"2024-04-25T19:15:19.426720+0000","last_active":"2024-04-25T19:19:18.093807+0000","last_peered":"2024-04-25T19:19:18.093807+0000","last_clean":"2024-04-25T19:19:18.093807+0000","last_became_active":"2024-04-25T19:15:19.426139+0000","last_became_peered":"2024-04-25T19:15:19.426139+0000","last_unstale":"2024-04-25T19:19:18.093807+0000","last_undegraded":"2024-04-25T19:19:18.093807+0000","last_fullsized":"2024-04-25T19:19:18.093807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:31.608993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'141","reported_seq":159,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.097157+0000","last_change":"2024-04-25T19:15:19.413805+0000","last_active":"2024-04-25T19:19:18.097157+0000","last_peered":"2024-04-25T19:19:18.097157+0000","last_clean":"2024-04-25T19:19:18.097157+0000","last_became_active":"2024-04-25T19:15:19.413455+0000","last_became_peered":"2024-04-25T19:15:19.413455+0000","last_unstale":"2024-04-25T19:19:18.097157+0000","last_undegraded":"2024-04-25T19:19:18.097157+0000","last_fullsized":"2024-04-25T19:19:18.097157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:05:45.008076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'123","reported_seq":140,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.921522+0000","last_change":"2024-04-25T19:15:19.423770+0000","last_active":"2024-04-25T19:19:17.921522+0000","last_peered":"2024-04-25T19:19:17.921522+0000","last_clean":"2024-04-25T19:19:17.921522+0000","last_became_active":"2024-04-25T19:15:19.423406+0000","last_became_peered":"2024-04-25T19:15:19.423406+0000","last_unstale":"2024-04-25T19:19:17.921522+0000","last_undegraded":"2024-04-25T19:19:17.921522+0000","last_fullsized":"2024-04-25T19:19:17.921522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:41:23.669832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'6","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.660321+0000","last_change":"2024-04-25T19:15:18.422035+0000","last_active":"2024-04-25T19:16:17.660321+0000","last_peered":"2024-04-25T19:16:17.660321+0000","last_clean":"2024-04-25T19:16:17.660321+0000","last_became_active":"2024-04-25T19:15:18.421658+0000","last_became_peered":"2024-04-25T19:15:18.421658+0000","last_unstale":"2024-04-25T19:16:17.660321+0000","last_undegraded":"2024-04-25T19:16:17.660321+0000","last_fullsized":"2024-04-25T19:16:17.660321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:23.349126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1063,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'129","reported_seq":146,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.994924+0000","last_change":"2024-04-25T19:15:19.421805+0000","last_active":"2024-04-25T19:19:17.994924+0000","last_peered":"2024-04-25T19:19:17.994924+0000","last_clean":"2024-04-25T19:19:17.994924+0000","last_became_active":"2024-04-25T19:15:19.421497+0000","last_became_peered":"2024-04-25T19:15:19.421497+0000","last_unstale":"2024-04-25T19:19:17.994924+0000","last_undegraded":"2024-04-25T19:19:17.994924+0000","last_fullsized":"2024-04-25T19:19:17.994924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:29:29.949508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'7","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.553158+0000","last_change":"2024-04-25T19:15:18.416894+0000","last_active":"2024-04-25T19:16:17.553158+0000","last_peered":"2024-04-25T19:16:17.553158+0000","last_clean":"2024-04-25T19:16:17.553158+0000","last_became_active":"2024-04-25T19:15:18.416539+0000","last_became_peered":"2024-04-25T19:15:18.416539+0000","last_unstale":"2024-04-25T19:16:17.553158+0000","last_undegraded":"2024-04-25T19:16:17.553158+0000","last_fullsized":"2024-04-25T19:16:17.553158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:30.388606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"21'3","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.659993+0000","last_change":"2024-04-25T19:15:18.420129+0000","last_active":"2024-04-25T19:16:17.659993+0000","last_peered":"2024-04-25T19:16:17.659993+0000","last_clean":"2024-04-25T19:16:17.659993+0000","last_became_active":"2024-04-25T19:15:18.419732+0000","last_became_peered":"2024-04-25T19:15:18.419732+0000","last_unstale":"2024-04-25T19:16:17.659993+0000","last_undegraded":"2024-04-25T19:16:17.659993+0000","last_fullsized":"2024-04-25T19:16:17.659993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:41:48.673989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'162","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.034836+0000","last_change":"2024-04-25T19:16:17.801934+0000","last_active":"2024-04-25T19:19:18.034836+0000","last_peered":"2024-04-25T19:19:18.034836+0000","last_clean":"2024-04-25T19:19:18.034836+0000","last_became_active":"2024-04-25T19:16:17.801451+0000","last_became_peered":"2024-04-25T19:16:17.801451+0000","last_unstale":"2024-04-25T19:19:18.034836+0000","last_undegraded":"2024-04-25T19:19:18.034836+0000","last_fullsized":"2024-04-25T19:19:18.034836+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:41:52.528120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"21'3","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.550543+0000","last_change":"2024-04-25T19:15:18.420711+0000","last_active":"2024-04-25T19:16:17.550543+0000","last_peered":"2024-04-25T19:16:17.550543+0000","last_clean":"2024-04-25T19:16:17.550543+0000","last_became_active":"2024-04-25T19:15:18.420362+0000","last_became_peered":"2024-04-25T19:15:18.420362+0000","last_unstale":"2024-04-25T19:16:17.550543+0000","last_undegraded":"2024-04-25T19:16:17.550543+0000","last_fullsized":"2024-04-25T19:16:17.550543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:48.245193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'147","reported_seq":164,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.906678+0000","last_change":"2024-04-25T19:15:19.424356+0000","last_active":"2024-04-25T19:19:17.906678+0000","last_peered":"2024-04-25T19:19:17.906678+0000","last_clean":"2024-04-25T19:19:17.906678+0000","last_became_active":"2024-04-25T19:15:19.423556+0000","last_became_peered":"2024-04-25T19:15:19.423556+0000","last_unstale":"2024-04-25T19:19:17.906678+0000","last_undegraded":"2024-04-25T19:19:17.906678+0000","last_fullsized":"2024-04-25T19:19:17.906678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:19:35.427261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'7","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:42.864151+0000","last_change":"2024-04-25T19:15:18.419250+0000","last_active":"2024-04-25T19:17:42.864151+0000","last_peered":"2024-04-25T19:17:42.864151+0000","last_clean":"2024-04-25T19:17:42.864151+0000","last_became_active":"2024-04-25T19:15:18.418927+0000","last_became_peered":"2024-04-25T19:15:18.418927+0000","last_unstale":"2024-04-25T19:17:42.864151+0000","last_undegraded":"2024-04-25T19:17:42.864151+0000","last_fullsized":"2024-04-25T19:17:42.864151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:26.012965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'219","reported_seq":236,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.335276+0000","last_change":"2024-04-25T19:15:19.430117+0000","last_active":"2024-04-25T19:19:18.335276+0000","last_peered":"2024-04-25T19:19:18.335276+0000","last_clean":"2024-04-25T19:19:18.335276+0000","last_became_active":"2024-04-25T19:15:19.428985+0000","last_became_peered":"2024-04-25T19:15:19.428985+0000","last_unstale":"2024-04-25T19:19:18.335276+0000","last_undegraded":"2024-04-25T19:19:18.335276+0000","last_fullsized":"2024-04-25T19:19:18.335276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:36:29.069541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"21'4","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.550765+0000","last_change":"2024-04-25T19:15:18.422931+0000","last_active":"2024-04-25T19:16:17.550765+0000","last_peered":"2024-04-25T19:16:17.550765+0000","last_clean":"2024-04-25T19:16:17.550765+0000","last_became_active":"2024-04-25T19:15:18.422610+0000","last_became_peered":"2024-04-25T19:15:18.422610+0000","last_unstale":"2024-04-25T19:16:17.550765+0000","last_undegraded":"2024-04-25T19:16:17.550765+0000","last_fullsized":"2024-04-25T19:16:17.550765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:18.489484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'132","reported_seq":149,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.080062+0000","last_change":"2024-04-25T19:15:19.416109+0000","last_active":"2024-04-25T19:19:18.080062+0000","last_peered":"2024-04-25T19:19:18.080062+0000","last_clean":"2024-04-25T19:19:18.080062+0000","last_became_active":"2024-04-25T19:15:19.415734+0000","last_became_peered":"2024-04-25T19:15:19.415734+0000","last_unstale":"2024-04-25T19:19:18.080062+0000","last_undegraded":"2024-04-25T19:19:18.080062+0000","last_fullsized":"2024-04-25T19:19:18.080062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:57:31.406303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'10","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:16.549699+0000","last_change":"2024-04-25T19:15:18.428677+0000","last_active":"2024-04-25T19:19:16.549699+0000","last_peered":"2024-04-25T19:19:16.549699+0000","last_clean":"2024-04-25T19:19:16.549699+0000","last_became_active":"2024-04-25T19:15:18.428432+0000","last_became_peered":"2024-04-25T19:15:18.428432+0000","last_unstale":"2024-04-25T19:19:16.549699+0000","last_undegraded":"2024-04-25T19:19:16.549699+0000","last_fullsized":"2024-04-25T19:19:16.549699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:38.478595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'138","reported_seq":155,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.036477+0000","last_change":"2024-04-25T19:15:19.428245+0000","last_active":"2024-04-25T19:19:18.036477+0000","last_peered":"2024-04-25T19:19:18.036477+0000","last_clean":"2024-04-25T19:19:18.036477+0000","last_became_active":"2024-04-25T19:15:19.427941+0000","last_became_peered":"2024-04-25T19:15:19.427941+0000","last_unstale":"2024-04-25T19:19:18.036477+0000","last_undegraded":"2024-04-25T19:19:18.036477+0000","last_fullsized":"2024-04-25T19:19:18.036477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:09.947923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'99","reported_seq":118,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:19.439568+0000","last_change":"2024-04-25T19:15:18.420881+0000","last_active":"2024-04-25T19:19:19.439568+0000","last_peered":"2024-04-25T19:19:19.439568+0000","last_clean":"2024-04-25T19:19:19.439568+0000","last_became_active":"2024-04-25T19:15:18.420303+0000","last_became_peered":"2024-04-25T19:15:18.420303+0000","last_unstale":"2024-04-25T19:19:19.439568+0000","last_undegraded":"2024-04-25T19:19:19.439568+0000","last_fullsized":"2024-04-25T19:19:19.439568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:39:24.356298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":216545,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'144","reported_seq":155,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.986213+0000","last_change":"2024-04-25T19:16:17.661342+0000","last_active":"2024-04-25T19:19:17.986213+0000","last_peered":"2024-04-25T19:19:17.986213+0000","last_clean":"2024-04-25T19:19:17.986213+0000","last_became_active":"2024-04-25T19:16:17.661023+0000","last_became_peered":"2024-04-25T19:16:17.661023+0000","last_unstale":"2024-04-25T19:19:17.986213+0000","last_undegraded":"2024-04-25T19:19:17.986213+0000","last_fullsized":"2024-04-25T19:19:17.986213+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:31:11.190902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.13","version":"21'9","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.800767+0000","last_change":"2024-04-25T19:15:18.421008+0000","last_active":"2024-04-25T19:16:17.800767+0000","last_peered":"2024-04-25T19:16:17.800767+0000","last_clean":"2024-04-25T19:16:17.800767+0000","last_became_active":"2024-04-25T19:15:18.420402+0000","last_became_peered":"2024-04-25T19:15:18.420402+0000","last_unstale":"2024-04-25T19:16:17.800767+0000","last_undegraded":"2024-04-25T19:16:17.800767+0000","last_fullsized":"2024-04-25T19:16:17.800767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:05:28.723958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'148","reported_seq":165,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.054624+0000","last_change":"2024-04-25T19:15:19.424805+0000","last_active":"2024-04-25T19:19:18.054624+0000","last_peered":"2024-04-25T19:19:18.054624+0000","last_clean":"2024-04-25T19:19:18.054624+0000","last_became_active":"2024-04-25T19:15:19.424512+0000","last_became_peered":"2024-04-25T19:15:19.424512+0000","last_unstale":"2024-04-25T19:19:18.054624+0000","last_undegraded":"2024-04-25T19:19:18.054624+0000","last_fullsized":"2024-04-25T19:19:18.054624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:50.423186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.550627+0000","last_change":"2024-04-25T19:15:18.426805+0000","last_active":"2024-04-25T19:16:17.550627+0000","last_peered":"2024-04-25T19:16:17.550627+0000","last_clean":"2024-04-25T19:16:17.550627+0000","last_became_active":"2024-04-25T19:15:18.426472+0000","last_became_peered":"2024-04-25T19:15:18.426472+0000","last_unstale":"2024-04-25T19:16:17.550627+0000","last_undegraded":"2024-04-25T19:16:17.550627+0000","last_fullsized":"2024-04-25T19:16:17.550627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:29.373363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'156","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.047258+0000","last_change":"2024-04-25T19:15:19.428780+0000","last_active":"2024-04-25T19:19:18.047258+0000","last_peered":"2024-04-25T19:19:18.047258+0000","last_clean":"2024-04-25T19:19:18.047258+0000","last_became_active":"2024-04-25T19:15:19.428450+0000","last_became_peered":"2024-04-25T19:15:19.428450+0000","last_unstale":"2024-04-25T19:19:18.047258+0000","last_undegraded":"2024-04-25T19:19:18.047258+0000","last_fullsized":"2024-04-25T19:19:18.047258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:01.203880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"21'6","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.552957+0000","last_change":"2024-04-25T19:15:18.417740+0000","last_active":"2024-04-25T19:16:17.552957+0000","last_peered":"2024-04-25T19:16:17.552957+0000","last_clean":"2024-04-25T19:16:17.552957+0000","last_became_active":"2024-04-25T19:15:18.417404+0000","last_became_peered":"2024-04-25T19:15:18.417404+0000","last_unstale":"2024-04-25T19:16:17.552957+0000","last_undegraded":"2024-04-25T19:16:17.552957+0000","last_fullsized":"2024-04-25T19:16:17.552957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:21.737283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1029,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'159","reported_seq":176,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.060341+0000","last_change":"2024-04-25T19:15:19.411214+0000","last_active":"2024-04-25T19:19:18.060341+0000","last_peered":"2024-04-25T19:19:18.060341+0000","last_clean":"2024-04-25T19:19:18.060341+0000","last_became_active":"2024-04-25T19:15:19.410814+0000","last_became_peered":"2024-04-25T19:15:19.410814+0000","last_unstale":"2024-04-25T19:19:18.060341+0000","last_undegraded":"2024-04-25T19:19:18.060341+0000","last_fullsized":"2024-04-25T19:19:18.060341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:42.430277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'18","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:37.867718+0000","last_change":"2024-04-25T19:15:18.422226+0000","last_active":"2024-04-25T19:17:37.867718+0000","last_peered":"2024-04-25T19:17:37.867718+0000","last_clean":"2024-04-25T19:17:37.867718+0000","last_became_active":"2024-04-25T19:15:18.421900+0000","last_became_peered":"2024-04-25T19:15:18.421900+0000","last_unstale":"2024-04-25T19:17:37.867718+0000","last_undegraded":"2024-04-25T19:17:37.867718+0000","last_fullsized":"2024-04-25T19:17:37.867718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:18.040114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'183","reported_seq":200,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.048026+0000","last_change":"2024-04-25T19:15:19.428885+0000","last_active":"2024-04-25T19:19:18.048026+0000","last_peered":"2024-04-25T19:19:18.048026+0000","last_clean":"2024-04-25T19:19:18.048026+0000","last_became_active":"2024-04-25T19:15:19.428563+0000","last_became_peered":"2024-04-25T19:15:19.428563+0000","last_unstale":"2024-04-25T19:19:18.048026+0000","last_undegraded":"2024-04-25T19:19:18.048026+0000","last_fullsized":"2024-04-25T19:19:18.048026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:43.469189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'2","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.800374+0000","last_change":"2024-04-25T19:15:18.422125+0000","last_active":"2024-04-25T19:16:17.800374+0000","last_peered":"2024-04-25T19:16:17.800374+0000","last_clean":"2024-04-25T19:16:17.800374+0000","last_became_active":"2024-04-25T19:15:18.421777+0000","last_became_peered":"2024-04-25T19:15:18.421777+0000","last_unstale":"2024-04-25T19:16:17.800374+0000","last_undegraded":"2024-04-25T19:16:17.800374+0000","last_fullsized":"2024-04-25T19:16:17.800374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:24.289010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'141","reported_seq":158,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.052159+0000","last_change":"2024-04-25T19:15:19.429675+0000","last_active":"2024-04-25T19:19:18.052159+0000","last_peered":"2024-04-25T19:19:18.052159+0000","last_clean":"2024-04-25T19:19:18.052159+0000","last_became_active":"2024-04-25T19:15:19.429316+0000","last_became_peered":"2024-04-25T19:15:19.429316+0000","last_unstale":"2024-04-25T19:19:18.052159+0000","last_undegraded":"2024-04-25T19:19:18.052159+0000","last_fullsized":"2024-04-25T19:19:18.052159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:09:55.521944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.552135+0000","last_change":"2024-04-25T19:15:18.418661+0000","last_active":"2024-04-25T19:16:17.552135+0000","last_peered":"2024-04-25T19:16:17.552135+0000","last_clean":"2024-04-25T19:16:17.552135+0000","last_became_active":"2024-04-25T19:15:18.418359+0000","last_became_peered":"2024-04-25T19:15:18.418359+0000","last_unstale":"2024-04-25T19:16:17.552135+0000","last_undegraded":"2024-04-25T19:16:17.552135+0000","last_fullsized":"2024-04-25T19:16:17.552135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:49:22.023652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'129","reported_seq":146,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.027304+0000","last_change":"2024-04-25T19:15:19.434853+0000","last_active":"2024-04-25T19:19:18.027304+0000","last_peered":"2024-04-25T19:19:18.027304+0000","last_clean":"2024-04-25T19:19:18.027304+0000","last_became_active":"2024-04-25T19:15:19.434508+0000","last_became_peered":"2024-04-25T19:15:19.434508+0000","last_unstale":"2024-04-25T19:19:18.027304+0000","last_undegraded":"2024-04-25T19:19:18.027304+0000","last_fullsized":"2024-04-25T19:19:18.027304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:52.965864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'3","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.552800+0000","last_change":"2024-04-25T19:15:18.428626+0000","last_active":"2024-04-25T19:16:17.552800+0000","last_peered":"2024-04-25T19:16:17.552800+0000","last_clean":"2024-04-25T19:16:17.552800+0000","last_became_active":"2024-04-25T19:15:18.428332+0000","last_became_peered":"2024-04-25T19:15:18.428332+0000","last_unstale":"2024-04-25T19:16:17.552800+0000","last_undegraded":"2024-04-25T19:16:17.552800+0000","last_fullsized":"2024-04-25T19:16:17.552800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:15.595305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'138","reported_seq":155,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.016419+0000","last_change":"2024-04-25T19:15:19.428754+0000","last_active":"2024-04-25T19:19:18.016419+0000","last_peered":"2024-04-25T19:19:18.016419+0000","last_clean":"2024-04-25T19:19:18.016419+0000","last_became_active":"2024-04-25T19:15:19.427992+0000","last_became_peered":"2024-04-25T19:15:19.427992+0000","last_unstale":"2024-04-25T19:19:18.016419+0000","last_undegraded":"2024-04-25T19:19:18.016419+0000","last_fullsized":"2024-04-25T19:19:18.016419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:43:48.443923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"21'6","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.551964+0000","last_change":"2024-04-25T19:15:18.419645+0000","last_active":"2024-04-25T19:16:17.551964+0000","last_peered":"2024-04-25T19:16:17.551964+0000","last_clean":"2024-04-25T19:16:17.551964+0000","last_became_active":"2024-04-25T19:15:18.419286+0000","last_became_peered":"2024-04-25T19:15:18.419286+0000","last_unstale":"2024-04-25T19:16:17.551964+0000","last_undegraded":"2024-04-25T19:16:17.551964+0000","last_fullsized":"2024-04-25T19:16:17.551964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:08.295208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1611,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'144","reported_seq":161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.964325+0000","last_change":"2024-04-25T19:15:19.432395+0000","last_active":"2024-04-25T19:19:17.964325+0000","last_peered":"2024-04-25T19:19:17.964325+0000","last_clean":"2024-04-25T19:19:17.964325+0000","last_became_active":"2024-04-25T19:15:19.431739+0000","last_became_peered":"2024-04-25T19:15:19.431739+0000","last_unstale":"2024-04-25T19:19:17.964325+0000","last_undegraded":"2024-04-25T19:19:17.964325+0000","last_fullsized":"2024-04-25T19:19:17.964325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:50:03.964030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":116,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:17.871195+0000","last_change":"2024-04-25T19:15:15.604825+0000","last_active":"2024-04-25T19:17:17.871195+0000","last_peered":"2024-04-25T19:17:17.871195+0000","last_clean":"2024-04-25T19:17:17.871195+0000","last_became_active":"2024-04-25T19:15:15.603730+0000","last_became_peered":"2024-04-25T19:15:15.603730+0000","last_unstale":"2024-04-25T19:17:17.871195+0000","last_undegraded":"2024-04-25T19:17:17.871195+0000","last_fullsized":"2024-04-25T19:17:17.871195+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:14.367890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:14.367890+0000","last_clean_scrub_stamp":"2024-04-25T19:15:14.367890+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:02.411899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'6","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:22.863164+0000","last_change":"2024-04-25T19:15:18.426331+0000","last_active":"2024-04-25T19:17:22.863164+0000","last_peered":"2024-04-25T19:17:22.863164+0000","last_clean":"2024-04-25T19:17:22.863164+0000","last_became_active":"2024-04-25T19:15:18.425993+0000","last_became_peered":"2024-04-25T19:15:18.425993+0000","last_unstale":"2024-04-25T19:17:22.863164+0000","last_undegraded":"2024-04-25T19:17:22.863164+0000","last_fullsized":"2024-04-25T19:17:22.863164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:17.440484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'150","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.096775+0000","last_change":"2024-04-25T19:15:19.417337+0000","last_active":"2024-04-25T19:19:18.096775+0000","last_peered":"2024-04-25T19:19:18.096775+0000","last_clean":"2024-04-25T19:19:18.096775+0000","last_became_active":"2024-04-25T19:15:19.416808+0000","last_became_peered":"2024-04-25T19:15:19.416808+0000","last_unstale":"2024-04-25T19:19:18.096775+0000","last_undegraded":"2024-04-25T19:19:18.096775+0000","last_fullsized":"2024-04-25T19:19:18.096775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:57.797904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"21'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:17.871167+0000","last_change":"2024-04-25T19:15:18.426358+0000","last_active":"2024-04-25T19:17:17.871167+0000","last_peered":"2024-04-25T19:17:17.871167+0000","last_clean":"2024-04-25T19:17:17.871167+0000","last_became_active":"2024-04-25T19:15:18.426037+0000","last_became_peered":"2024-04-25T19:15:18.426037+0000","last_unstale":"2024-04-25T19:17:17.871167+0000","last_undegraded":"2024-04-25T19:17:17.871167+0000","last_fullsized":"2024-04-25T19:17:17.871167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:26:28.945667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'126","reported_seq":144,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.100470+0000","last_change":"2024-04-25T19:15:19.426219+0000","last_active":"2024-04-25T19:19:18.100470+0000","last_peered":"2024-04-25T19:19:18.100470+0000","last_clean":"2024-04-25T19:19:18.100470+0000","last_became_active":"2024-04-25T19:15:19.425626+0000","last_became_peered":"2024-04-25T19:15:19.425626+0000","last_unstale":"2024-04-25T19:19:18.100470+0000","last_undegraded":"2024-04-25T19:19:18.100470+0000","last_fullsized":"2024-04-25T19:19:18.100470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:58.372454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'11","reported_seq":35,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:16.540140+0000","last_change":"2024-04-25T19:15:18.424560+0000","last_active":"2024-04-25T19:19:16.540140+0000","last_peered":"2024-04-25T19:19:16.540140+0000","last_clean":"2024-04-25T19:19:16.540140+0000","last_became_active":"2024-04-25T19:15:18.424179+0000","last_became_peered":"2024-04-25T19:15:18.424179+0000","last_unstale":"2024-04-25T19:19:16.540140+0000","last_undegraded":"2024-04-25T19:19:16.540140+0000","last_fullsized":"2024-04-25T19:19:16.540140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:06.184399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'123","reported_seq":141,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.076512+0000","last_change":"2024-04-25T19:15:19.415219+0000","last_active":"2024-04-25T19:19:18.076512+0000","last_peered":"2024-04-25T19:19:18.076512+0000","last_clean":"2024-04-25T19:19:18.076512+0000","last_became_active":"2024-04-25T19:15:19.414877+0000","last_became_peered":"2024-04-25T19:15:19.414877+0000","last_unstale":"2024-04-25T19:19:18.076512+0000","last_undegraded":"2024-04-25T19:19:18.076512+0000","last_fullsized":"2024-04-25T19:19:18.076512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:15.063419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'5","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:22.863125+0000","last_change":"2024-04-25T19:15:18.423402+0000","last_active":"2024-04-25T19:17:22.863125+0000","last_peered":"2024-04-25T19:17:22.863125+0000","last_clean":"2024-04-25T19:17:22.863125+0000","last_became_active":"2024-04-25T19:15:18.422330+0000","last_became_peered":"2024-04-25T19:15:18.422330+0000","last_unstale":"2024-04-25T19:17:22.863125+0000","last_undegraded":"2024-04-25T19:17:22.863125+0000","last_fullsized":"2024-04-25T19:17:22.863125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:39:17.512502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'138","reported_seq":156,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.097425+0000","last_change":"2024-04-25T19:15:19.421096+0000","last_active":"2024-04-25T19:19:18.097425+0000","last_peered":"2024-04-25T19:19:18.097425+0000","last_clean":"2024-04-25T19:19:18.097425+0000","last_became_active":"2024-04-25T19:15:19.420598+0000","last_became_peered":"2024-04-25T19:15:19.420598+0000","last_unstale":"2024-04-25T19:19:18.097425+0000","last_undegraded":"2024-04-25T19:19:18.097425+0000","last_fullsized":"2024-04-25T19:19:18.097425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:20.733888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:22.862361+0000","last_change":"2024-04-25T19:15:18.422979+0000","last_active":"2024-04-25T19:17:22.862361+0000","last_peered":"2024-04-25T19:17:22.862361+0000","last_clean":"2024-04-25T19:17:22.862361+0000","last_became_active":"2024-04-25T19:15:18.422621+0000","last_became_peered":"2024-04-25T19:15:18.422621+0000","last_unstale":"2024-04-25T19:17:22.862361+0000","last_undegraded":"2024-04-25T19:17:22.862361+0000","last_fullsized":"2024-04-25T19:17:22.862361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:47:56.160615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'138","reported_seq":156,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.095676+0000","last_change":"2024-04-25T19:15:19.426379+0000","last_active":"2024-04-25T19:19:18.095676+0000","last_peered":"2024-04-25T19:19:18.095676+0000","last_clean":"2024-04-25T19:19:18.095676+0000","last_became_active":"2024-04-25T19:15:19.425997+0000","last_became_peered":"2024-04-25T19:15:19.425997+0000","last_unstale":"2024-04-25T19:19:18.095676+0000","last_undegraded":"2024-04-25T19:19:18.095676+0000","last_fullsized":"2024-04-25T19:19:18.095676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:32.972274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"21'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:22.871729+0000","last_change":"2024-04-25T19:15:18.427748+0000","last_active":"2024-04-25T19:17:22.871729+0000","last_peered":"2024-04-25T19:17:22.871729+0000","last_clean":"2024-04-25T19:17:22.871729+0000","last_became_active":"2024-04-25T19:15:18.427374+0000","last_became_peered":"2024-04-25T19:15:18.427374+0000","last_unstale":"2024-04-25T19:17:22.871729+0000","last_undegraded":"2024-04-25T19:17:22.871729+0000","last_fullsized":"2024-04-25T19:17:22.871729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:47:41.469558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'135","reported_seq":153,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.083369+0000","last_change":"2024-04-25T19:15:19.422196+0000","last_active":"2024-04-25T19:19:18.083369+0000","last_peered":"2024-04-25T19:19:18.083369+0000","last_clean":"2024-04-25T19:19:18.083369+0000","last_became_active":"2024-04-25T19:15:19.421835+0000","last_became_peered":"2024-04-25T19:15:19.421835+0000","last_unstale":"2024-04-25T19:19:18.083369+0000","last_undegraded":"2024-04-25T19:19:18.083369+0000","last_fullsized":"2024-04-25T19:19:18.083369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:38.500369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'24","reported_seq":43,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:14.465958+0000","last_change":"2024-04-25T19:15:18.419086+0000","last_active":"2024-04-25T19:19:14.465958+0000","last_peered":"2024-04-25T19:19:14.465958+0000","last_clean":"2024-04-25T19:19:14.465958+0000","last_became_active":"2024-04-25T19:15:18.418775+0000","last_became_peered":"2024-04-25T19:15:18.418775+0000","last_unstale":"2024-04-25T19:19:14.465958+0000","last_undegraded":"2024-04-25T19:19:14.465958+0000","last_fullsized":"2024-04-25T19:19:14.465958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:20.074753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'138","reported_seq":155,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.072440+0000","last_change":"2024-04-25T19:15:19.432155+0000","last_active":"2024-04-25T19:19:18.072440+0000","last_peered":"2024-04-25T19:19:18.072440+0000","last_clean":"2024-04-25T19:19:18.072440+0000","last_became_active":"2024-04-25T19:15:19.431905+0000","last_became_peered":"2024-04-25T19:15:19.431905+0000","last_unstale":"2024-04-25T19:19:18.072440+0000","last_undegraded":"2024-04-25T19:19:18.072440+0000","last_fullsized":"2024-04-25T19:19:18.072440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:52.752469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'8","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:27.862195+0000","last_change":"2024-04-25T19:15:18.423144+0000","last_active":"2024-04-25T19:17:27.862195+0000","last_peered":"2024-04-25T19:17:27.862195+0000","last_clean":"2024-04-25T19:17:27.862195+0000","last_became_active":"2024-04-25T19:15:18.422838+0000","last_became_peered":"2024-04-25T19:15:18.422838+0000","last_unstale":"2024-04-25T19:17:27.862195+0000","last_undegraded":"2024-04-25T19:17:27.862195+0000","last_fullsized":"2024-04-25T19:17:27.862195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:28.026912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'156","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.014340+0000","last_change":"2024-04-25T19:15:19.420582+0000","last_active":"2024-04-25T19:19:18.014340+0000","last_peered":"2024-04-25T19:19:18.014340+0000","last_clean":"2024-04-25T19:19:18.014340+0000","last_became_active":"2024-04-25T19:15:19.420270+0000","last_became_peered":"2024-04-25T19:15:19.420270+0000","last_unstale":"2024-04-25T19:19:18.014340+0000","last_undegraded":"2024-04-25T19:19:18.014340+0000","last_fullsized":"2024-04-25T19:19:18.014340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:06.827819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"21'2","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.549438+0000","last_change":"2024-04-25T19:15:18.425469+0000","last_active":"2024-04-25T19:16:17.549438+0000","last_peered":"2024-04-25T19:16:17.549438+0000","last_clean":"2024-04-25T19:16:17.549438+0000","last_became_active":"2024-04-25T19:15:18.425084+0000","last_became_peered":"2024-04-25T19:15:18.425084+0000","last_unstale":"2024-04-25T19:16:17.549438+0000","last_undegraded":"2024-04-25T19:16:17.549438+0000","last_fullsized":"2024-04-25T19:16:17.549438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:14.363158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'132","reported_seq":149,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.814179+0000","last_change":"2024-04-25T19:15:19.425159+0000","last_active":"2024-04-25T19:19:17.814179+0000","last_peered":"2024-04-25T19:19:17.814179+0000","last_clean":"2024-04-25T19:19:17.814179+0000","last_became_active":"2024-04-25T19:15:19.424847+0000","last_became_peered":"2024-04-25T19:15:19.424847+0000","last_unstale":"2024-04-25T19:19:17.814179+0000","last_undegraded":"2024-04-25T19:19:17.814179+0000","last_fullsized":"2024-04-25T19:19:17.814179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:28.715286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"21'7","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.552420+0000","last_change":"2024-04-25T19:15:18.424645+0000","last_active":"2024-04-25T19:16:17.552420+0000","last_peered":"2024-04-25T19:16:17.552420+0000","last_clean":"2024-04-25T19:16:17.552420+0000","last_became_active":"2024-04-25T19:15:18.424241+0000","last_became_peered":"2024-04-25T19:15:18.424241+0000","last_unstale":"2024-04-25T19:16:17.552420+0000","last_undegraded":"2024-04-25T19:16:17.552420+0000","last_fullsized":"2024-04-25T19:16:17.552420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:29.373363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'144","reported_seq":161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.074330+0000","last_change":"2024-04-25T19:15:19.421095+0000","last_active":"2024-04-25T19:19:18.074330+0000","last_peered":"2024-04-25T19:19:18.074330+0000","last_clean":"2024-04-25T19:19:18.074330+0000","last_became_active":"2024-04-25T19:15:19.420795+0000","last_became_peered":"2024-04-25T19:15:19.420795+0000","last_unstale":"2024-04-25T19:19:18.074330+0000","last_undegraded":"2024-04-25T19:19:18.074330+0000","last_fullsized":"2024-04-25T19:19:18.074330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:13.061506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'3","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.552520+0000","last_change":"2024-04-25T19:15:18.423841+0000","last_active":"2024-04-25T19:16:17.552520+0000","last_peered":"2024-04-25T19:16:17.552520+0000","last_clean":"2024-04-25T19:16:17.552520+0000","last_became_active":"2024-04-25T19:15:18.423500+0000","last_became_peered":"2024-04-25T19:15:18.423500+0000","last_unstale":"2024-04-25T19:16:17.552520+0000","last_undegraded":"2024-04-25T19:16:17.552520+0000","last_fullsized":"2024-04-25T19:16:17.552520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:58.933085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'135","reported_seq":152,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.018424+0000","last_change":"2024-04-25T19:15:19.416232+0000","last_active":"2024-04-25T19:19:18.018424+0000","last_peered":"2024-04-25T19:19:18.018424+0000","last_clean":"2024-04-25T19:19:18.018424+0000","last_became_active":"2024-04-25T19:15:19.415828+0000","last_became_peered":"2024-04-25T19:15:19.415828+0000","last_unstale":"2024-04-25T19:19:18.018424+0000","last_undegraded":"2024-04-25T19:19:18.018424+0000","last_fullsized":"2024-04-25T19:19:18.018424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:59.751840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:32.872674+0000","last_change":"2024-04-25T19:15:18.423918+0000","last_active":"2024-04-25T19:17:32.872674+0000","last_peered":"2024-04-25T19:17:32.872674+0000","last_clean":"2024-04-25T19:17:32.872674+0000","last_became_active":"2024-04-25T19:15:18.423588+0000","last_became_peered":"2024-04-25T19:15:18.423588+0000","last_unstale":"2024-04-25T19:17:32.872674+0000","last_undegraded":"2024-04-25T19:17:32.872674+0000","last_fullsized":"2024-04-25T19:17:32.872674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:16.378285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'123","reported_seq":140,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.998130+0000","last_change":"2024-04-25T19:15:19.422984+0000","last_active":"2024-04-25T19:19:17.998130+0000","last_peered":"2024-04-25T19:19:17.998130+0000","last_clean":"2024-04-25T19:19:17.998130+0000","last_became_active":"2024-04-25T19:15:19.422651+0000","last_became_peered":"2024-04-25T19:15:19.422651+0000","last_unstale":"2024-04-25T19:19:17.998130+0000","last_undegraded":"2024-04-25T19:19:17.998130+0000","last_fullsized":"2024-04-25T19:19:17.998130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:54.984914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'168","reported_seq":185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.068520+0000","last_change":"2024-04-25T19:15:19.423982+0000","last_active":"2024-04-25T19:19:18.068520+0000","last_peered":"2024-04-25T19:19:18.068520+0000","last_clean":"2024-04-25T19:19:18.068520+0000","last_became_active":"2024-04-25T19:15:19.423341+0000","last_became_peered":"2024-04-25T19:15:19.423341+0000","last_unstale":"2024-04-25T19:19:18.068520+0000","last_undegraded":"2024-04-25T19:19:18.068520+0000","last_fullsized":"2024-04-25T19:19:18.068520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:51.808352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.549697+0000","last_change":"2024-04-25T19:15:18.424455+0000","last_active":"2024-04-25T19:16:17.549697+0000","last_peered":"2024-04-25T19:16:17.549697+0000","last_clean":"2024-04-25T19:16:17.549697+0000","last_became_active":"2024-04-25T19:15:18.424119+0000","last_became_peered":"2024-04-25T19:15:18.424119+0000","last_unstale":"2024-04-25T19:16:17.549697+0000","last_undegraded":"2024-04-25T19:16:17.549697+0000","last_fullsized":"2024-04-25T19:16:17.549697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:43.135567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'132","reported_seq":149,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.023065+0000","last_change":"2024-04-25T19:15:19.424064+0000","last_active":"2024-04-25T19:19:18.023065+0000","last_peered":"2024-04-25T19:19:18.023065+0000","last_clean":"2024-04-25T19:19:18.023065+0000","last_became_active":"2024-04-25T19:15:19.423187+0000","last_became_peered":"2024-04-25T19:15:19.423187+0000","last_unstale":"2024-04-25T19:19:18.023065+0000","last_undegraded":"2024-04-25T19:19:18.023065+0000","last_fullsized":"2024-04-25T19:19:18.023065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:43:31.590700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"23'13","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:47.874568+0000","last_change":"2024-04-25T19:15:18.423151+0000","last_active":"2024-04-25T19:17:47.874568+0000","last_peered":"2024-04-25T19:17:47.874568+0000","last_clean":"2024-04-25T19:17:47.874568+0000","last_became_active":"2024-04-25T19:15:18.422831+0000","last_became_peered":"2024-04-25T19:15:18.422831+0000","last_unstale":"2024-04-25T19:17:47.874568+0000","last_undegraded":"2024-04-25T19:17:47.874568+0000","last_fullsized":"2024-04-25T19:17:47.874568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:43.244963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'129","reported_seq":147,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.095343+0000","last_change":"2024-04-25T19:15:19.416717+0000","last_active":"2024-04-25T19:19:18.095343+0000","last_peered":"2024-04-25T19:19:18.095343+0000","last_clean":"2024-04-25T19:19:18.095343+0000","last_became_active":"2024-04-25T19:15:19.416163+0000","last_became_peered":"2024-04-25T19:15:19.416163+0000","last_unstale":"2024-04-25T19:19:18.095343+0000","last_undegraded":"2024-04-25T19:19:18.095343+0000","last_fullsized":"2024-04-25T19:19:18.095343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:46.955159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:17.863896+0000","last_change":"2024-04-25T19:15:18.422875+0000","last_active":"2024-04-25T19:17:17.863896+0000","last_peered":"2024-04-25T19:17:17.863896+0000","last_clean":"2024-04-25T19:17:17.863896+0000","last_became_active":"2024-04-25T19:15:18.422489+0000","last_became_peered":"2024-04-25T19:15:18.422489+0000","last_unstale":"2024-04-25T19:17:17.863896+0000","last_undegraded":"2024-04-25T19:17:17.863896+0000","last_fullsized":"2024-04-25T19:17:17.863896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:30.602500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'147","reported_seq":165,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.080832+0000","last_change":"2024-04-25T19:15:19.416987+0000","last_active":"2024-04-25T19:19:18.080832+0000","last_peered":"2024-04-25T19:19:18.080832+0000","last_clean":"2024-04-25T19:19:18.080832+0000","last_became_active":"2024-04-25T19:15:19.414173+0000","last_became_peered":"2024-04-25T19:15:19.414173+0000","last_unstale":"2024-04-25T19:19:18.080832+0000","last_undegraded":"2024-04-25T19:19:18.080832+0000","last_fullsized":"2024-04-25T19:19:18.080832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:32:43.188653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'13","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:22.862662+0000","last_change":"2024-04-25T19:15:18.423867+0000","last_active":"2024-04-25T19:17:22.862662+0000","last_peered":"2024-04-25T19:17:22.862662+0000","last_clean":"2024-04-25T19:17:22.862662+0000","last_became_active":"2024-04-25T19:15:18.423532+0000","last_became_peered":"2024-04-25T19:15:18.423532+0000","last_unstale":"2024-04-25T19:17:22.862662+0000","last_undegraded":"2024-04-25T19:17:22.862662+0000","last_fullsized":"2024-04-25T19:17:22.862662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:04.531338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9217,"ondisk_log_size":9217,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":224795,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":344,"num_write_kb":333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":696320,"data_stored":449590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64838,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574903,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39832,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332008,"statfs":{"total":96636764160,"available":96595976192,"internally_reserved":0,"allocated":344064,"data_stored":680704,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574903,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35808,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336032,"statfs":{"total":96636764160,"available":96600096768,"internally_reserved":0,"allocated":483328,"data_stored":305571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39780,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332060,"statfs":{"total":96636764160,"available":96596029440,"internally_reserved":0,"allocated":290816,"data_stored":92168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574903,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39756,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332084,"statfs":{"total":96636764160,"available":96596054016,"internally_reserved":0,"allocated":266240,"data_stored":89566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574903,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39976,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331864,"statfs":{"total":96636764160,"available":96595828736,"internally_reserved":0,"allocated":491520,"data_stored":306110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35608,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336232,"statfs":{"total":96636764160,"available":96600301568,"internally_reserved":0,"allocated":278528,"data_stored":90203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35600,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336240,"statfs":{"total":96636764160,"available":96600309760,"internally_reserved":0,"allocated":270336,"data_stored":89327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574903,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39824,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332016,"statfs":{"total":96636764160,"available":96595984384,"internally_reserved":0,"allocated":335872,"data_stored":679037,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":249856,"data_stored":218315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":217776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T19:19:34.271 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T19:19:34.271 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T19:19:34.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.414+0000 7f0eba8f9640 1 Processor -- start 2024-04-25T19:19:34.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.414+0000 7f0eba8f9640 1 -- start start 2024-04-25T19:19:34.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.415+0000 7f0eba8f9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f0eb406de20 0x7f0eb406d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:34.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.415+0000 7f0eba8f9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0eb4156f10 0x7f0eb415d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:34.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.415+0000 7f0eba8f9640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0eb406e310 con 0x7f0eb40cc9e0 2024-04-25T19:19:34.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.415+0000 7f0eba8f9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f0eb415d2f0 con 0x7f0eb4156f10 2024-04-25T19:19:34.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.415+0000 7f0eba8f9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f0eb415dcc0 con 0x7f0eb406de20 2024-04-25T19:19:34.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.415+0000 7f0eb8e96640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0eb4156f10 0x7f0eb415d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:34.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.415+0000 7f0eb8e96640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0eb4156f10 0x7f0eb415d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50590/0 (socket says 172.21.15.76:50590) 2024-04-25T19:19:34.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.415+0000 7f0eb8e96640 1 -- 172.21.15.76:0/1540802043 learned_addr learned my addr 172.21.15.76:0/1540802043 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:34.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.415+0000 7f0eb3fff640 1 --2- 172.21.15.76:0/1540802043 >> v2:172.21.15.105:3300/0 conn(0x7f0eb406de20 0x7f0eb406d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:34.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.415+0000 7f0eb8e96640 1 -- 172.21.15.76:0/1540802043 >> v2:172.21.15.105:3300/0 conn(0x7f0eb406de20 msgr2=0x7f0eb406d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:34.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.415+0000 7f0eb8e96640 1 --2- 172.21.15.76:0/1540802043 >> v2:172.21.15.105:3300/0 conn(0x7f0eb406de20 0x7f0eb406d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.415+0000 7f0eb8e96640 1 -- 172.21.15.76:0/1540802043 >> v1:172.21.15.105:6789/0 conn(0x7f0eb40cc9e0 legacy=0x7f0eb40cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:34.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.415+0000 7f0eb8e96640 1 -- 172.21.15.76:0/1540802043 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb415ccc0 con 0x7f0eb4156f10 2024-04-25T19:19:34.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.416+0000 7f0eb8e96640 1 --2- 172.21.15.76:0/1540802043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0eb4156f10 0x7f0eb415d780 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40694c0 tx=0x7f0eac02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd3c0d57ae4650e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:34.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.416+0000 7f0eb2ffd640 1 -- 172.21.15.76:0/1540802043 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac036070 con 0x7f0eb4156f10 2024-04-25T19:19:34.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.416+0000 7f0eb2ffd640 1 -- 172.21.15.76:0/1540802043 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eac031600 con 0x7f0eb4156f10 2024-04-25T19:19:34.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.416+0000 7f0eb2ffd640 1 -- 172.21.15.76:0/1540802043 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac0319d0 con 0x7f0eb4156f10 2024-04-25T19:19:34.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.417+0000 7f0eba8f9640 1 -- 172.21.15.76:0/1540802043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0eb4156f10 msgr2=0x7f0eb415d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:34.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.417+0000 7f0eba8f9640 1 --2- 172.21.15.76:0/1540802043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0eb4156f10 0x7f0eb415d780 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40694c0 tx=0x7f0eac02a660 comp rx=0 tx=0).stop 2024-04-25T19:19:34.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.417+0000 7f0eba8f9640 1 -- 172.21.15.76:0/1540802043 shutdown_connections 2024-04-25T19:19:34.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.417+0000 7f0eba8f9640 1 --2- 172.21.15.76:0/1540802043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0eb4156f10 0x7f0eb415d780 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.417+0000 7f0eba8f9640 1 --2- 172.21.15.76:0/1540802043 >> v2:172.21.15.105:3300/0 conn(0x7f0eb406de20 0x7f0eb406d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.417+0000 7f0eba8f9640 1 -- 172.21.15.76:0/1540802043 >> 172.21.15.76:0/1540802043 conn(0x7f0eb40684e0 msgr2=0x7f0eb40688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:34.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.417+0000 7f0eba8f9640 1 -- 172.21.15.76:0/1540802043 shutdown_connections 2024-04-25T19:19:34.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.417+0000 7f0eba8f9640 1 -- 172.21.15.76:0/1540802043 wait complete. 2024-04-25T19:19:34.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.419+0000 7f0eba8f9640 1 Processor -- start 2024-04-25T19:19:34.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.419+0000 7f0eba8f9640 1 -- start start 2024-04-25T19:19:34.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.420+0000 7f0eba8f9640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0eb406de20 0x7f0eb41f6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:34.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.420+0000 7f0eba8f9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eb40cc9e0 0x7f0eb40cf070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:34.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.420+0000 7f0eba8f9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0eb4156f10 0x7f0eb4149b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:34.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.420+0000 7f0eba8f9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb415cf80 con 0x7f0eb4156f10 2024-04-25T19:19:34.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.420+0000 7f0eba8f9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb40cb980 con 0x7f0eb40cc9e0 2024-04-25T19:19:34.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.420+0000 7f0eba8f9640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f0eb4157320 con 0x7f0eb406de20 2024-04-25T19:19:34.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.420+0000 7f0eb8e96640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0eb4156f10 0x7f0eb4149b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:34.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.420+0000 7f0eb8e96640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0eb4156f10 0x7f0eb4149b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50604/0 (socket says 172.21.15.76:50604) 2024-04-25T19:19:34.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.420+0000 7f0eb8e96640 1 -- 172.21.15.76:0/1349843369 learned_addr learned my addr 172.21.15.76:0/1349843369 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:34.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.420+0000 7f0eb3fff640 1 --2- 172.21.15.76:0/1349843369 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0eb406de20 0x7f0eb41f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:34.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.420+0000 7f0eb37fe640 1 --2- 172.21.15.76:0/1349843369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eb40cc9e0 0x7f0eb40cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:34.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.420+0000 7f0eb8e96640 1 -- 172.21.15.76:0/1349843369 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0eb406de20 msgr2=0x7f0eb41f6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:34.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.421+0000 7f0eb8e96640 1 --2- 172.21.15.76:0/1349843369 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0eb406de20 0x7f0eb41f6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.421+0000 7f0eb8e96640 1 -- 172.21.15.76:0/1349843369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eb40cc9e0 msgr2=0x7f0eb40cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:34.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.421+0000 7f0eb8e96640 1 --2- 172.21.15.76:0/1349843369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eb40cc9e0 0x7f0eb40cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.421+0000 7f0eb8e96640 1 -- 172.21.15.76:0/1349843369 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb41f70f0 con 0x7f0eb4156f10 2024-04-25T19:19:34.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.421+0000 7f0eb8e96640 1 --2- 172.21.15.76:0/1349843369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0eb4156f10 0x7f0eb4149b40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0eac02ac50 tx=0x7f0eac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:34.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.421+0000 7f0eb0ff9640 1 -- 172.21.15.76:0/1349843369 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac036050 con 0x7f0eb4156f10 2024-04-25T19:19:34.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.421+0000 7f0eb0ff9640 1 -- 172.21.15.76:0/1349843369 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eac042070 con 0x7f0eb4156f10 2024-04-25T19:19:34.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.421+0000 7f0eb0ff9640 1 -- 172.21.15.76:0/1349843369 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac009510 con 0x7f0eb4156f10 2024-04-25T19:19:34.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.421+0000 7f0eba8f9640 1 -- 172.21.15.76:0/1349843369 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb41f3b50 con 0x7f0eb4156f10 2024-04-25T19:19:34.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.421+0000 7f0eba8f9640 1 -- 172.21.15.76:0/1349843369 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb414fd30 con 0x7f0eb4156f10 2024-04-25T19:19:34.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.422+0000 7f0eba8f9640 1 -- 172.21.15.76:0/1349843369 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eb406a390 con 0x7f0eb4156f10 2024-04-25T19:19:34.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.423+0000 7f0eb0ff9640 1 -- 172.21.15.76:0/1349843369 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0eac007bd0 con 0x7f0eb4156f10 2024-04-25T19:19:34.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.423+0000 7f0eb0ff9640 1 --2- 172.21.15.76:0/1349843369 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0ea00744e0 0x7f0ea00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:34.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.423+0000 7f0eb0ff9640 1 -- 172.21.15.76:0/1349843369 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f0eac0b5af0 con 0x7f0eb4156f10 2024-04-25T19:19:34.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.423+0000 7f0eb3fff640 1 --2- 172.21.15.76:0/1349843369 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0ea00744e0 0x7f0ea00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:34.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.424+0000 7f0eb3fff640 1 --2- 172.21.15.76:0/1349843369 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0ea00744e0 0x7f0ea00769f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4006fd0 tx=0x7f0ea4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:34.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.429+0000 7f0eb0ff9640 1 -- 172.21.15.76:0/1349843369 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0eac04b020 con 0x7f0eb4156f10 2024-04-25T19:19:34.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.602+0000 7f0eba8f9640 1 -- 172.21.15.76:0/1349843369 --> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0eb4068a80 con 0x7f0ea00744e0 2024-04-25T19:19:34.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.609+0000 7f0eb0ff9640 1 -- 172.21.15.76:0/1349843369 <== mgr.4109 v2:172.21.15.105:6832/2585775531 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+253023 (secure 0 0 0) 0x7f0eb4068a80 con 0x7f0ea00744e0 2024-04-25T19:19:34.613 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:19:34.614 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-04-25T19:19:34.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.612+0000 7f0eba8f9640 1 -- 172.21.15.76:0/1349843369 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0ea00744e0 msgr2=0x7f0ea00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:34.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.612+0000 7f0eba8f9640 1 --2- 172.21.15.76:0/1349843369 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0ea00744e0 0x7f0ea00769f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4006fd0 tx=0x7f0ea4008040 comp rx=0 tx=0).stop 2024-04-25T19:19:34.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.612+0000 7f0eba8f9640 1 -- 172.21.15.76:0/1349843369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0eb4156f10 msgr2=0x7f0eb4149b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:34.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.612+0000 7f0eba8f9640 1 --2- 172.21.15.76:0/1349843369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0eb4156f10 0x7f0eb4149b40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0eac02ac50 tx=0x7f0eac004a40 comp rx=0 tx=0).stop 2024-04-25T19:19:34.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.612+0000 7f0eba8f9640 1 -- 172.21.15.76:0/1349843369 shutdown_connections 2024-04-25T19:19:34.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.612+0000 7f0eba8f9640 1 --2- 172.21.15.76:0/1349843369 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f0ea00744e0 0x7f0ea00769f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.612+0000 7f0eba8f9640 1 --2- 172.21.15.76:0/1349843369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0eb4156f10 0x7f0eb4149b40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.612+0000 7f0eba8f9640 1 --2- 172.21.15.76:0/1349843369 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eb40cc9e0 0x7f0eb40cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.612+0000 7f0eba8f9640 1 --2- 172.21.15.76:0/1349843369 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f0eb406de20 0x7f0eb41f6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.612+0000 7f0eba8f9640 1 -- 172.21.15.76:0/1349843369 >> 172.21.15.76:0/1349843369 conn(0x7f0eb40684e0 msgr2=0x7f0eb40cce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:34.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.612+0000 7f0eba8f9640 1 -- 172.21.15.76:0/1349843369 shutdown_connections 2024-04-25T19:19:34.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.612+0000 7f0eba8f9640 1 -- 172.21.15.76:0/1349843369 wait complete. 2024-04-25T19:19:34.625 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-04-25T19:19:33.896521+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":815163,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":70,"num_read_kb":59,"num_write":9661,"num_write_kb":25167785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9595,"ondisk_log_size":9595,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":306184,"kb_used_data":2696,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754668536,"statfs":{"total":773094113280,"available":772780580864,"internally_reserved":0,"allocated":2760704,"data_stored":2332686,"data_compressed":64838,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":310693424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1413477513,"num_objects":-337,"num_object_clones":0,"num_object_copies":-674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":340,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004062"},"pg_stats":[{"pgid":"3.3f","version":"23'138","reported_seq":156,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.068380+0000","last_change":"2024-04-25T19:15:19.433569+0000","last_active":"2024-04-25T19:19:18.068380+0000","last_peered":"2024-04-25T19:19:18.068380+0000","last_clean":"2024-04-25T19:19:18.068380+0000","last_became_active":"2024-04-25T19:15:19.431332+0000","last_became_peered":"2024-04-25T19:15:19.431332+0000","last_unstale":"2024-04-25T19:19:18.068380+0000","last_undegraded":"2024-04-25T19:19:18.068380+0000","last_fullsized":"2024-04-25T19:19:18.068380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:57:01.213834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'126","reported_seq":144,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.090933+0000","last_change":"2024-04-25T19:15:19.418695+0000","last_active":"2024-04-25T19:19:18.090933+0000","last_peered":"2024-04-25T19:19:18.090933+0000","last_clean":"2024-04-25T19:19:18.090933+0000","last_became_active":"2024-04-25T19:15:19.418235+0000","last_became_peered":"2024-04-25T19:15:19.418235+0000","last_unstale":"2024-04-25T19:19:18.090933+0000","last_undegraded":"2024-04-25T19:19:18.090933+0000","last_fullsized":"2024-04-25T19:19:18.090933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:30.516472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'144","reported_seq":162,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.024866+0000","last_change":"2024-04-25T19:15:19.416983+0000","last_active":"2024-04-25T19:19:18.024866+0000","last_peered":"2024-04-25T19:19:18.024866+0000","last_clean":"2024-04-25T19:19:18.024866+0000","last_became_active":"2024-04-25T19:15:19.416443+0000","last_became_peered":"2024-04-25T19:15:19.416443+0000","last_unstale":"2024-04-25T19:19:18.024866+0000","last_undegraded":"2024-04-25T19:19:18.024866+0000","last_fullsized":"2024-04-25T19:19:18.024866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:02.794164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'165","reported_seq":183,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.098994+0000","last_change":"2024-04-25T19:15:19.429058+0000","last_active":"2024-04-25T19:19:18.098994+0000","last_peered":"2024-04-25T19:19:18.098994+0000","last_clean":"2024-04-25T19:19:18.098994+0000","last_became_active":"2024-04-25T19:15:19.428419+0000","last_became_peered":"2024-04-25T19:15:19.428419+0000","last_unstale":"2024-04-25T19:19:18.098994+0000","last_undegraded":"2024-04-25T19:19:18.098994+0000","last_fullsized":"2024-04-25T19:19:18.098994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:23:56.817616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'144","reported_seq":161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.073061+0000","last_change":"2024-04-25T19:15:19.427615+0000","last_active":"2024-04-25T19:19:18.073061+0000","last_peered":"2024-04-25T19:19:18.073061+0000","last_clean":"2024-04-25T19:19:18.073061+0000","last_became_active":"2024-04-25T19:15:19.426968+0000","last_became_peered":"2024-04-25T19:15:19.426968+0000","last_unstale":"2024-04-25T19:19:18.073061+0000","last_undegraded":"2024-04-25T19:19:18.073061+0000","last_fullsized":"2024-04-25T19:19:18.073061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:42:00.686712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'129","reported_seq":146,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.070490+0000","last_change":"2024-04-25T19:15:19.417602+0000","last_active":"2024-04-25T19:19:18.070490+0000","last_peered":"2024-04-25T19:19:18.070490+0000","last_clean":"2024-04-25T19:19:18.070490+0000","last_became_active":"2024-04-25T19:15:19.417292+0000","last_became_peered":"2024-04-25T19:15:19.417292+0000","last_unstale":"2024-04-25T19:19:18.070490+0000","last_undegraded":"2024-04-25T19:19:18.070490+0000","last_fullsized":"2024-04-25T19:19:18.070490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:37:43.583809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'159","reported_seq":176,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.041488+0000","last_change":"2024-04-25T19:15:19.428180+0000","last_active":"2024-04-25T19:19:18.041488+0000","last_peered":"2024-04-25T19:19:18.041488+0000","last_clean":"2024-04-25T19:19:18.041488+0000","last_became_active":"2024-04-25T19:15:19.427852+0000","last_became_peered":"2024-04-25T19:15:19.427852+0000","last_unstale":"2024-04-25T19:19:18.041488+0000","last_undegraded":"2024-04-25T19:19:18.041488+0000","last_fullsized":"2024-04-25T19:19:18.041488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:51.300880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'177","reported_seq":194,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.083993+0000","last_change":"2024-04-25T19:15:19.418253+0000","last_active":"2024-04-25T19:19:18.083993+0000","last_peered":"2024-04-25T19:19:18.083993+0000","last_clean":"2024-04-25T19:19:18.083993+0000","last_became_active":"2024-04-25T19:15:19.417183+0000","last_became_peered":"2024-04-25T19:15:19.417183+0000","last_unstale":"2024-04-25T19:19:18.083993+0000","last_undegraded":"2024-04-25T19:19:18.083993+0000","last_fullsized":"2024-04-25T19:19:18.083993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:05:13.181646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'141","reported_seq":158,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.050350+0000","last_change":"2024-04-25T19:15:19.431671+0000","last_active":"2024-04-25T19:19:18.050350+0000","last_peered":"2024-04-25T19:19:18.050350+0000","last_clean":"2024-04-25T19:19:18.050350+0000","last_became_active":"2024-04-25T19:15:19.431240+0000","last_became_peered":"2024-04-25T19:15:19.431240+0000","last_unstale":"2024-04-25T19:19:18.050350+0000","last_undegraded":"2024-04-25T19:19:18.050350+0000","last_fullsized":"2024-04-25T19:19:18.050350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:22.759462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'141","reported_seq":163,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.058501+0000","last_change":"2024-04-25T19:16:17.555086+0000","last_active":"2024-04-25T19:19:18.058501+0000","last_peered":"2024-04-25T19:19:18.058501+0000","last_clean":"2024-04-25T19:19:18.058501+0000","last_became_active":"2024-04-25T19:16:17.554771+0000","last_became_peered":"2024-04-25T19:16:17.554771+0000","last_unstale":"2024-04-25T19:19:18.058501+0000","last_undegraded":"2024-04-25T19:19:18.058501+0000","last_fullsized":"2024-04-25T19:19:18.058501+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:22:35.638155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'183","reported_seq":200,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.036612+0000","last_change":"2024-04-25T19:15:19.417596+0000","last_active":"2024-04-25T19:19:18.036612+0000","last_peered":"2024-04-25T19:19:18.036612+0000","last_clean":"2024-04-25T19:19:18.036612+0000","last_became_active":"2024-04-25T19:15:19.417230+0000","last_became_peered":"2024-04-25T19:15:19.417230+0000","last_unstale":"2024-04-25T19:19:18.036612+0000","last_undegraded":"2024-04-25T19:19:18.036612+0000","last_fullsized":"2024-04-25T19:19:18.036612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:28:12.387651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'135","reported_seq":152,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.052108+0000","last_change":"2024-04-25T19:15:19.420564+0000","last_active":"2024-04-25T19:19:18.052108+0000","last_peered":"2024-04-25T19:19:18.052108+0000","last_clean":"2024-04-25T19:19:18.052108+0000","last_became_active":"2024-04-25T19:15:19.420044+0000","last_became_peered":"2024-04-25T19:15:19.420044+0000","last_unstale":"2024-04-25T19:19:18.052108+0000","last_undegraded":"2024-04-25T19:19:18.052108+0000","last_fullsized":"2024-04-25T19:19:18.052108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:04.066030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'141","reported_seq":158,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.030803+0000","last_change":"2024-04-25T19:15:19.415500+0000","last_active":"2024-04-25T19:19:18.030803+0000","last_peered":"2024-04-25T19:19:18.030803+0000","last_clean":"2024-04-25T19:19:18.030803+0000","last_became_active":"2024-04-25T19:15:19.414007+0000","last_became_peered":"2024-04-25T19:15:19.414007+0000","last_unstale":"2024-04-25T19:19:18.030803+0000","last_undegraded":"2024-04-25T19:19:18.030803+0000","last_fullsized":"2024-04-25T19:19:18.030803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:56:12.320555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'141","reported_seq":158,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.052414+0000","last_change":"2024-04-25T19:15:19.423884+0000","last_active":"2024-04-25T19:19:18.052414+0000","last_peered":"2024-04-25T19:19:18.052414+0000","last_clean":"2024-04-25T19:19:18.052414+0000","last_became_active":"2024-04-25T19:15:19.422575+0000","last_became_peered":"2024-04-25T19:15:19.422575+0000","last_unstale":"2024-04-25T19:19:18.052414+0000","last_undegraded":"2024-04-25T19:19:18.052414+0000","last_fullsized":"2024-04-25T19:19:18.052414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:30.988950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'144","reported_seq":161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.064766+0000","last_change":"2024-04-25T19:15:19.424267+0000","last_active":"2024-04-25T19:19:18.064766+0000","last_peered":"2024-04-25T19:19:18.064766+0000","last_clean":"2024-04-25T19:19:18.064766+0000","last_became_active":"2024-04-25T19:15:19.422953+0000","last_became_peered":"2024-04-25T19:15:19.422953+0000","last_unstale":"2024-04-25T19:19:18.064766+0000","last_undegraded":"2024-04-25T19:19:18.064766+0000","last_fullsized":"2024-04-25T19:19:18.064766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:02:42.823233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'138","reported_seq":149,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.018289+0000","last_change":"2024-04-25T19:16:17.801933+0000","last_active":"2024-04-25T19:19:18.018289+0000","last_peered":"2024-04-25T19:19:18.018289+0000","last_clean":"2024-04-25T19:19:18.018289+0000","last_became_active":"2024-04-25T19:16:17.801500+0000","last_became_peered":"2024-04-25T19:16:17.801500+0000","last_unstale":"2024-04-25T19:19:18.018289+0000","last_undegraded":"2024-04-25T19:19:18.018289+0000","last_fullsized":"2024-04-25T19:19:18.018289+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:41.633057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"23'138","reported_seq":155,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.038730+0000","last_change":"2024-04-25T19:15:19.434476+0000","last_active":"2024-04-25T19:19:18.038730+0000","last_peered":"2024-04-25T19:19:18.038730+0000","last_clean":"2024-04-25T19:19:18.038730+0000","last_became_active":"2024-04-25T19:15:19.433836+0000","last_became_peered":"2024-04-25T19:15:19.433836+0000","last_unstale":"2024-04-25T19:19:18.038730+0000","last_undegraded":"2024-04-25T19:19:18.038730+0000","last_fullsized":"2024-04-25T19:19:18.038730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:13.234829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"23'147","reported_seq":164,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.181499+0000","last_change":"2024-04-25T19:15:19.415521+0000","last_active":"2024-04-25T19:19:18.181499+0000","last_peered":"2024-04-25T19:19:18.181499+0000","last_clean":"2024-04-25T19:19:18.181499+0000","last_became_active":"2024-04-25T19:15:19.415177+0000","last_became_peered":"2024-04-25T19:15:19.415177+0000","last_unstale":"2024-04-25T19:19:18.181499+0000","last_undegraded":"2024-04-25T19:19:18.181499+0000","last_fullsized":"2024-04-25T19:19:18.181499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:24.355619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'165","reported_seq":182,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.085552+0000","last_change":"2024-04-25T19:15:19.430291+0000","last_active":"2024-04-25T19:19:18.085552+0000","last_peered":"2024-04-25T19:19:18.085552+0000","last_clean":"2024-04-25T19:19:18.085552+0000","last_became_active":"2024-04-25T19:15:19.427226+0000","last_became_peered":"2024-04-25T19:15:19.427226+0000","last_unstale":"2024-04-25T19:19:18.085552+0000","last_undegraded":"2024-04-25T19:19:18.085552+0000","last_fullsized":"2024-04-25T19:19:18.085552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:38.464211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"23'153","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.064081+0000","last_change":"2024-04-25T19:15:19.421696+0000","last_active":"2024-04-25T19:19:18.064081+0000","last_peered":"2024-04-25T19:19:18.064081+0000","last_clean":"2024-04-25T19:19:18.064081+0000","last_became_active":"2024-04-25T19:15:19.421300+0000","last_became_peered":"2024-04-25T19:15:19.421300+0000","last_unstale":"2024-04-25T19:19:18.064081+0000","last_undegraded":"2024-04-25T19:19:18.064081+0000","last_fullsized":"2024-04-25T19:19:18.064081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:15.226398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'126","reported_seq":143,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.090958+0000","last_change":"2024-04-25T19:15:19.427880+0000","last_active":"2024-04-25T19:19:18.090958+0000","last_peered":"2024-04-25T19:19:18.090958+0000","last_clean":"2024-04-25T19:19:18.090958+0000","last_became_active":"2024-04-25T19:15:19.427579+0000","last_became_peered":"2024-04-25T19:15:19.427579+0000","last_unstale":"2024-04-25T19:19:18.090958+0000","last_undegraded":"2024-04-25T19:19:18.090958+0000","last_fullsized":"2024-04-25T19:19:18.090958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:47:10.657715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'156","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.083315+0000","last_change":"2024-04-25T19:15:19.430409+0000","last_active":"2024-04-25T19:19:18.083315+0000","last_peered":"2024-04-25T19:19:18.083315+0000","last_clean":"2024-04-25T19:19:18.083315+0000","last_became_active":"2024-04-25T19:15:19.428305+0000","last_became_peered":"2024-04-25T19:15:19.428305+0000","last_unstale":"2024-04-25T19:19:18.083315+0000","last_undegraded":"2024-04-25T19:19:18.083315+0000","last_fullsized":"2024-04-25T19:19:18.083315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:10:01.268963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"23'126","reported_seq":143,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.827387+0000","last_change":"2024-04-25T19:15:19.424435+0000","last_active":"2024-04-25T19:19:17.827387+0000","last_peered":"2024-04-25T19:19:17.827387+0000","last_clean":"2024-04-25T19:19:17.827387+0000","last_became_active":"2024-04-25T19:15:19.422763+0000","last_became_peered":"2024-04-25T19:15:19.422763+0000","last_unstale":"2024-04-25T19:19:17.827387+0000","last_undegraded":"2024-04-25T19:19:17.827387+0000","last_fullsized":"2024-04-25T19:19:17.827387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:58.092412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'174","reported_seq":191,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.937851+0000","last_change":"2024-04-25T19:15:19.420275+0000","last_active":"2024-04-25T19:19:17.937851+0000","last_peered":"2024-04-25T19:19:17.937851+0000","last_clean":"2024-04-25T19:19:17.937851+0000","last_became_active":"2024-04-25T19:15:19.419936+0000","last_became_peered":"2024-04-25T19:15:19.419936+0000","last_unstale":"2024-04-25T19:19:17.937851+0000","last_undegraded":"2024-04-25T19:19:17.937851+0000","last_fullsized":"2024-04-25T19:19:17.937851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:55.710620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'132","reported_seq":149,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.061024+0000","last_change":"2024-04-25T19:15:19.423408+0000","last_active":"2024-04-25T19:19:18.061024+0000","last_peered":"2024-04-25T19:19:18.061024+0000","last_clean":"2024-04-25T19:19:18.061024+0000","last_became_active":"2024-04-25T19:15:19.423116+0000","last_became_peered":"2024-04-25T19:15:19.423116+0000","last_unstale":"2024-04-25T19:19:18.061024+0000","last_undegraded":"2024-04-25T19:19:18.061024+0000","last_fullsized":"2024-04-25T19:19:18.061024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:18:57.787013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'183","reported_seq":200,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.071336+0000","last_change":"2024-04-25T19:15:19.428826+0000","last_active":"2024-04-25T19:19:18.071336+0000","last_peered":"2024-04-25T19:19:18.071336+0000","last_clean":"2024-04-25T19:19:18.071336+0000","last_became_active":"2024-04-25T19:15:19.428474+0000","last_became_peered":"2024-04-25T19:15:19.428474+0000","last_unstale":"2024-04-25T19:19:18.071336+0000","last_undegraded":"2024-04-25T19:19:18.071336+0000","last_fullsized":"2024-04-25T19:19:18.071336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:19.974851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'138","reported_seq":155,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.059590+0000","last_change":"2024-04-25T19:15:19.423353+0000","last_active":"2024-04-25T19:19:18.059590+0000","last_peered":"2024-04-25T19:19:18.059590+0000","last_clean":"2024-04-25T19:19:18.059590+0000","last_became_active":"2024-04-25T19:15:19.422795+0000","last_became_peered":"2024-04-25T19:15:19.422795+0000","last_unstale":"2024-04-25T19:19:18.059590+0000","last_undegraded":"2024-04-25T19:19:18.059590+0000","last_fullsized":"2024-04-25T19:19:18.059590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:34.119976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'129","reported_seq":146,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.066581+0000","last_change":"2024-04-25T19:15:19.421809+0000","last_active":"2024-04-25T19:19:18.066581+0000","last_peered":"2024-04-25T19:19:18.066581+0000","last_clean":"2024-04-25T19:19:18.066581+0000","last_became_active":"2024-04-25T19:15:19.421515+0000","last_became_peered":"2024-04-25T19:15:19.421515+0000","last_unstale":"2024-04-25T19:19:18.066581+0000","last_undegraded":"2024-04-25T19:19:18.066581+0000","last_fullsized":"2024-04-25T19:19:18.066581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:22.603291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'111","reported_seq":129,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.068349+0000","last_change":"2024-04-25T19:15:19.410183+0000","last_active":"2024-04-25T19:19:18.068349+0000","last_peered":"2024-04-25T19:19:18.068349+0000","last_clean":"2024-04-25T19:19:18.068349+0000","last_became_active":"2024-04-25T19:15:19.409857+0000","last_became_peered":"2024-04-25T19:15:19.409857+0000","last_unstale":"2024-04-25T19:19:18.068349+0000","last_undegraded":"2024-04-25T19:19:18.068349+0000","last_fullsized":"2024-04-25T19:19:18.068349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:30.180786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'123","reported_seq":141,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.068201+0000","last_change":"2024-04-25T19:15:19.425524+0000","last_active":"2024-04-25T19:19:18.068201+0000","last_peered":"2024-04-25T19:19:18.068201+0000","last_clean":"2024-04-25T19:19:18.068201+0000","last_became_active":"2024-04-25T19:15:19.424578+0000","last_became_peered":"2024-04-25T19:15:19.424578+0000","last_unstale":"2024-04-25T19:19:18.068201+0000","last_undegraded":"2024-04-25T19:19:18.068201+0000","last_fullsized":"2024-04-25T19:19:18.068201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:02.103879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"23'123","reported_seq":141,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.093807+0000","last_change":"2024-04-25T19:15:19.426720+0000","last_active":"2024-04-25T19:19:18.093807+0000","last_peered":"2024-04-25T19:19:18.093807+0000","last_clean":"2024-04-25T19:19:18.093807+0000","last_became_active":"2024-04-25T19:15:19.426139+0000","last_became_peered":"2024-04-25T19:15:19.426139+0000","last_unstale":"2024-04-25T19:19:18.093807+0000","last_undegraded":"2024-04-25T19:19:18.093807+0000","last_fullsized":"2024-04-25T19:19:18.093807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:31.608993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'141","reported_seq":159,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.097157+0000","last_change":"2024-04-25T19:15:19.413805+0000","last_active":"2024-04-25T19:19:18.097157+0000","last_peered":"2024-04-25T19:19:18.097157+0000","last_clean":"2024-04-25T19:19:18.097157+0000","last_became_active":"2024-04-25T19:15:19.413455+0000","last_became_peered":"2024-04-25T19:15:19.413455+0000","last_unstale":"2024-04-25T19:19:18.097157+0000","last_undegraded":"2024-04-25T19:19:18.097157+0000","last_fullsized":"2024-04-25T19:19:18.097157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:05:45.008076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'123","reported_seq":140,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.921522+0000","last_change":"2024-04-25T19:15:19.423770+0000","last_active":"2024-04-25T19:19:17.921522+0000","last_peered":"2024-04-25T19:19:17.921522+0000","last_clean":"2024-04-25T19:19:17.921522+0000","last_became_active":"2024-04-25T19:15:19.423406+0000","last_became_peered":"2024-04-25T19:15:19.423406+0000","last_unstale":"2024-04-25T19:19:17.921522+0000","last_undegraded":"2024-04-25T19:19:17.921522+0000","last_fullsized":"2024-04-25T19:19:17.921522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:41:23.669832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'6","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.660321+0000","last_change":"2024-04-25T19:15:18.422035+0000","last_active":"2024-04-25T19:16:17.660321+0000","last_peered":"2024-04-25T19:16:17.660321+0000","last_clean":"2024-04-25T19:16:17.660321+0000","last_became_active":"2024-04-25T19:15:18.421658+0000","last_became_peered":"2024-04-25T19:15:18.421658+0000","last_unstale":"2024-04-25T19:16:17.660321+0000","last_undegraded":"2024-04-25T19:16:17.660321+0000","last_fullsized":"2024-04-25T19:16:17.660321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:23.349126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1063,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'129","reported_seq":146,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.994924+0000","last_change":"2024-04-25T19:15:19.421805+0000","last_active":"2024-04-25T19:19:17.994924+0000","last_peered":"2024-04-25T19:19:17.994924+0000","last_clean":"2024-04-25T19:19:17.994924+0000","last_became_active":"2024-04-25T19:15:19.421497+0000","last_became_peered":"2024-04-25T19:15:19.421497+0000","last_unstale":"2024-04-25T19:19:17.994924+0000","last_undegraded":"2024-04-25T19:19:17.994924+0000","last_fullsized":"2024-04-25T19:19:17.994924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:29:29.949508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'7","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.553158+0000","last_change":"2024-04-25T19:15:18.416894+0000","last_active":"2024-04-25T19:16:17.553158+0000","last_peered":"2024-04-25T19:16:17.553158+0000","last_clean":"2024-04-25T19:16:17.553158+0000","last_became_active":"2024-04-25T19:15:18.416539+0000","last_became_peered":"2024-04-25T19:15:18.416539+0000","last_unstale":"2024-04-25T19:16:17.553158+0000","last_undegraded":"2024-04-25T19:16:17.553158+0000","last_fullsized":"2024-04-25T19:16:17.553158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:30.388606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"21'3","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.659993+0000","last_change":"2024-04-25T19:15:18.420129+0000","last_active":"2024-04-25T19:16:17.659993+0000","last_peered":"2024-04-25T19:16:17.659993+0000","last_clean":"2024-04-25T19:16:17.659993+0000","last_became_active":"2024-04-25T19:15:18.419732+0000","last_became_peered":"2024-04-25T19:15:18.419732+0000","last_unstale":"2024-04-25T19:16:17.659993+0000","last_undegraded":"2024-04-25T19:16:17.659993+0000","last_fullsized":"2024-04-25T19:16:17.659993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:41:48.673989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'162","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.034836+0000","last_change":"2024-04-25T19:16:17.801934+0000","last_active":"2024-04-25T19:19:18.034836+0000","last_peered":"2024-04-25T19:19:18.034836+0000","last_clean":"2024-04-25T19:19:18.034836+0000","last_became_active":"2024-04-25T19:16:17.801451+0000","last_became_peered":"2024-04-25T19:16:17.801451+0000","last_unstale":"2024-04-25T19:19:18.034836+0000","last_undegraded":"2024-04-25T19:19:18.034836+0000","last_fullsized":"2024-04-25T19:19:18.034836+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:41:52.528120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"21'3","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.550543+0000","last_change":"2024-04-25T19:15:18.420711+0000","last_active":"2024-04-25T19:16:17.550543+0000","last_peered":"2024-04-25T19:16:17.550543+0000","last_clean":"2024-04-25T19:16:17.550543+0000","last_became_active":"2024-04-25T19:15:18.420362+0000","last_became_peered":"2024-04-25T19:15:18.420362+0000","last_unstale":"2024-04-25T19:16:17.550543+0000","last_undegraded":"2024-04-25T19:16:17.550543+0000","last_fullsized":"2024-04-25T19:16:17.550543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:48.245193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'147","reported_seq":164,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.906678+0000","last_change":"2024-04-25T19:15:19.424356+0000","last_active":"2024-04-25T19:19:17.906678+0000","last_peered":"2024-04-25T19:19:17.906678+0000","last_clean":"2024-04-25T19:19:17.906678+0000","last_became_active":"2024-04-25T19:15:19.423556+0000","last_became_peered":"2024-04-25T19:15:19.423556+0000","last_unstale":"2024-04-25T19:19:17.906678+0000","last_undegraded":"2024-04-25T19:19:17.906678+0000","last_fullsized":"2024-04-25T19:19:17.906678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:19:35.427261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'7","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:42.864151+0000","last_change":"2024-04-25T19:15:18.419250+0000","last_active":"2024-04-25T19:17:42.864151+0000","last_peered":"2024-04-25T19:17:42.864151+0000","last_clean":"2024-04-25T19:17:42.864151+0000","last_became_active":"2024-04-25T19:15:18.418927+0000","last_became_peered":"2024-04-25T19:15:18.418927+0000","last_unstale":"2024-04-25T19:17:42.864151+0000","last_undegraded":"2024-04-25T19:17:42.864151+0000","last_fullsized":"2024-04-25T19:17:42.864151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:26.012965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'219","reported_seq":236,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.335276+0000","last_change":"2024-04-25T19:15:19.430117+0000","last_active":"2024-04-25T19:19:18.335276+0000","last_peered":"2024-04-25T19:19:18.335276+0000","last_clean":"2024-04-25T19:19:18.335276+0000","last_became_active":"2024-04-25T19:15:19.428985+0000","last_became_peered":"2024-04-25T19:15:19.428985+0000","last_unstale":"2024-04-25T19:19:18.335276+0000","last_undegraded":"2024-04-25T19:19:18.335276+0000","last_fullsized":"2024-04-25T19:19:18.335276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:36:29.069541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"21'4","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.550765+0000","last_change":"2024-04-25T19:15:18.422931+0000","last_active":"2024-04-25T19:16:17.550765+0000","last_peered":"2024-04-25T19:16:17.550765+0000","last_clean":"2024-04-25T19:16:17.550765+0000","last_became_active":"2024-04-25T19:15:18.422610+0000","last_became_peered":"2024-04-25T19:15:18.422610+0000","last_unstale":"2024-04-25T19:16:17.550765+0000","last_undegraded":"2024-04-25T19:16:17.550765+0000","last_fullsized":"2024-04-25T19:16:17.550765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:18.489484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'132","reported_seq":149,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.080062+0000","last_change":"2024-04-25T19:15:19.416109+0000","last_active":"2024-04-25T19:19:18.080062+0000","last_peered":"2024-04-25T19:19:18.080062+0000","last_clean":"2024-04-25T19:19:18.080062+0000","last_became_active":"2024-04-25T19:15:19.415734+0000","last_became_peered":"2024-04-25T19:15:19.415734+0000","last_unstale":"2024-04-25T19:19:18.080062+0000","last_undegraded":"2024-04-25T19:19:18.080062+0000","last_fullsized":"2024-04-25T19:19:18.080062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:57:31.406303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'10","reported_seq":29,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:16.549699+0000","last_change":"2024-04-25T19:15:18.428677+0000","last_active":"2024-04-25T19:19:16.549699+0000","last_peered":"2024-04-25T19:19:16.549699+0000","last_clean":"2024-04-25T19:19:16.549699+0000","last_became_active":"2024-04-25T19:15:18.428432+0000","last_became_peered":"2024-04-25T19:15:18.428432+0000","last_unstale":"2024-04-25T19:19:16.549699+0000","last_undegraded":"2024-04-25T19:19:16.549699+0000","last_fullsized":"2024-04-25T19:19:16.549699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:38.478595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'138","reported_seq":155,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.036477+0000","last_change":"2024-04-25T19:15:19.428245+0000","last_active":"2024-04-25T19:19:18.036477+0000","last_peered":"2024-04-25T19:19:18.036477+0000","last_clean":"2024-04-25T19:19:18.036477+0000","last_became_active":"2024-04-25T19:15:19.427941+0000","last_became_peered":"2024-04-25T19:15:19.427941+0000","last_unstale":"2024-04-25T19:19:18.036477+0000","last_undegraded":"2024-04-25T19:19:18.036477+0000","last_fullsized":"2024-04-25T19:19:18.036477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:09.947923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'99","reported_seq":118,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:19.439568+0000","last_change":"2024-04-25T19:15:18.420881+0000","last_active":"2024-04-25T19:19:19.439568+0000","last_peered":"2024-04-25T19:19:19.439568+0000","last_clean":"2024-04-25T19:19:19.439568+0000","last_became_active":"2024-04-25T19:15:18.420303+0000","last_became_peered":"2024-04-25T19:15:18.420303+0000","last_unstale":"2024-04-25T19:19:19.439568+0000","last_undegraded":"2024-04-25T19:19:19.439568+0000","last_fullsized":"2024-04-25T19:19:19.439568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:39:24.356298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":216545,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'144","reported_seq":155,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.986213+0000","last_change":"2024-04-25T19:16:17.661342+0000","last_active":"2024-04-25T19:19:17.986213+0000","last_peered":"2024-04-25T19:19:17.986213+0000","last_clean":"2024-04-25T19:19:17.986213+0000","last_became_active":"2024-04-25T19:16:17.661023+0000","last_became_peered":"2024-04-25T19:16:17.661023+0000","last_unstale":"2024-04-25T19:19:17.986213+0000","last_undegraded":"2024-04-25T19:19:17.986213+0000","last_fullsized":"2024-04-25T19:19:17.986213+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:31:11.190902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.13","version":"21'9","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.800767+0000","last_change":"2024-04-25T19:15:18.421008+0000","last_active":"2024-04-25T19:16:17.800767+0000","last_peered":"2024-04-25T19:16:17.800767+0000","last_clean":"2024-04-25T19:16:17.800767+0000","last_became_active":"2024-04-25T19:15:18.420402+0000","last_became_peered":"2024-04-25T19:15:18.420402+0000","last_unstale":"2024-04-25T19:16:17.800767+0000","last_undegraded":"2024-04-25T19:16:17.800767+0000","last_fullsized":"2024-04-25T19:16:17.800767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:05:28.723958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'148","reported_seq":165,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.054624+0000","last_change":"2024-04-25T19:15:19.424805+0000","last_active":"2024-04-25T19:19:18.054624+0000","last_peered":"2024-04-25T19:19:18.054624+0000","last_clean":"2024-04-25T19:19:18.054624+0000","last_became_active":"2024-04-25T19:15:19.424512+0000","last_became_peered":"2024-04-25T19:15:19.424512+0000","last_unstale":"2024-04-25T19:19:18.054624+0000","last_undegraded":"2024-04-25T19:19:18.054624+0000","last_fullsized":"2024-04-25T19:19:18.054624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:50.423186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.550627+0000","last_change":"2024-04-25T19:15:18.426805+0000","last_active":"2024-04-25T19:16:17.550627+0000","last_peered":"2024-04-25T19:16:17.550627+0000","last_clean":"2024-04-25T19:16:17.550627+0000","last_became_active":"2024-04-25T19:15:18.426472+0000","last_became_peered":"2024-04-25T19:15:18.426472+0000","last_unstale":"2024-04-25T19:16:17.550627+0000","last_undegraded":"2024-04-25T19:16:17.550627+0000","last_fullsized":"2024-04-25T19:16:17.550627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:29.373363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'156","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.047258+0000","last_change":"2024-04-25T19:15:19.428780+0000","last_active":"2024-04-25T19:19:18.047258+0000","last_peered":"2024-04-25T19:19:18.047258+0000","last_clean":"2024-04-25T19:19:18.047258+0000","last_became_active":"2024-04-25T19:15:19.428450+0000","last_became_peered":"2024-04-25T19:15:19.428450+0000","last_unstale":"2024-04-25T19:19:18.047258+0000","last_undegraded":"2024-04-25T19:19:18.047258+0000","last_fullsized":"2024-04-25T19:19:18.047258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:01.203880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"21'6","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.552957+0000","last_change":"2024-04-25T19:15:18.417740+0000","last_active":"2024-04-25T19:16:17.552957+0000","last_peered":"2024-04-25T19:16:17.552957+0000","last_clean":"2024-04-25T19:16:17.552957+0000","last_became_active":"2024-04-25T19:15:18.417404+0000","last_became_peered":"2024-04-25T19:15:18.417404+0000","last_unstale":"2024-04-25T19:16:17.552957+0000","last_undegraded":"2024-04-25T19:16:17.552957+0000","last_fullsized":"2024-04-25T19:16:17.552957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:21.737283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1029,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'159","reported_seq":176,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.060341+0000","last_change":"2024-04-25T19:15:19.411214+0000","last_active":"2024-04-25T19:19:18.060341+0000","last_peered":"2024-04-25T19:19:18.060341+0000","last_clean":"2024-04-25T19:19:18.060341+0000","last_became_active":"2024-04-25T19:15:19.410814+0000","last_became_peered":"2024-04-25T19:15:19.410814+0000","last_unstale":"2024-04-25T19:19:18.060341+0000","last_undegraded":"2024-04-25T19:19:18.060341+0000","last_fullsized":"2024-04-25T19:19:18.060341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:42.430277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'18","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:37.867718+0000","last_change":"2024-04-25T19:15:18.422226+0000","last_active":"2024-04-25T19:17:37.867718+0000","last_peered":"2024-04-25T19:17:37.867718+0000","last_clean":"2024-04-25T19:17:37.867718+0000","last_became_active":"2024-04-25T19:15:18.421900+0000","last_became_peered":"2024-04-25T19:15:18.421900+0000","last_unstale":"2024-04-25T19:17:37.867718+0000","last_undegraded":"2024-04-25T19:17:37.867718+0000","last_fullsized":"2024-04-25T19:17:37.867718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:18.040114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'183","reported_seq":200,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.048026+0000","last_change":"2024-04-25T19:15:19.428885+0000","last_active":"2024-04-25T19:19:18.048026+0000","last_peered":"2024-04-25T19:19:18.048026+0000","last_clean":"2024-04-25T19:19:18.048026+0000","last_became_active":"2024-04-25T19:15:19.428563+0000","last_became_peered":"2024-04-25T19:15:19.428563+0000","last_unstale":"2024-04-25T19:19:18.048026+0000","last_undegraded":"2024-04-25T19:19:18.048026+0000","last_fullsized":"2024-04-25T19:19:18.048026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:43.469189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'2","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.800374+0000","last_change":"2024-04-25T19:15:18.422125+0000","last_active":"2024-04-25T19:16:17.800374+0000","last_peered":"2024-04-25T19:16:17.800374+0000","last_clean":"2024-04-25T19:16:17.800374+0000","last_became_active":"2024-04-25T19:15:18.421777+0000","last_became_peered":"2024-04-25T19:15:18.421777+0000","last_unstale":"2024-04-25T19:16:17.800374+0000","last_undegraded":"2024-04-25T19:16:17.800374+0000","last_fullsized":"2024-04-25T19:16:17.800374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:24.289010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'141","reported_seq":158,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.052159+0000","last_change":"2024-04-25T19:15:19.429675+0000","last_active":"2024-04-25T19:19:18.052159+0000","last_peered":"2024-04-25T19:19:18.052159+0000","last_clean":"2024-04-25T19:19:18.052159+0000","last_became_active":"2024-04-25T19:15:19.429316+0000","last_became_peered":"2024-04-25T19:15:19.429316+0000","last_unstale":"2024-04-25T19:19:18.052159+0000","last_undegraded":"2024-04-25T19:19:18.052159+0000","last_fullsized":"2024-04-25T19:19:18.052159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:09:55.521944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.552135+0000","last_change":"2024-04-25T19:15:18.418661+0000","last_active":"2024-04-25T19:16:17.552135+0000","last_peered":"2024-04-25T19:16:17.552135+0000","last_clean":"2024-04-25T19:16:17.552135+0000","last_became_active":"2024-04-25T19:15:18.418359+0000","last_became_peered":"2024-04-25T19:15:18.418359+0000","last_unstale":"2024-04-25T19:16:17.552135+0000","last_undegraded":"2024-04-25T19:16:17.552135+0000","last_fullsized":"2024-04-25T19:16:17.552135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:49:22.023652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'129","reported_seq":146,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.027304+0000","last_change":"2024-04-25T19:15:19.434853+0000","last_active":"2024-04-25T19:19:18.027304+0000","last_peered":"2024-04-25T19:19:18.027304+0000","last_clean":"2024-04-25T19:19:18.027304+0000","last_became_active":"2024-04-25T19:15:19.434508+0000","last_became_peered":"2024-04-25T19:15:19.434508+0000","last_unstale":"2024-04-25T19:19:18.027304+0000","last_undegraded":"2024-04-25T19:19:18.027304+0000","last_fullsized":"2024-04-25T19:19:18.027304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:52.965864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'3","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.552800+0000","last_change":"2024-04-25T19:15:18.428626+0000","last_active":"2024-04-25T19:16:17.552800+0000","last_peered":"2024-04-25T19:16:17.552800+0000","last_clean":"2024-04-25T19:16:17.552800+0000","last_became_active":"2024-04-25T19:15:18.428332+0000","last_became_peered":"2024-04-25T19:15:18.428332+0000","last_unstale":"2024-04-25T19:16:17.552800+0000","last_undegraded":"2024-04-25T19:16:17.552800+0000","last_fullsized":"2024-04-25T19:16:17.552800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:15.595305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'138","reported_seq":155,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.016419+0000","last_change":"2024-04-25T19:15:19.428754+0000","last_active":"2024-04-25T19:19:18.016419+0000","last_peered":"2024-04-25T19:19:18.016419+0000","last_clean":"2024-04-25T19:19:18.016419+0000","last_became_active":"2024-04-25T19:15:19.427992+0000","last_became_peered":"2024-04-25T19:15:19.427992+0000","last_unstale":"2024-04-25T19:19:18.016419+0000","last_undegraded":"2024-04-25T19:19:18.016419+0000","last_fullsized":"2024-04-25T19:19:18.016419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:43:48.443923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"21'6","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.551964+0000","last_change":"2024-04-25T19:15:18.419645+0000","last_active":"2024-04-25T19:16:17.551964+0000","last_peered":"2024-04-25T19:16:17.551964+0000","last_clean":"2024-04-25T19:16:17.551964+0000","last_became_active":"2024-04-25T19:15:18.419286+0000","last_became_peered":"2024-04-25T19:15:18.419286+0000","last_unstale":"2024-04-25T19:16:17.551964+0000","last_undegraded":"2024-04-25T19:16:17.551964+0000","last_fullsized":"2024-04-25T19:16:17.551964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:08.295208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1611,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'144","reported_seq":161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.964325+0000","last_change":"2024-04-25T19:15:19.432395+0000","last_active":"2024-04-25T19:19:17.964325+0000","last_peered":"2024-04-25T19:19:17.964325+0000","last_clean":"2024-04-25T19:19:17.964325+0000","last_became_active":"2024-04-25T19:15:19.431739+0000","last_became_peered":"2024-04-25T19:15:19.431739+0000","last_unstale":"2024-04-25T19:19:17.964325+0000","last_undegraded":"2024-04-25T19:19:17.964325+0000","last_fullsized":"2024-04-25T19:19:17.964325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:50:03.964030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":116,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:17.871195+0000","last_change":"2024-04-25T19:15:15.604825+0000","last_active":"2024-04-25T19:17:17.871195+0000","last_peered":"2024-04-25T19:17:17.871195+0000","last_clean":"2024-04-25T19:17:17.871195+0000","last_became_active":"2024-04-25T19:15:15.603730+0000","last_became_peered":"2024-04-25T19:15:15.603730+0000","last_unstale":"2024-04-25T19:17:17.871195+0000","last_undegraded":"2024-04-25T19:17:17.871195+0000","last_fullsized":"2024-04-25T19:17:17.871195+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:14.367890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:14.367890+0000","last_clean_scrub_stamp":"2024-04-25T19:15:14.367890+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:02.411899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'6","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:22.863164+0000","last_change":"2024-04-25T19:15:18.426331+0000","last_active":"2024-04-25T19:17:22.863164+0000","last_peered":"2024-04-25T19:17:22.863164+0000","last_clean":"2024-04-25T19:17:22.863164+0000","last_became_active":"2024-04-25T19:15:18.425993+0000","last_became_peered":"2024-04-25T19:15:18.425993+0000","last_unstale":"2024-04-25T19:17:22.863164+0000","last_undegraded":"2024-04-25T19:17:22.863164+0000","last_fullsized":"2024-04-25T19:17:22.863164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:17.440484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'150","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.096775+0000","last_change":"2024-04-25T19:15:19.417337+0000","last_active":"2024-04-25T19:19:18.096775+0000","last_peered":"2024-04-25T19:19:18.096775+0000","last_clean":"2024-04-25T19:19:18.096775+0000","last_became_active":"2024-04-25T19:15:19.416808+0000","last_became_peered":"2024-04-25T19:15:19.416808+0000","last_unstale":"2024-04-25T19:19:18.096775+0000","last_undegraded":"2024-04-25T19:19:18.096775+0000","last_fullsized":"2024-04-25T19:19:18.096775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:57.797904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"21'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:17.871167+0000","last_change":"2024-04-25T19:15:18.426358+0000","last_active":"2024-04-25T19:17:17.871167+0000","last_peered":"2024-04-25T19:17:17.871167+0000","last_clean":"2024-04-25T19:17:17.871167+0000","last_became_active":"2024-04-25T19:15:18.426037+0000","last_became_peered":"2024-04-25T19:15:18.426037+0000","last_unstale":"2024-04-25T19:17:17.871167+0000","last_undegraded":"2024-04-25T19:17:17.871167+0000","last_fullsized":"2024-04-25T19:17:17.871167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:26:28.945667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'126","reported_seq":144,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.100470+0000","last_change":"2024-04-25T19:15:19.426219+0000","last_active":"2024-04-25T19:19:18.100470+0000","last_peered":"2024-04-25T19:19:18.100470+0000","last_clean":"2024-04-25T19:19:18.100470+0000","last_became_active":"2024-04-25T19:15:19.425626+0000","last_became_peered":"2024-04-25T19:15:19.425626+0000","last_unstale":"2024-04-25T19:19:18.100470+0000","last_undegraded":"2024-04-25T19:19:18.100470+0000","last_fullsized":"2024-04-25T19:19:18.100470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:58.372454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'11","reported_seq":35,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:16.540140+0000","last_change":"2024-04-25T19:15:18.424560+0000","last_active":"2024-04-25T19:19:16.540140+0000","last_peered":"2024-04-25T19:19:16.540140+0000","last_clean":"2024-04-25T19:19:16.540140+0000","last_became_active":"2024-04-25T19:15:18.424179+0000","last_became_peered":"2024-04-25T19:15:18.424179+0000","last_unstale":"2024-04-25T19:19:16.540140+0000","last_undegraded":"2024-04-25T19:19:16.540140+0000","last_fullsized":"2024-04-25T19:19:16.540140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:06.184399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'123","reported_seq":141,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.076512+0000","last_change":"2024-04-25T19:15:19.415219+0000","last_active":"2024-04-25T19:19:18.076512+0000","last_peered":"2024-04-25T19:19:18.076512+0000","last_clean":"2024-04-25T19:19:18.076512+0000","last_became_active":"2024-04-25T19:15:19.414877+0000","last_became_peered":"2024-04-25T19:15:19.414877+0000","last_unstale":"2024-04-25T19:19:18.076512+0000","last_undegraded":"2024-04-25T19:19:18.076512+0000","last_fullsized":"2024-04-25T19:19:18.076512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:53:15.063419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'5","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:22.863125+0000","last_change":"2024-04-25T19:15:18.423402+0000","last_active":"2024-04-25T19:17:22.863125+0000","last_peered":"2024-04-25T19:17:22.863125+0000","last_clean":"2024-04-25T19:17:22.863125+0000","last_became_active":"2024-04-25T19:15:18.422330+0000","last_became_peered":"2024-04-25T19:15:18.422330+0000","last_unstale":"2024-04-25T19:17:22.863125+0000","last_undegraded":"2024-04-25T19:17:22.863125+0000","last_fullsized":"2024-04-25T19:17:22.863125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:39:17.512502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'138","reported_seq":156,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.097425+0000","last_change":"2024-04-25T19:15:19.421096+0000","last_active":"2024-04-25T19:19:18.097425+0000","last_peered":"2024-04-25T19:19:18.097425+0000","last_clean":"2024-04-25T19:19:18.097425+0000","last_became_active":"2024-04-25T19:15:19.420598+0000","last_became_peered":"2024-04-25T19:15:19.420598+0000","last_unstale":"2024-04-25T19:19:18.097425+0000","last_undegraded":"2024-04-25T19:19:18.097425+0000","last_fullsized":"2024-04-25T19:19:18.097425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:20.733888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"21'1","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:22.862361+0000","last_change":"2024-04-25T19:15:18.422979+0000","last_active":"2024-04-25T19:17:22.862361+0000","last_peered":"2024-04-25T19:17:22.862361+0000","last_clean":"2024-04-25T19:17:22.862361+0000","last_became_active":"2024-04-25T19:15:18.422621+0000","last_became_peered":"2024-04-25T19:15:18.422621+0000","last_unstale":"2024-04-25T19:17:22.862361+0000","last_undegraded":"2024-04-25T19:17:22.862361+0000","last_fullsized":"2024-04-25T19:17:22.862361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:47:56.160615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'138","reported_seq":156,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.095676+0000","last_change":"2024-04-25T19:15:19.426379+0000","last_active":"2024-04-25T19:19:18.095676+0000","last_peered":"2024-04-25T19:19:18.095676+0000","last_clean":"2024-04-25T19:19:18.095676+0000","last_became_active":"2024-04-25T19:15:19.425997+0000","last_became_peered":"2024-04-25T19:15:19.425997+0000","last_unstale":"2024-04-25T19:19:18.095676+0000","last_undegraded":"2024-04-25T19:19:18.095676+0000","last_fullsized":"2024-04-25T19:19:18.095676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:32.972274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"21'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:22.871729+0000","last_change":"2024-04-25T19:15:18.427748+0000","last_active":"2024-04-25T19:17:22.871729+0000","last_peered":"2024-04-25T19:17:22.871729+0000","last_clean":"2024-04-25T19:17:22.871729+0000","last_became_active":"2024-04-25T19:15:18.427374+0000","last_became_peered":"2024-04-25T19:15:18.427374+0000","last_unstale":"2024-04-25T19:17:22.871729+0000","last_undegraded":"2024-04-25T19:17:22.871729+0000","last_fullsized":"2024-04-25T19:17:22.871729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:47:41.469558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'135","reported_seq":153,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.083369+0000","last_change":"2024-04-25T19:15:19.422196+0000","last_active":"2024-04-25T19:19:18.083369+0000","last_peered":"2024-04-25T19:19:18.083369+0000","last_clean":"2024-04-25T19:19:18.083369+0000","last_became_active":"2024-04-25T19:15:19.421835+0000","last_became_peered":"2024-04-25T19:15:19.421835+0000","last_unstale":"2024-04-25T19:19:18.083369+0000","last_undegraded":"2024-04-25T19:19:18.083369+0000","last_fullsized":"2024-04-25T19:19:18.083369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:38.500369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'24","reported_seq":43,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:14.465958+0000","last_change":"2024-04-25T19:15:18.419086+0000","last_active":"2024-04-25T19:19:14.465958+0000","last_peered":"2024-04-25T19:19:14.465958+0000","last_clean":"2024-04-25T19:19:14.465958+0000","last_became_active":"2024-04-25T19:15:18.418775+0000","last_became_peered":"2024-04-25T19:15:18.418775+0000","last_unstale":"2024-04-25T19:19:14.465958+0000","last_undegraded":"2024-04-25T19:19:14.465958+0000","last_fullsized":"2024-04-25T19:19:14.465958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:20.074753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'138","reported_seq":155,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.072440+0000","last_change":"2024-04-25T19:15:19.432155+0000","last_active":"2024-04-25T19:19:18.072440+0000","last_peered":"2024-04-25T19:19:18.072440+0000","last_clean":"2024-04-25T19:19:18.072440+0000","last_became_active":"2024-04-25T19:15:19.431905+0000","last_became_peered":"2024-04-25T19:15:19.431905+0000","last_unstale":"2024-04-25T19:19:18.072440+0000","last_undegraded":"2024-04-25T19:19:18.072440+0000","last_fullsized":"2024-04-25T19:19:18.072440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:52.752469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'8","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:27.862195+0000","last_change":"2024-04-25T19:15:18.423144+0000","last_active":"2024-04-25T19:17:27.862195+0000","last_peered":"2024-04-25T19:17:27.862195+0000","last_clean":"2024-04-25T19:17:27.862195+0000","last_became_active":"2024-04-25T19:15:18.422838+0000","last_became_peered":"2024-04-25T19:15:18.422838+0000","last_unstale":"2024-04-25T19:17:27.862195+0000","last_undegraded":"2024-04-25T19:17:27.862195+0000","last_fullsized":"2024-04-25T19:17:27.862195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:28.026912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'156","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.014340+0000","last_change":"2024-04-25T19:15:19.420582+0000","last_active":"2024-04-25T19:19:18.014340+0000","last_peered":"2024-04-25T19:19:18.014340+0000","last_clean":"2024-04-25T19:19:18.014340+0000","last_became_active":"2024-04-25T19:15:19.420270+0000","last_became_peered":"2024-04-25T19:15:19.420270+0000","last_unstale":"2024-04-25T19:19:18.014340+0000","last_undegraded":"2024-04-25T19:19:18.014340+0000","last_fullsized":"2024-04-25T19:19:18.014340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:06.827819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"21'2","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.549438+0000","last_change":"2024-04-25T19:15:18.425469+0000","last_active":"2024-04-25T19:16:17.549438+0000","last_peered":"2024-04-25T19:16:17.549438+0000","last_clean":"2024-04-25T19:16:17.549438+0000","last_became_active":"2024-04-25T19:15:18.425084+0000","last_became_peered":"2024-04-25T19:15:18.425084+0000","last_unstale":"2024-04-25T19:16:17.549438+0000","last_undegraded":"2024-04-25T19:16:17.549438+0000","last_fullsized":"2024-04-25T19:16:17.549438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:14.363158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'132","reported_seq":149,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.814179+0000","last_change":"2024-04-25T19:15:19.425159+0000","last_active":"2024-04-25T19:19:17.814179+0000","last_peered":"2024-04-25T19:19:17.814179+0000","last_clean":"2024-04-25T19:19:17.814179+0000","last_became_active":"2024-04-25T19:15:19.424847+0000","last_became_peered":"2024-04-25T19:15:19.424847+0000","last_unstale":"2024-04-25T19:19:17.814179+0000","last_undegraded":"2024-04-25T19:19:17.814179+0000","last_fullsized":"2024-04-25T19:19:17.814179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:28.715286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"21'7","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.552420+0000","last_change":"2024-04-25T19:15:18.424645+0000","last_active":"2024-04-25T19:16:17.552420+0000","last_peered":"2024-04-25T19:16:17.552420+0000","last_clean":"2024-04-25T19:16:17.552420+0000","last_became_active":"2024-04-25T19:15:18.424241+0000","last_became_peered":"2024-04-25T19:15:18.424241+0000","last_unstale":"2024-04-25T19:16:17.552420+0000","last_undegraded":"2024-04-25T19:16:17.552420+0000","last_fullsized":"2024-04-25T19:16:17.552420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:29.373363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'144","reported_seq":161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.074330+0000","last_change":"2024-04-25T19:15:19.421095+0000","last_active":"2024-04-25T19:19:18.074330+0000","last_peered":"2024-04-25T19:19:18.074330+0000","last_clean":"2024-04-25T19:19:18.074330+0000","last_became_active":"2024-04-25T19:15:19.420795+0000","last_became_peered":"2024-04-25T19:15:19.420795+0000","last_unstale":"2024-04-25T19:19:18.074330+0000","last_undegraded":"2024-04-25T19:19:18.074330+0000","last_fullsized":"2024-04-25T19:19:18.074330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:13.061506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'3","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.552520+0000","last_change":"2024-04-25T19:15:18.423841+0000","last_active":"2024-04-25T19:16:17.552520+0000","last_peered":"2024-04-25T19:16:17.552520+0000","last_clean":"2024-04-25T19:16:17.552520+0000","last_became_active":"2024-04-25T19:15:18.423500+0000","last_became_peered":"2024-04-25T19:15:18.423500+0000","last_unstale":"2024-04-25T19:16:17.552520+0000","last_undegraded":"2024-04-25T19:16:17.552520+0000","last_fullsized":"2024-04-25T19:16:17.552520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:58.933085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'135","reported_seq":152,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.018424+0000","last_change":"2024-04-25T19:15:19.416232+0000","last_active":"2024-04-25T19:19:18.018424+0000","last_peered":"2024-04-25T19:19:18.018424+0000","last_clean":"2024-04-25T19:19:18.018424+0000","last_became_active":"2024-04-25T19:15:19.415828+0000","last_became_peered":"2024-04-25T19:15:19.415828+0000","last_unstale":"2024-04-25T19:19:18.018424+0000","last_undegraded":"2024-04-25T19:19:18.018424+0000","last_fullsized":"2024-04-25T19:19:18.018424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:59.751840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:32.872674+0000","last_change":"2024-04-25T19:15:18.423918+0000","last_active":"2024-04-25T19:17:32.872674+0000","last_peered":"2024-04-25T19:17:32.872674+0000","last_clean":"2024-04-25T19:17:32.872674+0000","last_became_active":"2024-04-25T19:15:18.423588+0000","last_became_peered":"2024-04-25T19:15:18.423588+0000","last_unstale":"2024-04-25T19:17:32.872674+0000","last_undegraded":"2024-04-25T19:17:32.872674+0000","last_fullsized":"2024-04-25T19:17:32.872674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:16.378285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'123","reported_seq":140,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.998130+0000","last_change":"2024-04-25T19:15:19.422984+0000","last_active":"2024-04-25T19:19:17.998130+0000","last_peered":"2024-04-25T19:19:17.998130+0000","last_clean":"2024-04-25T19:19:17.998130+0000","last_became_active":"2024-04-25T19:15:19.422651+0000","last_became_peered":"2024-04-25T19:15:19.422651+0000","last_unstale":"2024-04-25T19:19:17.998130+0000","last_undegraded":"2024-04-25T19:19:17.998130+0000","last_fullsized":"2024-04-25T19:19:17.998130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:54.984914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'168","reported_seq":185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.068520+0000","last_change":"2024-04-25T19:15:19.423982+0000","last_active":"2024-04-25T19:19:18.068520+0000","last_peered":"2024-04-25T19:19:18.068520+0000","last_clean":"2024-04-25T19:19:18.068520+0000","last_became_active":"2024-04-25T19:15:19.423341+0000","last_became_peered":"2024-04-25T19:15:19.423341+0000","last_unstale":"2024-04-25T19:19:18.068520+0000","last_undegraded":"2024-04-25T19:19:18.068520+0000","last_fullsized":"2024-04-25T19:19:18.068520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:51.808352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.549697+0000","last_change":"2024-04-25T19:15:18.424455+0000","last_active":"2024-04-25T19:16:17.549697+0000","last_peered":"2024-04-25T19:16:17.549697+0000","last_clean":"2024-04-25T19:16:17.549697+0000","last_became_active":"2024-04-25T19:15:18.424119+0000","last_became_peered":"2024-04-25T19:15:18.424119+0000","last_unstale":"2024-04-25T19:16:17.549697+0000","last_undegraded":"2024-04-25T19:16:17.549697+0000","last_fullsized":"2024-04-25T19:16:17.549697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:43.135567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'132","reported_seq":149,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.023065+0000","last_change":"2024-04-25T19:15:19.424064+0000","last_active":"2024-04-25T19:19:18.023065+0000","last_peered":"2024-04-25T19:19:18.023065+0000","last_clean":"2024-04-25T19:19:18.023065+0000","last_became_active":"2024-04-25T19:15:19.423187+0000","last_became_peered":"2024-04-25T19:15:19.423187+0000","last_unstale":"2024-04-25T19:19:18.023065+0000","last_undegraded":"2024-04-25T19:19:18.023065+0000","last_fullsized":"2024-04-25T19:19:18.023065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:43:31.590700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"23'13","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:47.874568+0000","last_change":"2024-04-25T19:15:18.423151+0000","last_active":"2024-04-25T19:17:47.874568+0000","last_peered":"2024-04-25T19:17:47.874568+0000","last_clean":"2024-04-25T19:17:47.874568+0000","last_became_active":"2024-04-25T19:15:18.422831+0000","last_became_peered":"2024-04-25T19:15:18.422831+0000","last_unstale":"2024-04-25T19:17:47.874568+0000","last_undegraded":"2024-04-25T19:17:47.874568+0000","last_fullsized":"2024-04-25T19:17:47.874568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:43.244963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'129","reported_seq":147,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.095343+0000","last_change":"2024-04-25T19:15:19.416717+0000","last_active":"2024-04-25T19:19:18.095343+0000","last_peered":"2024-04-25T19:19:18.095343+0000","last_clean":"2024-04-25T19:19:18.095343+0000","last_became_active":"2024-04-25T19:15:19.416163+0000","last_became_peered":"2024-04-25T19:15:19.416163+0000","last_unstale":"2024-04-25T19:19:18.095343+0000","last_undegraded":"2024-04-25T19:19:18.095343+0000","last_fullsized":"2024-04-25T19:19:18.095343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:46.955159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:17.863896+0000","last_change":"2024-04-25T19:15:18.422875+0000","last_active":"2024-04-25T19:17:17.863896+0000","last_peered":"2024-04-25T19:17:17.863896+0000","last_clean":"2024-04-25T19:17:17.863896+0000","last_became_active":"2024-04-25T19:15:18.422489+0000","last_became_peered":"2024-04-25T19:15:18.422489+0000","last_unstale":"2024-04-25T19:17:17.863896+0000","last_undegraded":"2024-04-25T19:17:17.863896+0000","last_fullsized":"2024-04-25T19:17:17.863896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:30.602500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'147","reported_seq":165,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.080832+0000","last_change":"2024-04-25T19:15:19.416987+0000","last_active":"2024-04-25T19:19:18.080832+0000","last_peered":"2024-04-25T19:19:18.080832+0000","last_clean":"2024-04-25T19:19:18.080832+0000","last_became_active":"2024-04-25T19:15:19.414173+0000","last_became_peered":"2024-04-25T19:15:19.414173+0000","last_unstale":"2024-04-25T19:19:18.080832+0000","last_undegraded":"2024-04-25T19:19:18.080832+0000","last_fullsized":"2024-04-25T19:19:18.080832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:32:43.188653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'13","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:22.862662+0000","last_change":"2024-04-25T19:15:18.423867+0000","last_active":"2024-04-25T19:17:22.862662+0000","last_peered":"2024-04-25T19:17:22.862662+0000","last_clean":"2024-04-25T19:17:22.862662+0000","last_became_active":"2024-04-25T19:15:18.423532+0000","last_became_peered":"2024-04-25T19:15:18.423532+0000","last_unstale":"2024-04-25T19:17:22.862662+0000","last_undegraded":"2024-04-25T19:17:22.862662+0000","last_fullsized":"2024-04-25T19:17:22.862662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:04.531338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9217,"ondisk_log_size":9217,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":224795,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":344,"num_write_kb":333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":696320,"data_stored":449590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64838,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574903,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39832,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332008,"statfs":{"total":96636764160,"available":96595976192,"internally_reserved":0,"allocated":344064,"data_stored":680704,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574903,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35808,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336032,"statfs":{"total":96636764160,"available":96600096768,"internally_reserved":0,"allocated":483328,"data_stored":305571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39780,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332060,"statfs":{"total":96636764160,"available":96596029440,"internally_reserved":0,"allocated":290816,"data_stored":92168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574903,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39756,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332084,"statfs":{"total":96636764160,"available":96596054016,"internally_reserved":0,"allocated":266240,"data_stored":89566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574903,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39976,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331864,"statfs":{"total":96636764160,"available":96595828736,"internally_reserved":0,"allocated":491520,"data_stored":306110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35608,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336232,"statfs":{"total":96636764160,"available":96600301568,"internally_reserved":0,"allocated":278528,"data_stored":90203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35600,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336240,"statfs":{"total":96636764160,"available":96600309760,"internally_reserved":0,"allocated":270336,"data_stored":89327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574903,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39824,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332016,"statfs":{"total":96636764160,"available":96595984384,"internally_reserved":0,"allocated":335872,"data_stored":679037,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":249856,"data_stored":218315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":217776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T19:19:34.628 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:19:34.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.773+0000 7f4b61d6c640 1 Processor -- start 2024-04-25T19:19:34.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.773+0000 7f4b61d6c640 1 -- start start 2024-04-25T19:19:34.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.774+0000 7f4b61d6c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4b5c0c5a70 0x7f4b5c15d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:34.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.774+0000 7f4b61d6c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f4b5c0cc3e0 0x7f4b5c0c8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:34.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.774+0000 7f4b61d6c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b5c15cdb0 con 0x7f4b5c153be0 2024-04-25T19:19:34.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.774+0000 7f4b61d6c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4b5c0cabf0 con 0x7f4b5c0c5a70 2024-04-25T19:19:34.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.774+0000 7f4b61d6c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f4b5c0caed0 con 0x7f4b5c0cc3e0 2024-04-25T19:19:34.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.774+0000 7f4b5b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4b5c0c5a70 0x7f4b5c15d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:34.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.774+0000 7f4b5b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4b5c0c5a70 0x7f4b5c15d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50608/0 (socket says 172.21.15.76:50608) 2024-04-25T19:19:34.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.774+0000 7f4b5b7fe640 1 -- 172.21.15.76:0/1687362565 learned_addr learned my addr 172.21.15.76:0/1687362565 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:34.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.774+0000 7f4b53fff640 1 --2- 172.21.15.76:0/1687362565 >> v2:172.21.15.105:3300/0 conn(0x7f4b5c0cc3e0 0x7f4b5c0c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:34.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.774+0000 7f4b5b7fe640 1 -- 172.21.15.76:0/1687362565 >> v2:172.21.15.105:3300/0 conn(0x7f4b5c0cc3e0 msgr2=0x7f4b5c0c8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:34.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.774+0000 7f4b5b7fe640 1 --2- 172.21.15.76:0/1687362565 >> v2:172.21.15.105:3300/0 conn(0x7f4b5c0cc3e0 0x7f4b5c0c8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.774+0000 7f4b5b7fe640 1 -- 172.21.15.76:0/1687362565 >> v1:172.21.15.105:6789/0 conn(0x7f4b5c153be0 legacy=0x7f4b5c0714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:34.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.774+0000 7f4b5b7fe640 1 -- 172.21.15.76:0/1687362565 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5c0cb1b0 con 0x7f4b5c0c5a70 2024-04-25T19:19:34.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.775+0000 7f4b5b7fe640 1 --2- 172.21.15.76:0/1687362565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b5c0c5a70 0x7f4b5c15d590 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c15a950 tx=0x7f4b4c02c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b53b2de1af1413 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:34.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.775+0000 7f4b5affd640 1 -- 172.21.15.76:0/1687362565 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c0325a0 con 0x7f4b5c0c5a70 2024-04-25T19:19:34.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.775+0000 7f4b5affd640 1 -- 172.21.15.76:0/1687362565 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b4c038070 con 0x7f4b5c0c5a70 2024-04-25T19:19:34.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.775+0000 7f4b5affd640 1 -- 172.21.15.76:0/1687362565 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c032960 con 0x7f4b5c0c5a70 2024-04-25T19:19:34.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.776+0000 7f4b61d6c640 1 -- 172.21.15.76:0/1687362565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b5c0c5a70 msgr2=0x7f4b5c15d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:34.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.776+0000 7f4b61d6c640 1 --2- 172.21.15.76:0/1687362565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b5c0c5a70 0x7f4b5c15d590 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c15a950 tx=0x7f4b4c02c8e0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.776+0000 7f4b61d6c640 1 -- 172.21.15.76:0/1687362565 shutdown_connections 2024-04-25T19:19:34.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.776+0000 7f4b61d6c640 1 --2- 172.21.15.76:0/1687362565 >> v2:172.21.15.105:3300/0 conn(0x7f4b5c0cc3e0 0x7f4b5c0c8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.776+0000 7f4b61d6c640 1 --2- 172.21.15.76:0/1687362565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b5c0c5a70 0x7f4b5c15d590 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.776+0000 7f4b61d6c640 1 -- 172.21.15.76:0/1687362565 >> 172.21.15.76:0/1687362565 conn(0x7f4b5c06d6d0 msgr2=0x7f4b5c06dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:34.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.776+0000 7f4b61d6c640 1 -- 172.21.15.76:0/1687362565 shutdown_connections 2024-04-25T19:19:34.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.776+0000 7f4b61d6c640 1 -- 172.21.15.76:0/1687362565 wait complete. 2024-04-25T19:19:34.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.778+0000 7f4b61d6c640 1 Processor -- start 2024-04-25T19:19:34.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.778+0000 7f4b61d6c640 1 -- start start 2024-04-25T19:19:34.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.779+0000 7f4b61d6c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b5c0c5a70 0x7f4b5c1515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:34.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.779+0000 7f4b61d6c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b5c0cc3e0 0x7f4b5c14ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:34.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.779+0000 7f4b61d6c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b5c153be0 0x7f4b5c152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:34.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.779+0000 7f4b61d6c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c1597c0 con 0x7f4b5c0cc3e0 2024-04-25T19:19:34.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.779+0000 7f4b61d6c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5c0c9730 con 0x7f4b5c153be0 2024-04-25T19:19:34.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.779+0000 7f4b61d6c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f4b5c159940 con 0x7f4b5c0c5a70 2024-04-25T19:19:34.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.779+0000 7f4b53fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b5c0cc3e0 0x7f4b5c14ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:34.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.779+0000 7f4b53fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b5c0cc3e0 0x7f4b5c14ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50618/0 (socket says 172.21.15.76:50618) 2024-04-25T19:19:34.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.779+0000 7f4b53fff640 1 -- 172.21.15.76:0/2786199929 learned_addr learned my addr 172.21.15.76:0/2786199929 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:34.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.779+0000 7f4b5b7fe640 1 --2- 172.21.15.76:0/2786199929 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b5c0c5a70 0x7f4b5c1515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:34.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.779+0000 7f4b5bfff640 1 --2- 172.21.15.76:0/2786199929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b5c153be0 0x7f4b5c152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:34.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.779+0000 7f4b53fff640 1 -- 172.21.15.76:0/2786199929 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b5c0c5a70 msgr2=0x7f4b5c1515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:34.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.779+0000 7f4b53fff640 1 --2- 172.21.15.76:0/2786199929 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b5c0c5a70 0x7f4b5c1515b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.779+0000 7f4b53fff640 1 -- 172.21.15.76:0/2786199929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b5c153be0 msgr2=0x7f4b5c152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:34.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.779+0000 7f4b53fff640 1 --2- 172.21.15.76:0/2786199929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b5c153be0 0x7f4b5c152450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.780+0000 7f4b53fff640 1 -- 172.21.15.76:0/2786199929 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5c153120 con 0x7f4b5c0cc3e0 2024-04-25T19:19:34.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.780+0000 7f4b5b7fe640 1 --2- 172.21.15.76:0/2786199929 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b5c0c5a70 0x7f4b5c1515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:19:34.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.780+0000 7f4b5bfff640 1 --2- 172.21.15.76:0/2786199929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b5c153be0 0x7f4b5c152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:19:34.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.780+0000 7f4b53fff640 1 --2- 172.21.15.76:0/2786199929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b5c0cc3e0 0x7f4b5c14ef90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4b4400d620 tx=0x7f4b4400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:34.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.780+0000 7f4b58ff9640 1 -- 172.21.15.76:0/2786199929 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b440125f0 con 0x7f4b5c0cc3e0 2024-04-25T19:19:34.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.780+0000 7f4b58ff9640 1 -- 172.21.15.76:0/2786199929 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b44019070 con 0x7f4b5c0cc3e0 2024-04-25T19:19:34.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.780+0000 7f4b61d6c640 1 -- 172.21.15.76:0/2786199929 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5c153400 con 0x7f4b5c0cc3e0 2024-04-25T19:19:34.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.780+0000 7f4b58ff9640 1 -- 172.21.15.76:0/2786199929 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b440129d0 con 0x7f4b5c0cc3e0 2024-04-25T19:19:34.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.780+0000 7f4b61d6c640 1 -- 172.21.15.76:0/2786199929 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b5c0cd450 con 0x7f4b5c0cc3e0 2024-04-25T19:19:34.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.781+0000 7f4b61d6c640 1 -- 172.21.15.76:0/2786199929 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b5c06a390 con 0x7f4b5c0cc3e0 2024-04-25T19:19:34.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.781+0000 7f4b58ff9640 1 -- 172.21.15.76:0/2786199929 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4b44012b70 con 0x7f4b5c0cc3e0 2024-04-25T19:19:34.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.782+0000 7f4b58ff9640 1 --2- 172.21.15.76:0/2786199929 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4b3c0744e0 0x7f4b3c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:34.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.782+0000 7f4b58ff9640 1 -- 172.21.15.76:0/2786199929 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f4b44095540 con 0x7f4b5c0cc3e0 2024-04-25T19:19:34.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.782+0000 7f4b5b7fe640 1 --2- 172.21.15.76:0/2786199929 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4b3c0744e0 0x7f4b3c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:34.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.783+0000 7f4b5b7fe640 1 --2- 172.21.15.76:0/2786199929 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4b3c0744e0 0x7f4b3c0769f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c037000 tx=0x7f4b4c032000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:34.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.787+0000 7f4b58ff9640 1 -- 172.21.15.76:0/2786199929 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4b4405fd70 con 0x7f4b5c0cc3e0 2024-04-25T19:19:34.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.969+0000 7f4b61d6c640 1 -- 172.21.15.76:0/2786199929 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b5c0c9df0 con 0x7f4b5c0cc3e0 2024-04-25T19:19:34.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.970+0000 7f4b58ff9640 1 -- 172.21.15.76:0/2786199929 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16855 (secure 0 0 0) 0x7f4b4405f670 con 0x7f4b5c0cc3e0 2024-04-25T19:19:34.971 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:19:34.971 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":23,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","created":"2024-04-25T19:15:01.693137+0000","modified":"2024-04-25T19:16:17.545036+0000","last_up_change":"2024-04-25T19:15:11.891681+0000","last_in_change":"2024-04-25T19:15:03.898065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:15:13.846514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:15:16.606533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:15:17.719197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72a16c02-48f9-45aa-9447-fcf755ebf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409574998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409574998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409574998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409574998}]},"public_addr":"172.21.15.76:6817/1409574998","cluster_addr":"172.21.15.76:6819/1409574998","heartbeat_back_addr":"172.21.15.76:6823/1409574998","heartbeat_front_addr":"172.21.15.76:6821/1409574998","state":["exists","up"]},{"osd":1,"uuid":"818d50f0-76c6-45f8-bf79-5e304f14c5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6801","nonce":1302663821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6803","nonce":1302663821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6807","nonce":1302663821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6805","nonce":1302663821}]},"public_addr":"172.21.15.76:6801/1302663821","cluster_addr":"172.21.15.76:6803/1302663821","heartbeat_back_addr":"172.21.15.76:6807/1302663821","heartbeat_front_addr":"172.21.15.76:6805/1302663821","state":["exists","up"]},{"osd":2,"uuid":"5714c8f5-ae23-473c-8cf1-db73811acded","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6809","nonce":1992938181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6811","nonce":1992938181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6815","nonce":1992938181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6813","nonce":1992938181}]},"public_addr":"172.21.15.76:6809/1992938181","cluster_addr":"172.21.15.76:6811/1992938181","heartbeat_back_addr":"172.21.15.76:6815/1992938181","heartbeat_front_addr":"172.21.15.76:6813/1992938181","state":["exists","up"]},{"osd":3,"uuid":"13204cdb-d8e6-4f97-bc6e-9561f442decf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6825","nonce":3418257163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6827","nonce":3418257163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6831","nonce":3418257163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6829","nonce":3418257163}]},"public_addr":"172.21.15.76:6825/3418257163","cluster_addr":"172.21.15.76:6827/3418257163","heartbeat_back_addr":"172.21.15.76:6831/3418257163","heartbeat_front_addr":"172.21.15.76:6829/3418257163","state":["exists","up"]},{"osd":4,"uuid":"5bea5166-1986-4d56-b5f5-ecc8cbe7f3ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6801","nonce":2125002468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6803","nonce":2125002468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6807","nonce":2125002468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6805","nonce":2125002468}]},"public_addr":"172.21.15.105:6801/2125002468","cluster_addr":"172.21.15.105:6803/2125002468","heartbeat_back_addr":"172.21.15.105:6807/2125002468","heartbeat_front_addr":"172.21.15.105:6805/2125002468","state":["exists","up"]},{"osd":5,"uuid":"96f89dcc-270d-4421-b8e8-3b92ba5f9844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6819","nonce":3413103121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6821","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6823","nonce":3413103121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6831","nonce":3413103121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6828","nonce":3413103121}]},"public_addr":"172.21.15.105:6819/3413103121","cluster_addr":"172.21.15.105:6823/3413103121","heartbeat_back_addr":"172.21.15.105:6831/3413103121","heartbeat_front_addr":"172.21.15.105:6828/3413103121","state":["exists","up"]},{"osd":6,"uuid":"93eb087a-d9ba-4096-b85d-b74d96e819a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6809","nonce":2721832954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6811","nonce":2721832954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6815","nonce":2721832954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6813","nonce":2721832954}]},"public_addr":"172.21.15.105:6809/2721832954","cluster_addr":"172.21.15.105:6811/2721832954","heartbeat_back_addr":"172.21.15.105:6815/2721832954","heartbeat_front_addr":"172.21.15.105:6813/2721832954","state":["exists","up"]},{"osd":7,"uuid":"2f80f790-ad2d-460e-a37d-47bf1771ef43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6818","nonce":548350430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6822","nonce":548350430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6829","nonce":548350430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6825","nonce":548350430}]},"public_addr":"172.21.15.105:6818/548350430","cluster_addr":"172.21.15.105:6822/548350430","heartbeat_back_addr":"172.21.15.105:6829/548350430","heartbeat_front_addr":"172.21.15.105:6825/548350430","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.106901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.072819+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.126865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.053464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:08.982614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.104825+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.112714+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.107387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":1}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:19:34.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.972+0000 7f4b61d6c640 1 -- 172.21.15.76:0/2786199929 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4b3c0744e0 msgr2=0x7f4b3c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:34.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.972+0000 7f4b61d6c640 1 --2- 172.21.15.76:0/2786199929 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4b3c0744e0 0x7f4b3c0769f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c037000 tx=0x7f4b4c032000 comp rx=0 tx=0).stop 2024-04-25T19:19:34.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.972+0000 7f4b61d6c640 1 -- 172.21.15.76:0/2786199929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b5c0cc3e0 msgr2=0x7f4b5c14ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:34.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.972+0000 7f4b61d6c640 1 --2- 172.21.15.76:0/2786199929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b5c0cc3e0 0x7f4b5c14ef90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4b4400d620 tx=0x7f4b4400dae0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.972+0000 7f4b61d6c640 1 -- 172.21.15.76:0/2786199929 shutdown_connections 2024-04-25T19:19:34.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.972+0000 7f4b61d6c640 1 --2- 172.21.15.76:0/2786199929 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f4b3c0744e0 0x7f4b3c0769f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.972+0000 7f4b61d6c640 1 --2- 172.21.15.76:0/2786199929 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4b5c153be0 0x7f4b5c152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.972+0000 7f4b61d6c640 1 --2- 172.21.15.76:0/2786199929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b5c0cc3e0 0x7f4b5c14ef90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.972+0000 7f4b61d6c640 1 --2- 172.21.15.76:0/2786199929 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f4b5c0c5a70 0x7f4b5c1515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:34.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.972+0000 7f4b61d6c640 1 -- 172.21.15.76:0/2786199929 >> 172.21.15.76:0/2786199929 conn(0x7f4b5c06d6d0 msgr2=0x7f4b5c159070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:34.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.972+0000 7f4b61d6c640 1 -- 172.21.15.76:0/2786199929 shutdown_connections 2024-04-25T19:19:34.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:34.972+0000 7f4b61d6c640 1 -- 172.21.15.76:0/2786199929 wait complete. 2024-04-25T19:19:35.986 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T19:19:35.987 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T19:19:36.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.103+0000 7f54dc0d9640 1 Processor -- start 2024-04-25T19:19:36.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.104+0000 7f54dc0d9640 1 -- start start 2024-04-25T19:19:36.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.105+0000 7f54dc0d9640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f54d40c9730 0x7f54d415cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.105+0000 7f54dc0d9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f54d4157020 0x7f54d406c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.105+0000 7f54dc0d9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54d4160140 con 0x7f54d4156c10 2024-04-25T19:19:36.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.105+0000 7f54dc0d9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f54d41603e0 con 0x7f54d4157020 2024-04-25T19:19:36.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.105+0000 7f54dc0d9640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f54d4160680 con 0x7f54d40c9730 2024-04-25T19:19:36.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.105+0000 7f54da676640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f54d4157020 0x7f54d406c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.105+0000 7f54d9674640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f54d4156c10 0x7f54d4071a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35782/0 (socket says 172.21.15.76:35782) 2024-04-25T19:19:36.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.105+0000 7f54d9674640 1 -- 172.21.15.76:0/3275833172 learned_addr learned my addr 172.21.15.76:0/3275833172 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:36.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.105+0000 7f54d9e75640 1 --2- 172.21.15.76:0/3275833172 >> v2:172.21.15.105:3300/0 conn(0x7f54d40c9730 0x7f54d415cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.105+0000 7f54da676640 1 -- 172.21.15.76:0/3275833172 >> v2:172.21.15.105:3300/0 conn(0x7f54d40c9730 msgr2=0x7f54d415cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:36.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.105+0000 7f54da676640 1 --2- 172.21.15.76:0/3275833172 >> v2:172.21.15.105:3300/0 conn(0x7f54d40c9730 0x7f54d415cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.105+0000 7f54da676640 1 -- 172.21.15.76:0/3275833172 >> v1:172.21.15.76:6789/0 conn(0x7f54d4156c10 legacy=0x7f54d4071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.106+0000 7f54da676640 1 -- 172.21.15.76:0/3275833172 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d4071560 con 0x7f54d4157020 2024-04-25T19:19:36.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.106+0000 7f54d9e75640 1 --2- 172.21.15.76:0/3275833172 >> v2:172.21.15.105:3300/0 conn(0x7f54d40c9730 0x7f54d415cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:19:36.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.106+0000 7f54da676640 1 --2- 172.21.15.76:0/3275833172 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54d4157020 0x7f54d406c5c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f54d40707a0 tx=0x7f54cc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3180ec50c832787e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:36.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.106+0000 7f54d8e73640 1 -- 172.21.15.76:0/3275833172 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc037070 con 0x7f54d4157020 2024-04-25T19:19:36.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.106+0000 7f54d8e73640 1 -- 172.21.15.76:0/3275833172 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54cc032440 con 0x7f54d4157020 2024-04-25T19:19:36.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.106+0000 7f54d8e73640 1 -- 172.21.15.76:0/3275833172 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc032810 con 0x7f54d4157020 2024-04-25T19:19:36.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.107+0000 7f54dc0d9640 1 -- 172.21.15.76:0/3275833172 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54d4157020 msgr2=0x7f54d406c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.107+0000 7f54dc0d9640 1 --2- 172.21.15.76:0/3275833172 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54d4157020 0x7f54d406c5c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f54d40707a0 tx=0x7f54cc02a1a0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.107+0000 7f54dc0d9640 1 -- 172.21.15.76:0/3275833172 shutdown_connections 2024-04-25T19:19:36.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.107+0000 7f54dc0d9640 1 --2- 172.21.15.76:0/3275833172 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54d4157020 0x7f54d406c5c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.107+0000 7f54dc0d9640 1 --2- 172.21.15.76:0/3275833172 >> v2:172.21.15.105:3300/0 conn(0x7f54d40c9730 0x7f54d415cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.107+0000 7f54dc0d9640 1 -- 172.21.15.76:0/3275833172 >> 172.21.15.76:0/3275833172 conn(0x7f54d406d6d0 msgr2=0x7f54d406dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:36.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.107+0000 7f54dc0d9640 1 -- 172.21.15.76:0/3275833172 shutdown_connections 2024-04-25T19:19:36.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.107+0000 7f54dc0d9640 1 -- 172.21.15.76:0/3275833172 wait complete. 2024-04-25T19:19:36.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.109+0000 7f54dc0d9640 1 Processor -- start 2024-04-25T19:19:36.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.109+0000 7f54dc0d9640 1 -- start start 2024-04-25T19:19:36.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.110+0000 7f54dc0d9640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54d40c9730 0x7f54d4156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.110+0000 7f54dc0d9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54d4156c10 0x7f54d415bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.110+0000 7f54dc0d9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54d4157020 0x7f54d40cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.110+0000 7f54dc0d9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f54d4160f00 con 0x7f54d4156c10 2024-04-25T19:19:36.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.110+0000 7f54dc0d9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f54d4160c20 con 0x7f54d4157020 2024-04-25T19:19:36.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.110+0000 7f54dc0d9640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f54d4072170 con 0x7f54d40c9730 2024-04-25T19:19:36.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.110+0000 7f54d9674640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54d4156c10 0x7f54d415bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.110+0000 7f54d9674640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54d4156c10 0x7f54d415bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50638/0 (socket says 172.21.15.76:50638) 2024-04-25T19:19:36.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.110+0000 7f54d9674640 1 -- 172.21.15.76:0/2832474749 learned_addr learned my addr 172.21.15.76:0/2832474749 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:36.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.110+0000 7f54d9e75640 1 --2- 172.21.15.76:0/2832474749 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54d40c9730 0x7f54d4156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.110+0000 7f54da676640 1 --2- 172.21.15.76:0/2832474749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54d4157020 0x7f54d40cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.110+0000 7f54d9674640 1 -- 172.21.15.76:0/2832474749 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54d40c9730 msgr2=0x7f54d4156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.110+0000 7f54d9674640 1 --2- 172.21.15.76:0/2832474749 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f54d40c9730 0x7f54d4156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.110+0000 7f54d9674640 1 -- 172.21.15.76:0/2832474749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54d4157020 msgr2=0x7f54d40cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.110+0000 7f54d9674640 1 --2- 172.21.15.76:0/2832474749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54d4157020 0x7f54d40cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.110+0000 7f54d9674640 1 -- 172.21.15.76:0/2832474749 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d415cb50 con 0x7f54d4156c10 2024-04-25T19:19:36.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.111+0000 7f54d9674640 1 --2- 172.21.15.76:0/2832474749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54d4156c10 0x7f54d415bf30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f54c400c4b0 tx=0x7f54c400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:36.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.111+0000 7f54ca7fc640 1 -- 172.21.15.76:0/2832474749 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4017070 con 0x7f54d4156c10 2024-04-25T19:19:36.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.111+0000 7f54ca7fc640 1 -- 172.21.15.76:0/2832474749 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54c4005ce0 con 0x7f54d4156c10 2024-04-25T19:19:36.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.111+0000 7f54dc0d9640 1 -- 172.21.15.76:0/2832474749 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d41f62d0 con 0x7f54d4156c10 2024-04-25T19:19:36.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.111+0000 7f54ca7fc640 1 -- 172.21.15.76:0/2832474749 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4007270 con 0x7f54d4156c10 2024-04-25T19:19:36.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.111+0000 7f54dc0d9640 1 -- 172.21.15.76:0/2832474749 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d4155470 con 0x7f54d4156c10 2024-04-25T19:19:36.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.111+0000 7f54dc0d9640 1 -- 172.21.15.76:0/2832474749 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f54d406a390 con 0x7f54d4156c10 2024-04-25T19:19:36.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.112+0000 7f54ca7fc640 1 -- 172.21.15.76:0/2832474749 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f54c401a070 con 0x7f54d4156c10 2024-04-25T19:19:36.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.113+0000 7f54ca7fc640 1 --2- 172.21.15.76:0/2832474749 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f54b80744b0 0x7f54b80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.113+0000 7f54ca7fc640 1 -- 172.21.15.76:0/2832474749 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f54c4094470 con 0x7f54d4156c10 2024-04-25T19:19:36.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.113+0000 7f54ca7fc640 1 --2- 172.21.15.76:0/2832474749 >> [v2:172.21.15.76:6816/1409574998,v1:172.21.15.76:6817/1409574998] conn(0x7f54b807c880 0x7f54b807ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.113+0000 7f54ca7fc640 1 -- 172.21.15.76:0/2832474749 --> [v2:172.21.15.76:6816/1409574998,v1:172.21.15.76:6817/1409574998] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f54b807f420 con 0x7f54b807c880 2024-04-25T19:19:36.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.113+0000 7f54ca7fc640 1 -- 172.21.15.76:0/2832474749 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f54c409b050 con 0x7f54d4156c10 2024-04-25T19:19:36.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.113+0000 7f54d9e75640 1 --2- 172.21.15.76:0/2832474749 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f54b80744b0 0x7f54b80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.113+0000 7f54da676640 1 --2- 172.21.15.76:0/2832474749 >> [v2:172.21.15.76:6816/1409574998,v1:172.21.15.76:6817/1409574998] conn(0x7f54b807c880 0x7f54b807ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.113+0000 7f54da676640 1 --2- 172.21.15.76:0/2832474749 >> [v2:172.21.15.76:6816/1409574998,v1:172.21.15.76:6817/1409574998] conn(0x7f54b807c880 0x7f54b807ed50 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:36.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.113+0000 7f54d9e75640 1 --2- 172.21.15.76:0/2832474749 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f54b80744b0 0x7f54b80769c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f54d0007510 tx=0x7f54d00074a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:36.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.114+0000 7f54ca7fc640 1 -- 172.21.15.76:0/2832474749 <== osd.0 v2:172.21.15.76:6816/1409574998 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f54b807f420 con 0x7f54b807c880 2024-04-25T19:19:36.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.127+0000 7f54dc0d9640 1 -- 172.21.15.76:0/2832474749 --> [v2:172.21.15.76:6816/1409574998,v1:172.21.15.76:6817/1409574998] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f54d4155220 con 0x7f54b807c880 2024-04-25T19:19:36.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.127+0000 7f54ca7fc640 1 -- 172.21.15.76:0/2832474749 <== osd.0 v2:172.21.15.76:6816/1409574998 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f54d4155220 con 0x7f54b807c880 2024-04-25T19:19:36.129 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:19:36.129 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T19:19:36.129 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:19:36.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.127+0000 7f54dc0d9640 1 -- 172.21.15.76:0/2832474749 >> [v2:172.21.15.76:6816/1409574998,v1:172.21.15.76:6817/1409574998] conn(0x7f54b807c880 msgr2=0x7f54b807ed50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.127+0000 7f54dc0d9640 1 --2- 172.21.15.76:0/2832474749 >> [v2:172.21.15.76:6816/1409574998,v1:172.21.15.76:6817/1409574998] conn(0x7f54b807c880 0x7f54b807ed50 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.128+0000 7f54dc0d9640 1 -- 172.21.15.76:0/2832474749 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f54b80744b0 msgr2=0x7f54b80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.128+0000 7f54dc0d9640 1 --2- 172.21.15.76:0/2832474749 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f54b80744b0 0x7f54b80769c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f54d0007510 tx=0x7f54d00074a0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.128+0000 7f54dc0d9640 1 -- 172.21.15.76:0/2832474749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54d4156c10 msgr2=0x7f54d415bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.128+0000 7f54dc0d9640 1 --2- 172.21.15.76:0/2832474749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54d4156c10 0x7f54d415bf30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f54c400c4b0 tx=0x7f54c400c970 comp rx=0 tx=0).stop 2024-04-25T19:19:36.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.128+0000 7f54da676640 1 -- 172.21.15.76:0/2832474749 reap_dead start 2024-04-25T19:19:36.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.128+0000 7f54dc0d9640 1 -- 172.21.15.76:0/2832474749 shutdown_connections 2024-04-25T19:19:36.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.128+0000 7f54dc0d9640 1 -- 172.21.15.76:0/2832474749 >> 172.21.15.76:0/2832474749 conn(0x7f54d406d6d0 msgr2=0x7f54d4153930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:36.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.128+0000 7f54dc0d9640 1 -- 172.21.15.76:0/2832474749 shutdown_connections 2024-04-25T19:19:36.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.128+0000 7f54dc0d9640 1 -- 172.21.15.76:0/2832474749 wait complete. 2024-04-25T19:19:36.140 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T19:19:36.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.247+0000 7fa4a9e84640 1 Processor -- start 2024-04-25T19:19:36.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.247+0000 7fa4a9e84640 1 -- start start 2024-04-25T19:19:36.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.248+0000 7fa4a9e84640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa4a41570d0 0x7fa4a4160260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.248+0000 7fa4a9e84640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa4a406e040 0x7fa4a406cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.248+0000 7fa4a9e84640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4a41609a0 con 0x7fa4a406e450 2024-04-25T19:19:36.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.248+0000 7fa4a9e84640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa4a41574e0 con 0x7fa4a41570d0 2024-04-25T19:19:36.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.248+0000 7fa4a9e84640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa4a4160c80 con 0x7fa4a406e040 2024-04-25T19:19:36.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.248+0000 7fa4a37fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa4a41570d0 0x7fa4a4160260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.248+0000 7fa4a3fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa4a406e450 0x7fa4a40cdc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35786/0 (socket says 172.21.15.76:35786) 2024-04-25T19:19:36.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.248+0000 7fa4a3fff640 1 -- 172.21.15.76:0/1515907262 learned_addr learned my addr 172.21.15.76:0/1515907262 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:36.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.248+0000 7fa4a37fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa4a41570d0 0x7fa4a4160260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50640/0 (socket says 172.21.15.76:50640) 2024-04-25T19:19:36.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.248+0000 7fa4a2ffd640 1 --2- 172.21.15.76:0/1515907262 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa4a406e040 0x7fa4a406cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.248+0000 7fa4a37fe640 1 -- 172.21.15.76:0/1515907262 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa4a406e040 msgr2=0x7fa4a406cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:36.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.248+0000 7fa4a37fe640 1 --2- 172.21.15.76:0/1515907262 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa4a406e040 0x7fa4a406cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.248+0000 7fa4a37fe640 1 -- 172.21.15.76:0/1515907262 >> v1:172.21.15.76:6789/0 conn(0x7fa4a406e450 legacy=0x7fa4a40cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.248+0000 7fa4a27fc640 1 -- 172.21.15.76:0/1515907262 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 718181813 0 0) 0x7fa4a41609a0 con 0x7fa4a406e450 2024-04-25T19:19:36.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.248+0000 7fa4a37fe640 1 -- 172.21.15.76:0/1515907262 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a4160f60 con 0x7fa4a41570d0 2024-04-25T19:19:36.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.249+0000 7fa4a37fe640 1 --2- 172.21.15.76:0/1515907262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4a41570d0 0x7fa4a4160260 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa4940097a0 tx=0x7fa49402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0634511c9141d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:36.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.249+0000 7fa4a27fc640 1 -- 172.21.15.76:0/1515907262 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494037070 con 0x7fa4a41570d0 2024-04-25T19:19:36.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.249+0000 7fa4a27fc640 1 -- 172.21.15.76:0/1515907262 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa494032440 con 0x7fa4a41570d0 2024-04-25T19:19:36.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.249+0000 7fa4a27fc640 1 -- 172.21.15.76:0/1515907262 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494032810 con 0x7fa4a41570d0 2024-04-25T19:19:36.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.250+0000 7fa4a9e84640 1 -- 172.21.15.76:0/1515907262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4a41570d0 msgr2=0x7fa4a4160260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.250+0000 7fa4a9e84640 1 --2- 172.21.15.76:0/1515907262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4a41570d0 0x7fa4a4160260 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa4940097a0 tx=0x7fa49402a210 comp rx=0 tx=0).stop 2024-04-25T19:19:36.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.250+0000 7fa4a9e84640 1 -- 172.21.15.76:0/1515907262 shutdown_connections 2024-04-25T19:19:36.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.250+0000 7fa4a9e84640 1 --2- 172.21.15.76:0/1515907262 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa4a406e040 0x7fa4a406cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.250+0000 7fa4a9e84640 1 --2- 172.21.15.76:0/1515907262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4a41570d0 0x7fa4a4160260 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.250+0000 7fa4a9e84640 1 -- 172.21.15.76:0/1515907262 >> 172.21.15.76:0/1515907262 conn(0x7fa4a40d22d0 msgr2=0x7fa4a40d26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:36.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.250+0000 7fa4a9e84640 1 -- 172.21.15.76:0/1515907262 shutdown_connections 2024-04-25T19:19:36.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.250+0000 7fa4a9e84640 1 -- 172.21.15.76:0/1515907262 wait complete. 2024-04-25T19:19:36.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.252+0000 7fa4a9e84640 1 Processor -- start 2024-04-25T19:19:36.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.252+0000 7fa4a9e84640 1 -- start start 2024-04-25T19:19:36.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.253+0000 7fa4a9e84640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa4a406e040 0x7fa4a415f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.253+0000 7fa4a9e84640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4a406e450 0x7fa4a41e9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.253+0000 7fa4a9e84640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4a41570d0 0x7fa4a414c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.253+0000 7fa4a9e84640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a40ce380 con 0x7fa4a406e450 2024-04-25T19:19:36.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.253+0000 7fa4a9e84640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a406db20 con 0x7fa4a41570d0 2024-04-25T19:19:36.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.253+0000 7fa4a9e84640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa4a40ce500 con 0x7fa4a406e040 2024-04-25T19:19:36.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.253+0000 7fa4a2ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4a406e450 0x7fa4a41e9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.253+0000 7fa4a2ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4a406e450 0x7fa4a41e9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50654/0 (socket says 172.21.15.76:50654) 2024-04-25T19:19:36.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.253+0000 7fa4a2ffd640 1 -- 172.21.15.76:0/1834991897 learned_addr learned my addr 172.21.15.76:0/1834991897 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:36.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.253+0000 7fa4a3fff640 1 --2- 172.21.15.76:0/1834991897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4a41570d0 0x7fa4a414c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.253+0000 7fa4a37fe640 1 --2- 172.21.15.76:0/1834991897 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa4a406e040 0x7fa4a415f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.253+0000 7fa4a2ffd640 1 -- 172.21.15.76:0/1834991897 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa4a406e040 msgr2=0x7fa4a415f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.253+0000 7fa4a2ffd640 1 --2- 172.21.15.76:0/1834991897 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa4a406e040 0x7fa4a415f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.253+0000 7fa4a2ffd640 1 -- 172.21.15.76:0/1834991897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4a41570d0 msgr2=0x7fa4a414c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.253+0000 7fa4a2ffd640 1 --2- 172.21.15.76:0/1834991897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4a41570d0 0x7fa4a414c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.253+0000 7fa4a2ffd640 1 -- 172.21.15.76:0/1834991897 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a415faa0 con 0x7fa4a406e450 2024-04-25T19:19:36.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.253+0000 7fa4a3fff640 1 --2- 172.21.15.76:0/1834991897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4a41570d0 0x7fa4a414c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:19:36.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.254+0000 7fa4a2ffd640 1 --2- 172.21.15.76:0/1834991897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4a406e450 0x7fa4a41e9260 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa49800d600 tx=0x7fa49800dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:36.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.254+0000 7fa48bfff640 1 -- 172.21.15.76:0/1834991897 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4980125f0 con 0x7fa4a406e450 2024-04-25T19:19:36.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.254+0000 7fa48bfff640 1 -- 172.21.15.76:0/1834991897 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa498019070 con 0x7fa4a406e450 2024-04-25T19:19:36.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.254+0000 7fa4a9e84640 1 -- 172.21.15.76:0/1834991897 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a415e770 con 0x7fa4a406e450 2024-04-25T19:19:36.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.254+0000 7fa48bfff640 1 -- 172.21.15.76:0/1834991897 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4980129f0 con 0x7fa4a406e450 2024-04-25T19:19:36.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.254+0000 7fa4a9e84640 1 -- 172.21.15.76:0/1834991897 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a415ea10 con 0x7fa4a406e450 2024-04-25T19:19:36.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.255+0000 7fa4a9e84640 1 -- 172.21.15.76:0/1834991897 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4a406a390 con 0x7fa4a406e450 2024-04-25T19:19:36.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.255+0000 7fa48bfff640 1 -- 172.21.15.76:0/1834991897 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa498012b90 con 0x7fa4a406e450 2024-04-25T19:19:36.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.256+0000 7fa48bfff640 1 --2- 172.21.15.76:0/1834991897 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa4900744b0 0x7fa4900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.256+0000 7fa48bfff640 1 -- 172.21.15.76:0/1834991897 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fa498095730 con 0x7fa4a406e450 2024-04-25T19:19:36.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.256+0000 7fa4a37fe640 1 --2- 172.21.15.76:0/1834991897 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa4900744b0 0x7fa4900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.257+0000 7fa4a37fe640 1 --2- 172.21.15.76:0/1834991897 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa4900744b0 0x7fa4900769c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa494034000 tx=0x7fa49402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:36.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.261+0000 7fa48bfff640 1 -- 172.21.15.76:0/1834991897 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa49805fe40 con 0x7fa4a406e450 2024-04-25T19:19:36.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.439+0000 7fa4a9e84640 1 -- 172.21.15.76:0/1834991897 --> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fa4a415f1c0 con 0x7fa4900744b0 2024-04-25T19:19:36.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.439+0000 7fa48bfff640 1 -- 172.21.15.76:0/1834991897 <== mgr.4109 v2:172.21.15.105:6832/2585775531 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa4a415f1c0 con 0x7fa4900744b0 2024-04-25T19:19:36.441 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T19:19:36.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.442+0000 7fa4a9e84640 1 -- 172.21.15.76:0/1834991897 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa4900744b0 msgr2=0x7fa4900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.442+0000 7fa4a9e84640 1 --2- 172.21.15.76:0/1834991897 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa4900744b0 0x7fa4900769c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa494034000 tx=0x7fa49402e000 comp rx=0 tx=0).stop 2024-04-25T19:19:36.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.442+0000 7fa4a9e84640 1 -- 172.21.15.76:0/1834991897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4a406e450 msgr2=0x7fa4a41e9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.442+0000 7fa4a9e84640 1 --2- 172.21.15.76:0/1834991897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4a406e450 0x7fa4a41e9260 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa49800d600 tx=0x7fa49800dac0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.442+0000 7fa4a9e84640 1 -- 172.21.15.76:0/1834991897 shutdown_connections 2024-04-25T19:19:36.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.442+0000 7fa4a9e84640 1 --2- 172.21.15.76:0/1834991897 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa4900744b0 0x7fa4900769c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.442+0000 7fa4a9e84640 1 --2- 172.21.15.76:0/1834991897 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa4a41570d0 0x7fa4a414c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.442+0000 7fa4a9e84640 1 --2- 172.21.15.76:0/1834991897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4a406e450 0x7fa4a41e9260 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.442+0000 7fa4a9e84640 1 --2- 172.21.15.76:0/1834991897 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa4a406e040 0x7fa4a415f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.442+0000 7fa4a9e84640 1 -- 172.21.15.76:0/1834991897 >> 172.21.15.76:0/1834991897 conn(0x7fa4a40d22d0 msgr2=0x7fa4a40d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:36.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.442+0000 7fa4a9e84640 1 -- 172.21.15.76:0/1834991897 shutdown_connections 2024-04-25T19:19:36.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.442+0000 7fa4a9e84640 1 -- 172.21.15.76:0/1834991897 wait complete. 2024-04-25T19:19:36.455 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T19:19:36.455 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T19:19:36.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.566+0000 7f02dd7a8640 1 Processor -- start 2024-04-25T19:19:36.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.566+0000 7f02dd7a8640 1 -- start start 2024-04-25T19:19:36.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.567+0000 7f02dd7a8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f02d80d0810 0x7f02d806f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.567+0000 7f02dd7a8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02d80cea70 0x7f02d80cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.567+0000 7f02dd7a8640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02d815ce80 con 0x7f02d806e160 2024-04-25T19:19:36.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.567+0000 7f02dd7a8640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f02d815d160 con 0x7f02d80d0810 2024-04-25T19:19:36.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.567+0000 7f02dd7a8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f02d815d440 con 0x7f02d80cea70 2024-04-25T19:19:36.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.568+0000 7f02d6ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f02d80d0810 0x7f02d806f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.568+0000 7f02d77fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f02d806e160 0x7f02d806e570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35796/0 (socket says 172.21.15.76:35796) 2024-04-25T19:19:36.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.568+0000 7f02d77fe640 1 -- 172.21.15.76:0/2280217791 learned_addr learned my addr 172.21.15.76:0/2280217791 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:36.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.568+0000 7f02d67fc640 1 --2- 172.21.15.76:0/2280217791 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02d80cea70 0x7f02d80cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.568+0000 7f02d6ffd640 1 -- 172.21.15.76:0/2280217791 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02d80cea70 msgr2=0x7f02d80cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:36.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.568+0000 7f02d6ffd640 1 --2- 172.21.15.76:0/2280217791 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02d80cea70 0x7f02d80cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.568+0000 7f02d6ffd640 1 -- 172.21.15.76:0/2280217791 >> v1:172.21.15.76:6789/0 conn(0x7f02d806e160 legacy=0x7f02d806e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.568+0000 7f02d5ffb640 1 -- 172.21.15.76:0/2280217791 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 666474851 0 0) 0x7f02d815ce80 con 0x7f02d806e160 2024-04-25T19:19:36.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.568+0000 7f02d6ffd640 1 -- 172.21.15.76:0/2280217791 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d815d720 con 0x7f02d80d0810 2024-04-25T19:19:36.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.568+0000 7f02d67fc640 1 --2- 172.21.15.76:0/2280217791 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02d80cea70 0x7f02d80cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:19:36.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.569+0000 7f02d6ffd640 1 --2- 172.21.15.76:0/2280217791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02d80d0810 0x7f02d806f000 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009d90 tx=0x7f02c802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5d40398ad2aded server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:36.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.569+0000 7f02d5ffb640 1 -- 172.21.15.76:0/2280217791 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c8007740 con 0x7f02d80d0810 2024-04-25T19:19:36.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.569+0000 7f02d5ffb640 1 -- 172.21.15.76:0/2280217791 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02c80078e0 con 0x7f02d80d0810 2024-04-25T19:19:36.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.569+0000 7f02d5ffb640 1 -- 172.21.15.76:0/2280217791 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c8007cb0 con 0x7f02d80d0810 2024-04-25T19:19:36.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.569+0000 7f02dd7a8640 1 -- 172.21.15.76:0/2280217791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02d80d0810 msgr2=0x7f02d806f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.569+0000 7f02dd7a8640 1 --2- 172.21.15.76:0/2280217791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02d80d0810 0x7f02d806f000 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009d90 tx=0x7f02c802a660 comp rx=0 tx=0).stop 2024-04-25T19:19:36.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.569+0000 7f02dd7a8640 1 -- 172.21.15.76:0/2280217791 shutdown_connections 2024-04-25T19:19:36.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.569+0000 7f02dd7a8640 1 --2- 172.21.15.76:0/2280217791 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02d80cea70 0x7f02d80cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.569+0000 7f02dd7a8640 1 --2- 172.21.15.76:0/2280217791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02d80d0810 0x7f02d806f000 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.569+0000 7f02dd7a8640 1 -- 172.21.15.76:0/2280217791 >> 172.21.15.76:0/2280217791 conn(0x7f02d80b3c00 msgr2=0x7f02d80b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:36.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.570+0000 7f02dd7a8640 1 -- 172.21.15.76:0/2280217791 shutdown_connections 2024-04-25T19:19:36.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.570+0000 7f02dd7a8640 1 -- 172.21.15.76:0/2280217791 wait complete. 2024-04-25T19:19:36.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.571+0000 7f02dd7a8640 1 Processor -- start 2024-04-25T19:19:36.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.571+0000 7f02dd7a8640 1 -- start start 2024-04-25T19:19:36.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.572+0000 7f02dd7a8640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02d806e160 0x7f02d81f3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.573+0000 7f02dd7a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02d80cea70 0x7f02d81f6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.573+0000 7f02dd7a8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02d80d0810 0x7f02d8157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.573+0000 7f02dd7a8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f02d815cb70 con 0x7f02d80d0810 2024-04-25T19:19:36.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.573+0000 7f02dd7a8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f02d806dc40 con 0x7f02d80cea70 2024-04-25T19:19:36.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.573+0000 7f02dd7a8640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f02d806f540 con 0x7f02d806e160 2024-04-25T19:19:36.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.573+0000 7f02d77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02d80d0810 0x7f02d8157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.573+0000 7f02d77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02d80d0810 0x7f02d8157410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50672/0 (socket says 172.21.15.76:50672) 2024-04-25T19:19:36.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.573+0000 7f02d77fe640 1 -- 172.21.15.76:0/2811747810 learned_addr learned my addr 172.21.15.76:0/2811747810 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:36.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.573+0000 7f02d6ffd640 1 --2- 172.21.15.76:0/2811747810 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02d806e160 0x7f02d81f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.573+0000 7f02d67fc640 1 --2- 172.21.15.76:0/2811747810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02d80cea70 0x7f02d81f6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.573+0000 7f02d77fe640 1 -- 172.21.15.76:0/2811747810 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02d806e160 msgr2=0x7f02d81f3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.573+0000 7f02d77fe640 1 --2- 172.21.15.76:0/2811747810 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02d806e160 0x7f02d81f3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.573+0000 7f02d77fe640 1 -- 172.21.15.76:0/2811747810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02d80cea70 msgr2=0x7f02d81f6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.573+0000 7f02d77fe640 1 --2- 172.21.15.76:0/2811747810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02d80cea70 0x7f02d81f6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.573+0000 7f02d77fe640 1 -- 172.21.15.76:0/2811747810 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d81f2fe0 con 0x7f02d80d0810 2024-04-25T19:19:36.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.573+0000 7f02d67fc640 1 --2- 172.21.15.76:0/2811747810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02d80cea70 0x7f02d81f6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:19:36.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.573+0000 7f02d77fe640 1 --2- 172.21.15.76:0/2811747810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02d80d0810 0x7f02d8157410 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f02d000c3e0 tx=0x7f02d000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:36.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.573+0000 7f02bf7fe640 1 -- 172.21.15.76:0/2811747810 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0017070 con 0x7f02d80d0810 2024-04-25T19:19:36.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.573+0000 7f02bf7fe640 1 -- 172.21.15.76:0/2811747810 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02d0005ce0 con 0x7f02d80d0810 2024-04-25T19:19:36.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.573+0000 7f02dd7a8640 1 -- 172.21.15.76:0/2811747810 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d81f3cc0 con 0x7f02d80d0810 2024-04-25T19:19:36.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.573+0000 7f02bf7fe640 1 -- 172.21.15.76:0/2811747810 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d00072f0 con 0x7f02d80d0810 2024-04-25T19:19:36.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.573+0000 7f02dd7a8640 1 -- 172.21.15.76:0/2811747810 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d8159a40 con 0x7f02d80d0810 2024-04-25T19:19:36.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.574+0000 7f02dd7a8640 1 -- 172.21.15.76:0/2811747810 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f02d806a390 con 0x7f02d80d0810 2024-04-25T19:19:36.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.575+0000 7f02bf7fe640 1 -- 172.21.15.76:0/2811747810 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f02d001a070 con 0x7f02d80d0810 2024-04-25T19:19:36.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.575+0000 7f02bf7fe640 1 --2- 172.21.15.76:0/2811747810 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f02c40744b0 0x7f02c40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.575+0000 7f02bf7fe640 1 -- 172.21.15.76:0/2811747810 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f02d0094550 con 0x7f02d80d0810 2024-04-25T19:19:36.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.576+0000 7f02d6ffd640 1 --2- 172.21.15.76:0/2811747810 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f02c40744b0 0x7f02c40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.576+0000 7f02bf7fe640 1 --2- 172.21.15.76:0/2811747810 >> [v2:172.21.15.76:6800/1302663821,v1:172.21.15.76:6801/1302663821] conn(0x7f02c407c880 0x7f02c407ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.576+0000 7f02bf7fe640 1 -- 172.21.15.76:0/2811747810 --> [v2:172.21.15.76:6800/1302663821,v1:172.21.15.76:6801/1302663821] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f02c407f420 con 0x7f02c407c880 2024-04-25T19:19:36.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.576+0000 7f02bf7fe640 1 -- 172.21.15.76:0/2811747810 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f02d009b050 con 0x7f02d80d0810 2024-04-25T19:19:36.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.576+0000 7f02d67fc640 1 --2- 172.21.15.76:0/2811747810 >> [v2:172.21.15.76:6800/1302663821,v1:172.21.15.76:6801/1302663821] conn(0x7f02c407c880 0x7f02c407ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.576+0000 7f02d67fc640 1 --2- 172.21.15.76:0/2811747810 >> [v2:172.21.15.76:6800/1302663821,v1:172.21.15.76:6801/1302663821] conn(0x7f02c407c880 0x7f02c407ed50 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:36.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.576+0000 7f02d6ffd640 1 --2- 172.21.15.76:0/2811747810 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f02c40744b0 0x7f02c40769c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f02c8034000 tx=0x7f02c802f000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:36.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.577+0000 7f02bf7fe640 1 -- 172.21.15.76:0/2811747810 <== osd.1 v2:172.21.15.76:6800/1302663821 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f02c407f420 con 0x7f02c407c880 2024-04-25T19:19:36.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.589+0000 7f02dd7a8640 1 -- 172.21.15.76:0/2811747810 --> [v2:172.21.15.76:6800/1302663821,v1:172.21.15.76:6801/1302663821] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f02d81f65e0 con 0x7f02c407c880 2024-04-25T19:19:36.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.590+0000 7f02bf7fe640 1 -- 172.21.15.76:0/2811747810 <== osd.1 v2:172.21.15.76:6800/1302663821 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f02cc00a680 con 0x7f02c407c880 2024-04-25T19:19:36.591 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:19:36.591 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T19:19:36.591 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:19:36.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.590+0000 7f02dd7a8640 1 -- 172.21.15.76:0/2811747810 >> [v2:172.21.15.76:6800/1302663821,v1:172.21.15.76:6801/1302663821] conn(0x7f02c407c880 msgr2=0x7f02c407ed50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.590+0000 7f02dd7a8640 1 --2- 172.21.15.76:0/2811747810 >> [v2:172.21.15.76:6800/1302663821,v1:172.21.15.76:6801/1302663821] conn(0x7f02c407c880 0x7f02c407ed50 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.590+0000 7f02dd7a8640 1 -- 172.21.15.76:0/2811747810 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f02c40744b0 msgr2=0x7f02c40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.590+0000 7f02dd7a8640 1 --2- 172.21.15.76:0/2811747810 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f02c40744b0 0x7f02c40769c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f02c8034000 tx=0x7f02c802f000 comp rx=0 tx=0).stop 2024-04-25T19:19:36.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.590+0000 7f02dd7a8640 1 -- 172.21.15.76:0/2811747810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02d80d0810 msgr2=0x7f02d8157410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.590+0000 7f02dd7a8640 1 --2- 172.21.15.76:0/2811747810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02d80d0810 0x7f02d8157410 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f02d000c3e0 tx=0x7f02d000c8a0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.590+0000 7f02d77fe640 1 -- 172.21.15.76:0/2811747810 reap_dead start 2024-04-25T19:19:36.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.590+0000 7f02dd7a8640 1 -- 172.21.15.76:0/2811747810 shutdown_connections 2024-04-25T19:19:36.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.590+0000 7f02dd7a8640 1 -- 172.21.15.76:0/2811747810 >> 172.21.15.76:0/2811747810 conn(0x7f02d80b3c00 msgr2=0x7f02d815aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:36.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.590+0000 7f02dd7a8640 1 -- 172.21.15.76:0/2811747810 shutdown_connections 2024-04-25T19:19:36.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.590+0000 7f02dd7a8640 1 -- 172.21.15.76:0/2811747810 wait complete. 2024-04-25T19:19:36.602 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T19:19:36.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.711+0000 7f8fd1ff3640 1 Processor -- start 2024-04-25T19:19:36.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.711+0000 7f8fd1ff3640 1 -- start start 2024-04-25T19:19:36.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.713+0000 7f8fd1ff3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8fcc1570d0 0x7f8fcc160260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.713+0000 7f8fd1ff3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8fcc06e040 0x7f8fcc06cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.713+0000 7f8fd1ff3640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fcc1609a0 con 0x7f8fcc06e450 2024-04-25T19:19:36.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.713+0000 7f8fd1ff3640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f8fcc1574e0 con 0x7f8fcc06e040 2024-04-25T19:19:36.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.713+0000 7f8fd1ff3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8fcc160c80 con 0x7f8fcc1570d0 2024-04-25T19:19:36.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.713+0000 7f8fcaffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8fcc06e040 0x7f8fcc06cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.713+0000 7f8fcaffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8fcc06e040 0x7f8fcc06cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50678/0 (socket says 172.21.15.76:50678) 2024-04-25T19:19:36.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.713+0000 7f8fcaffd640 1 -- 172.21.15.76:0/3497133779 learned_addr learned my addr 172.21.15.76:0/3497133779 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:36.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.713+0000 7f8fcb7fe640 1 --2- 172.21.15.76:0/3497133779 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8fcc1570d0 0x7f8fcc160260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.713+0000 7f8fcaffd640 1 -- 172.21.15.76:0/3497133779 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8fcc1570d0 msgr2=0x7f8fcc160260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:36.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.713+0000 7f8fcaffd640 1 --2- 172.21.15.76:0/3497133779 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8fcc1570d0 0x7f8fcc160260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.713+0000 7f8fcaffd640 1 -- 172.21.15.76:0/3497133779 >> v1:172.21.15.105:6789/0 conn(0x7f8fcc06e450 legacy=0x7f8fcc0cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:36.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.713+0000 7f8fcaffd640 1 -- 172.21.15.76:0/3497133779 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fcc160f60 con 0x7f8fcc06e040 2024-04-25T19:19:36.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.714+0000 7f8fcaffd640 1 --2- 172.21.15.76:0/3497133779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fcc06e040 0x7f8fcc06cd70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009870 tx=0x7f8fc002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ccc0ebb3e9f73d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:36.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.714+0000 7f8fca7fc640 1 -- 172.21.15.76:0/3497133779 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0037070 con 0x7f8fcc06e040 2024-04-25T19:19:36.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.714+0000 7f8fca7fc640 1 -- 172.21.15.76:0/3497133779 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fc0032440 con 0x7f8fcc06e040 2024-04-25T19:19:36.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.714+0000 7f8fca7fc640 1 -- 172.21.15.76:0/3497133779 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0032810 con 0x7f8fcc06e040 2024-04-25T19:19:36.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.715+0000 7f8fd1ff3640 1 -- 172.21.15.76:0/3497133779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fcc06e040 msgr2=0x7f8fcc06cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.715+0000 7f8fd1ff3640 1 --2- 172.21.15.76:0/3497133779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fcc06e040 0x7f8fcc06cd70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009870 tx=0x7f8fc002a2a0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.715+0000 7f8fd1ff3640 1 -- 172.21.15.76:0/3497133779 shutdown_connections 2024-04-25T19:19:36.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.715+0000 7f8fd1ff3640 1 --2- 172.21.15.76:0/3497133779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fcc06e040 0x7f8fcc06cd70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.715+0000 7f8fd1ff3640 1 --2- 172.21.15.76:0/3497133779 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8fcc1570d0 0x7f8fcc160260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.715+0000 7f8fd1ff3640 1 -- 172.21.15.76:0/3497133779 >> 172.21.15.76:0/3497133779 conn(0x7f8fcc0d22d0 msgr2=0x7f8fcc0d26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:36.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.715+0000 7f8fd1ff3640 1 -- 172.21.15.76:0/3497133779 shutdown_connections 2024-04-25T19:19:36.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.715+0000 7f8fd1ff3640 1 -- 172.21.15.76:0/3497133779 wait complete. 2024-04-25T19:19:36.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.716+0000 7f8fd1ff3640 1 Processor -- start 2024-04-25T19:19:36.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.717+0000 7f8fd1ff3640 1 -- start start 2024-04-25T19:19:36.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.718+0000 7f8fd1ff3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fcc06e040 0x7f8fcc15b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.718+0000 7f8fd1ff3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fcc06e450 0x7f8fcc0b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.718+0000 7f8fd1ff3640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8fcc1570d0 0x7f8fcc0cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.718+0000 7f8fd1ff3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc0ce380 con 0x7f8fcc06e040 2024-04-25T19:19:36.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.718+0000 7f8fd1ff3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc06db20 con 0x7f8fcc06e450 2024-04-25T19:19:36.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.718+0000 7f8fd1ff3640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8fcc0ce500 con 0x7f8fcc1570d0 2024-04-25T19:19:36.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.718+0000 7f8fcb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fcc06e040 0x7f8fcc15b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.718+0000 7f8fcb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fcc06e040 0x7f8fcc15b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50680/0 (socket says 172.21.15.76:50680) 2024-04-25T19:19:36.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.718+0000 7f8fcb7fe640 1 -- 172.21.15.76:0/3379308536 learned_addr learned my addr 172.21.15.76:0/3379308536 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:36.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.718+0000 7f8fcbfff640 1 --2- 172.21.15.76:0/3379308536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8fcc1570d0 0x7f8fcc0cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.718+0000 7f8fcaffd640 1 --2- 172.21.15.76:0/3379308536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fcc06e450 0x7f8fcc0b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.718+0000 7f8fcb7fe640 1 -- 172.21.15.76:0/3379308536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8fcc1570d0 msgr2=0x7f8fcc0cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.718+0000 7f8fcb7fe640 1 --2- 172.21.15.76:0/3379308536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8fcc1570d0 0x7f8fcc0cb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.718+0000 7f8fcb7fe640 1 -- 172.21.15.76:0/3379308536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fcc06e450 msgr2=0x7f8fcc0b5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.718+0000 7f8fcb7fe640 1 --2- 172.21.15.76:0/3379308536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fcc06e450 0x7f8fcc0b5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.718+0000 7f8fcb7fe640 1 -- 172.21.15.76:0/3379308536 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fcc15b760 con 0x7f8fcc06e040 2024-04-25T19:19:36.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.718+0000 7f8fcbfff640 1 --2- 172.21.15.76:0/3379308536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8fcc1570d0 0x7f8fcc0cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:19:36.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.718+0000 7f8fcb7fe640 1 --2- 172.21.15.76:0/3379308536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fcc06e040 0x7f8fcc15b220 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00d5f0 tx=0x7f8fbc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:36.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.718+0000 7f8fb3fff640 1 -- 172.21.15.76:0/3379308536 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc0125f0 con 0x7f8fcc06e040 2024-04-25T19:19:36.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.718+0000 7f8fb3fff640 1 -- 172.21.15.76:0/3379308536 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fbc019070 con 0x7f8fcc06e040 2024-04-25T19:19:36.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.718+0000 7f8fd1ff3640 1 -- 172.21.15.76:0/3379308536 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fcc159a80 con 0x7f8fcc06e040 2024-04-25T19:19:36.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.718+0000 7f8fb3fff640 1 -- 172.21.15.76:0/3379308536 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc0129d0 con 0x7f8fcc06e040 2024-04-25T19:19:36.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.719+0000 7f8fd1ff3640 1 -- 172.21.15.76:0/3379308536 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fcc157e60 con 0x7f8fcc06e040 2024-04-25T19:19:36.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.719+0000 7f8fd1ff3640 1 -- 172.21.15.76:0/3379308536 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fcc06a390 con 0x7f8fcc06e040 2024-04-25T19:19:36.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.720+0000 7f8fb3fff640 1 -- 172.21.15.76:0/3379308536 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8fbc012b70 con 0x7f8fcc06e040 2024-04-25T19:19:36.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.721+0000 7f8fb3fff640 1 --2- 172.21.15.76:0/3379308536 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8fb80744e0 0x7f8fb80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:36.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.721+0000 7f8fb3fff640 1 -- 172.21.15.76:0/3379308536 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f8fbc095640 con 0x7f8fcc06e040 2024-04-25T19:19:36.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.721+0000 7f8fcaffd640 1 --2- 172.21.15.76:0/3379308536 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8fb80744e0 0x7f8fb80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:36.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.721+0000 7f8fcaffd640 1 --2- 172.21.15.76:0/3379308536 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8fb80744e0 0x7f8fb80769f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0034000 tx=0x7f8fc002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:36.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.725+0000 7f8fb3fff640 1 -- 172.21.15.76:0/3379308536 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8fbc05fde0 con 0x7f8fcc06e040 2024-04-25T19:19:36.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.903+0000 7f8fd1ff3640 1 -- 172.21.15.76:0/3379308536 --> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f8fcc0cd790 con 0x7f8fb80744e0 2024-04-25T19:19:36.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.903+0000 7f8fb3fff640 1 -- 172.21.15.76:0/3379308536 <== mgr.4109 v2:172.21.15.105:6832/2585775531 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8fcc0cd790 con 0x7f8fb80744e0 2024-04-25T19:19:36.905 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T19:19:36.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.906+0000 7f8fd1ff3640 1 -- 172.21.15.76:0/3379308536 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8fb80744e0 msgr2=0x7f8fb80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.906+0000 7f8fd1ff3640 1 --2- 172.21.15.76:0/3379308536 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8fb80744e0 0x7f8fb80769f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0034000 tx=0x7f8fc002e000 comp rx=0 tx=0).stop 2024-04-25T19:19:36.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.906+0000 7f8fd1ff3640 1 -- 172.21.15.76:0/3379308536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fcc06e040 msgr2=0x7f8fcc15b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:36.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.906+0000 7f8fd1ff3640 1 --2- 172.21.15.76:0/3379308536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fcc06e040 0x7f8fcc15b220 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00d5f0 tx=0x7f8fbc00dab0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.906+0000 7f8fd1ff3640 1 -- 172.21.15.76:0/3379308536 shutdown_connections 2024-04-25T19:19:36.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.906+0000 7f8fd1ff3640 1 --2- 172.21.15.76:0/3379308536 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8fb80744e0 0x7f8fb80769f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.906+0000 7f8fd1ff3640 1 --2- 172.21.15.76:0/3379308536 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8fcc1570d0 0x7f8fcc0cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.906+0000 7f8fd1ff3640 1 --2- 172.21.15.76:0/3379308536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8fcc06e450 0x7f8fcc0b5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.906+0000 7f8fd1ff3640 1 --2- 172.21.15.76:0/3379308536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fcc06e040 0x7f8fcc15b220 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:36.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.906+0000 7f8fd1ff3640 1 -- 172.21.15.76:0/3379308536 >> 172.21.15.76:0/3379308536 conn(0x7f8fcc0d22d0 msgr2=0x7f8fcc15e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:36.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.906+0000 7f8fd1ff3640 1 -- 172.21.15.76:0/3379308536 shutdown_connections 2024-04-25T19:19:36.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:36.906+0000 7f8fd1ff3640 1 -- 172.21.15.76:0/3379308536 wait complete. 2024-04-25T19:19:36.919 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T19:19:36.921 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T19:19:37.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.032+0000 7f89f139d640 1 Processor -- start 2024-04-25T19:19:37.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.032+0000 7f89f139d640 1 -- start start 2024-04-25T19:19:37.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.033+0000 7f89f139d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f89ec166340 0x7f89ec168760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.033+0000 7f89f139d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89ec15ccf0 con 0x7f89ec156da0 2024-04-25T19:19:37.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.033+0000 7f89f139d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89ec15cfd0 con 0x7f89ec0ca6d0 2024-04-25T19:19:37.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.033+0000 7f89f139d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f89ec06e930 con 0x7f89ec166340 2024-04-25T19:19:37.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.033+0000 7f89eb7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f89ec156da0 0x7f89ec160530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35810/0 (socket says 172.21.15.76:35810) 2024-04-25T19:19:37.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.033+0000 7f89eb7fe640 1 -- 172.21.15.76:0/2015423212 learned_addr learned my addr 172.21.15.76:0/2015423212 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:37.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.033+0000 7f89f0b9c640 1 --2- 172.21.15.76:0/2015423212 >> v2:172.21.15.105:3300/0 conn(0x7f89ec166340 0x7f89ec168760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.034+0000 7f89eaffd640 1 -- 172.21.15.76:0/2015423212 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2109810010 0 0) 0x7f89ec15ccf0 con 0x7f89ec156da0 2024-04-25T19:19:37.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.034+0000 7f89eaffd640 1 -- 172.21.15.76:0/2015423212 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89d4003520 con 0x7f89ec156da0 2024-04-25T19:19:37.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.034+0000 7f89eaffd640 1 -- 172.21.15.76:0/2015423212 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1139912268 0 0) 0x7f89d4003520 con 0x7f89ec156da0 2024-04-25T19:19:37.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.034+0000 7f89eaffd640 1 -- 172.21.15.76:0/2015423212 >> v2:172.21.15.105:3300/0 conn(0x7f89ec166340 msgr2=0x7f89ec168760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:37.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.034+0000 7f89eaffd640 1 --2- 172.21.15.76:0/2015423212 >> v2:172.21.15.105:3300/0 conn(0x7f89ec166340 0x7f89ec168760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.034+0000 7f89eaffd640 1 -- 172.21.15.76:0/2015423212 >> v1:172.21.15.105:6789/0 conn(0x7f89ec0ca6d0 legacy=0x7f89ec06dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.034+0000 7f89eaffd640 1 -- 172.21.15.76:0/2015423212 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ec15d2b0 con 0x7f89ec156da0 2024-04-25T19:19:37.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.034+0000 7f89eaffd640 1 -- 172.21.15.76:0/2015423212 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f89e0001670 con 0x7f89ec156da0 2024-04-25T19:19:37.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.034+0000 7f89eaffd640 1 -- 172.21.15.76:0/2015423212 >> v1:172.21.15.76:6789/0 conn(0x7f89ec156da0 legacy=0x7f89ec160530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.034+0000 7f89eaffd640 1 --2- 172.21.15.76:0/2015423212 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89d40039b0 0x7f89d4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.034+0000 7f89eaffd640 1 --2- 172.21.15.76:0/2015423212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89d4004800 0x7f89d4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.034+0000 7f89eaffd640 1 --2- 172.21.15.76:0/2015423212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89d40051c0 0x7f89d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.034+0000 7f89eaffd640 1 -- 172.21.15.76:0/2015423212 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f89d400c130 con 0x7f89d4004800 2024-04-25T19:19:37.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.034+0000 7f89eaffd640 1 -- 172.21.15.76:0/2015423212 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f89d400c2e0 con 0x7f89d40051c0 2024-04-25T19:19:37.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.034+0000 7f89eaffd640 1 -- 172.21.15.76:0/2015423212 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f89d400c4e0 con 0x7f89d40039b0 2024-04-25T19:19:37.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.034+0000 7f89eb7fe640 1 --2- 172.21.15.76:0/2015423212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89d4004800 0x7f89d4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.034+0000 7f89ebfff640 1 --2- 172.21.15.76:0/2015423212 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89d40039b0 0x7f89d4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.034+0000 7f89f0b9c640 1 --2- 172.21.15.76:0/2015423212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89d40051c0 0x7f89d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.034+0000 7f89eb7fe640 1 -- 172.21.15.76:0/2015423212 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89d40039b0 msgr2=0x7f89d4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:37.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.034+0000 7f89eb7fe640 1 --2- 172.21.15.76:0/2015423212 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89d40039b0 0x7f89d4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.034+0000 7f89eb7fe640 1 -- 172.21.15.76:0/2015423212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89d40051c0 msgr2=0x7f89d400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:37.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.034+0000 7f89eb7fe640 1 --2- 172.21.15.76:0/2015423212 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89d40051c0 0x7f89d400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.034+0000 7f89eb7fe640 1 -- 172.21.15.76:0/2015423212 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d4003520 con 0x7f89d4004800 2024-04-25T19:19:37.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.034+0000 7f89f0b9c640 1 -- 172.21.15.76:0/2015423212 reap_dead start 2024-04-25T19:19:37.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.035+0000 7f89eb7fe640 1 --2- 172.21.15.76:0/2015423212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89d4004800 0x7f89d4004c10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f89e000c210 tx=0x7f89d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255d27ccf84c11e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:37.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.035+0000 7f89eaffd640 1 -- 172.21.15.76:0/2015423212 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e0007d30 con 0x7f89d4004800 2024-04-25T19:19:37.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.035+0000 7f89eaffd640 1 -- 172.21.15.76:0/2015423212 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89e0005ad0 con 0x7f89d4004800 2024-04-25T19:19:37.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.035+0000 7f89eaffd640 1 -- 172.21.15.76:0/2015423212 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e0007070 con 0x7f89d4004800 2024-04-25T19:19:37.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.036+0000 7f89f139d640 1 -- 172.21.15.76:0/2015423212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89d4004800 msgr2=0x7f89d4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.036+0000 7f89f139d640 1 --2- 172.21.15.76:0/2015423212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89d4004800 0x7f89d4004c10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f89e000c210 tx=0x7f89d4005830 comp rx=0 tx=0).stop 2024-04-25T19:19:37.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.036+0000 7f89f139d640 1 -- 172.21.15.76:0/2015423212 shutdown_connections 2024-04-25T19:19:37.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.036+0000 7f89f139d640 1 --2- 172.21.15.76:0/2015423212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89d4004800 0x7f89d4004c10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.036+0000 7f89f139d640 1 -- 172.21.15.76:0/2015423212 >> 172.21.15.76:0/2015423212 conn(0x7f89ec068ed0 msgr2=0x7f89ec0725d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:37.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.036+0000 7f89f139d640 1 -- 172.21.15.76:0/2015423212 shutdown_connections 2024-04-25T19:19:37.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.036+0000 7f89f139d640 1 -- 172.21.15.76:0/2015423212 wait complete. 2024-04-25T19:19:37.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.038+0000 7f89f139d640 1 Processor -- start 2024-04-25T19:19:37.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.038+0000 7f89f139d640 1 -- start start 2024-04-25T19:19:37.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.039+0000 7f89f139d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89d4004800 0x7f89ec151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.039+0000 7f89f139d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89ec14a7e0 0x7f89ec14abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.039+0000 7f89f139d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89ec1f6570 0x7f89ec1f6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.039+0000 7f89f139d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f89ec15c9c0 con 0x7f89d4004800 2024-04-25T19:19:37.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.039+0000 7f89f139d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f89ec15d870 con 0x7f89ec1f6570 2024-04-25T19:19:37.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.039+0000 7f89f139d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f89ec06d6d0 con 0x7f89ec14a7e0 2024-04-25T19:19:37.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.039+0000 7f89ebfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89d4004800 0x7f89ec151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.039+0000 7f89ebfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89d4004800 0x7f89ec151f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50686/0 (socket says 172.21.15.76:50686) 2024-04-25T19:19:37.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.039+0000 7f89ebfff640 1 -- 172.21.15.76:0/1894278364 learned_addr learned my addr 172.21.15.76:0/1894278364 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:37.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.039+0000 7f89eb7fe640 1 --2- 172.21.15.76:0/1894278364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89ec14a7e0 0x7f89ec14abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.039+0000 7f89f0b9c640 1 --2- 172.21.15.76:0/1894278364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89ec1f6570 0x7f89ec1f6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.039+0000 7f89ebfff640 1 -- 172.21.15.76:0/1894278364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89ec14a7e0 msgr2=0x7f89ec14abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.039+0000 7f89ebfff640 1 --2- 172.21.15.76:0/1894278364 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f89ec14a7e0 0x7f89ec14abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.039+0000 7f89ebfff640 1 -- 172.21.15.76:0/1894278364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89ec1f6570 msgr2=0x7f89ec1f6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.039+0000 7f89ebfff640 1 --2- 172.21.15.76:0/1894278364 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89ec1f6570 0x7f89ec1f6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.039+0000 7f89ebfff640 1 -- 172.21.15.76:0/1894278364 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ec1524b0 con 0x7f89d4004800 2024-04-25T19:19:37.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.040+0000 7f89ebfff640 1 --2- 172.21.15.76:0/1894278364 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89d4004800 0x7f89ec151f70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00f5f0 tx=0x7f89dc00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:37.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.040+0000 7f89e8ff9640 1 -- 172.21.15.76:0/1894278364 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc0052e0 con 0x7f89d4004800 2024-04-25T19:19:37.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.040+0000 7f89e8ff9640 1 -- 172.21.15.76:0/1894278364 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89dc018070 con 0x7f89d4004800 2024-04-25T19:19:37.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.040+0000 7f89f139d640 1 -- 172.21.15.76:0/1894278364 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89ec154d30 con 0x7f89d4004800 2024-04-25T19:19:37.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.040+0000 7f89e8ff9640 1 -- 172.21.15.76:0/1894278364 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc003810 con 0x7f89d4004800 2024-04-25T19:19:37.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.040+0000 7f89f139d640 1 -- 172.21.15.76:0/1894278364 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89ec151770 con 0x7f89d4004800 2024-04-25T19:19:37.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.040+0000 7f89f139d640 1 -- 172.21.15.76:0/1894278364 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f89ec050740 con 0x7f89d4004800 2024-04-25T19:19:37.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.041+0000 7f89e8ff9640 1 -- 172.21.15.76:0/1894278364 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f89dc0039b0 con 0x7f89d4004800 2024-04-25T19:19:37.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.042+0000 7f89e8ff9640 1 --2- 172.21.15.76:0/1894278364 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f89d80744e0 0x7f89d80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.042+0000 7f89e8ff9640 1 -- 172.21.15.76:0/1894278364 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f89dc094ef0 con 0x7f89d4004800 2024-04-25T19:19:37.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.042+0000 7f89eb7fe640 1 --2- 172.21.15.76:0/1894278364 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f89d80744e0 0x7f89d80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.042+0000 7f89e8ff9640 1 --2- 172.21.15.76:0/1894278364 >> [v2:172.21.15.76:6808/1992938181,v1:172.21.15.76:6809/1992938181] conn(0x7f89d807c850 0x7f89d807ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.042+0000 7f89e8ff9640 1 -- 172.21.15.76:0/1894278364 --> [v2:172.21.15.76:6808/1992938181,v1:172.21.15.76:6809/1992938181] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f89dc005480 con 0x7f89d807c850 2024-04-25T19:19:37.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.042+0000 7f89e8ff9640 1 -- 172.21.15.76:0/1894278364 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f89dc0975f0 con 0x7f89d4004800 2024-04-25T19:19:37.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.042+0000 7f89f0b9c640 1 --2- 172.21.15.76:0/1894278364 >> [v2:172.21.15.76:6808/1992938181,v1:172.21.15.76:6809/1992938181] conn(0x7f89d807c850 0x7f89d807ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.042+0000 7f89eb7fe640 1 --2- 172.21.15.76:0/1894278364 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f89d80744e0 0x7f89d80769f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f89e0032000 tx=0x7f89e002d000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:37.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.042+0000 7f89f0b9c640 1 --2- 172.21.15.76:0/1894278364 >> [v2:172.21.15.76:6808/1992938181,v1:172.21.15.76:6809/1992938181] conn(0x7f89d807c850 0x7f89d807ed20 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:37.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.043+0000 7f89e8ff9640 1 -- 172.21.15.76:0/1894278364 <== osd.2 v2:172.21.15.76:6808/1992938181 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f89dc005480 con 0x7f89d807c850 2024-04-25T19:19:37.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.056+0000 7f89f139d640 1 -- 172.21.15.76:0/1894278364 --> [v2:172.21.15.76:6808/1992938181,v1:172.21.15.76:6809/1992938181] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f89ec0caf80 con 0x7f89d807c850 2024-04-25T19:19:37.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.056+0000 7f89e8ff9640 1 -- 172.21.15.76:0/1894278364 <== osd.2 v2:172.21.15.76:6808/1992938181 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f89ec0caf80 con 0x7f89d807c850 2024-04-25T19:19:37.058 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:19:37.058 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T19:19:37.058 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:19:37.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.057+0000 7f89f139d640 1 -- 172.21.15.76:0/1894278364 >> [v2:172.21.15.76:6808/1992938181,v1:172.21.15.76:6809/1992938181] conn(0x7f89d807c850 msgr2=0x7f89d807ed20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.057+0000 7f89f139d640 1 --2- 172.21.15.76:0/1894278364 >> [v2:172.21.15.76:6808/1992938181,v1:172.21.15.76:6809/1992938181] conn(0x7f89d807c850 0x7f89d807ed20 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.057+0000 7f89f139d640 1 -- 172.21.15.76:0/1894278364 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f89d80744e0 msgr2=0x7f89d80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.057+0000 7f89f139d640 1 --2- 172.21.15.76:0/1894278364 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f89d80744e0 0x7f89d80769f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f89e0032000 tx=0x7f89e002d000 comp rx=0 tx=0).stop 2024-04-25T19:19:37.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.057+0000 7f89f139d640 1 -- 172.21.15.76:0/1894278364 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89d4004800 msgr2=0x7f89ec151f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.057+0000 7f89f139d640 1 --2- 172.21.15.76:0/1894278364 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89d4004800 0x7f89ec151f70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00f5f0 tx=0x7f89dc00fab0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.057+0000 7f89f0b9c640 1 -- 172.21.15.76:0/1894278364 reap_dead start 2024-04-25T19:19:37.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.057+0000 7f89f139d640 1 -- 172.21.15.76:0/1894278364 shutdown_connections 2024-04-25T19:19:37.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.057+0000 7f89f139d640 1 -- 172.21.15.76:0/1894278364 >> 172.21.15.76:0/1894278364 conn(0x7f89ec068ed0 msgr2=0x7f89ec0c5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:37.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.057+0000 7f89f139d640 1 -- 172.21.15.76:0/1894278364 shutdown_connections 2024-04-25T19:19:37.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.057+0000 7f89f139d640 1 -- 172.21.15.76:0/1894278364 wait complete. 2024-04-25T19:19:37.069 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T19:19:37.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.178+0000 7fdd59669640 1 Processor -- start 2024-04-25T19:19:37.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.178+0000 7fdd59669640 1 -- start start 2024-04-25T19:19:37.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.179+0000 7fdd59669640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdd541600b0 0x7fdd5415d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.179+0000 7fdd59669640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdd540cce00 0x7fdd540cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.179+0000 7fdd59669640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd541606f0 con 0x7fdd540ca750 2024-04-25T19:19:37.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.179+0000 7fdd59669640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fdd541604c0 con 0x7fdd540cce00 2024-04-25T19:19:37.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.179+0000 7fdd59669640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fdd541609d0 con 0x7fdd541600b0 2024-04-25T19:19:37.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.179+0000 7fdd537fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdd540cce00 0x7fdd540cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.179+0000 7fdd537fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdd540cce00 0x7fdd540cab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50696/0 (socket says 172.21.15.76:50696) 2024-04-25T19:19:37.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.179+0000 7fdd537fe640 1 -- 172.21.15.76:0/2812660746 learned_addr learned my addr 172.21.15.76:0/2812660746 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:37.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.179+0000 7fdd527fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fdd540ca750 0x7fdd540cc6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35826/0 (socket says 172.21.15.76:35826) 2024-04-25T19:19:37.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.179+0000 7fdd52ffd640 1 --2- 172.21.15.76:0/2812660746 >> v2:172.21.15.105:3300/0 conn(0x7fdd541600b0 0x7fdd5415d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.180+0000 7fdd537fe640 1 -- 172.21.15.76:0/2812660746 >> v2:172.21.15.105:3300/0 conn(0x7fdd541600b0 msgr2=0x7fdd5415d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:37.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.180+0000 7fdd537fe640 1 --2- 172.21.15.76:0/2812660746 >> v2:172.21.15.105:3300/0 conn(0x7fdd541600b0 0x7fdd5415d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.180+0000 7fdd537fe640 1 -- 172.21.15.76:0/2812660746 >> v1:172.21.15.76:6789/0 conn(0x7fdd540ca750 legacy=0x7fdd540cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.180+0000 7fdd537fe640 1 -- 172.21.15.76:0/2812660746 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd54160cb0 con 0x7fdd540cce00 2024-04-25T19:19:37.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.180+0000 7fdd537fe640 1 --2- 172.21.15.76:0/2812660746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd540cce00 0x7fdd540cab60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdd54074350 tx=0x7fdd4c02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed3767afec38e9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:37.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.180+0000 7fdd51ffb640 1 -- 172.21.15.76:0/2812660746 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c036070 con 0x7fdd540cce00 2024-04-25T19:19:37.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.180+0000 7fdd51ffb640 1 -- 172.21.15.76:0/2812660746 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd4c031600 con 0x7fdd540cce00 2024-04-25T19:19:37.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.180+0000 7fdd51ffb640 1 -- 172.21.15.76:0/2812660746 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c0319f0 con 0x7fdd540cce00 2024-04-25T19:19:37.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.181+0000 7fdd59669640 1 -- 172.21.15.76:0/2812660746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd540cce00 msgr2=0x7fdd540cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.181+0000 7fdd59669640 1 --2- 172.21.15.76:0/2812660746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd540cce00 0x7fdd540cab60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdd54074350 tx=0x7fdd4c02a7c0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.181+0000 7fdd59669640 1 -- 172.21.15.76:0/2812660746 shutdown_connections 2024-04-25T19:19:37.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.181+0000 7fdd59669640 1 --2- 172.21.15.76:0/2812660746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd540cce00 0x7fdd540cab60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.181+0000 7fdd59669640 1 --2- 172.21.15.76:0/2812660746 >> v2:172.21.15.105:3300/0 conn(0x7fdd541600b0 0x7fdd5415d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.181+0000 7fdd59669640 1 -- 172.21.15.76:0/2812660746 >> 172.21.15.76:0/2812660746 conn(0x7fdd54068b10 msgr2=0x7fdd54072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:37.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.181+0000 7fdd59669640 1 -- 172.21.15.76:0/2812660746 shutdown_connections 2024-04-25T19:19:37.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.181+0000 7fdd59669640 1 -- 172.21.15.76:0/2812660746 wait complete. 2024-04-25T19:19:37.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.183+0000 7fdd59669640 1 Processor -- start 2024-04-25T19:19:37.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.183+0000 7fdd59669640 1 -- start start 2024-04-25T19:19:37.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.184+0000 7fdd59669640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd540ca750 0x7fdd540ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.184+0000 7fdd59669640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd540cce00 0x7fdd5414e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.184+0000 7fdd59669640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd541600b0 0x7fdd541f66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.184+0000 7fdd59669640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdd54160f90 con 0x7fdd540ca750 2024-04-25T19:19:37.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.184+0000 7fdd59669640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5415ceb0 con 0x7fdd540cce00 2024-04-25T19:19:37.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.184+0000 7fdd59669640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fdd540cb3f0 con 0x7fdd541600b0 2024-04-25T19:19:37.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.184+0000 7fdd52ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd540ca750 0x7fdd540ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.184+0000 7fdd52ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd540ca750 0x7fdd540ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50698/0 (socket says 172.21.15.76:50698) 2024-04-25T19:19:37.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.184+0000 7fdd52ffd640 1 -- 172.21.15.76:0/2018283623 learned_addr learned my addr 172.21.15.76:0/2018283623 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:37.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.185+0000 7fdd527fc640 1 --2- 172.21.15.76:0/2018283623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd540cce00 0x7fdd5414e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.185+0000 7fdd52ffd640 1 -- 172.21.15.76:0/2018283623 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd541600b0 msgr2=0x7fdd541f66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.185+0000 7fdd537fe640 1 --2- 172.21.15.76:0/2018283623 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd541600b0 0x7fdd541f66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.185+0000 7fdd52ffd640 1 --2- 172.21.15.76:0/2018283623 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd541600b0 0x7fdd541f66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.185+0000 7fdd52ffd640 1 -- 172.21.15.76:0/2018283623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd540cce00 msgr2=0x7fdd5414e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.185+0000 7fdd52ffd640 1 --2- 172.21.15.76:0/2018283623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd540cce00 0x7fdd5414e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.185+0000 7fdd52ffd640 1 -- 172.21.15.76:0/2018283623 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd540ce5d0 con 0x7fdd540ca750 2024-04-25T19:19:37.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.185+0000 7fdd527fc640 1 --2- 172.21.15.76:0/2018283623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd540cce00 0x7fdd5414e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:19:37.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.185+0000 7fdd537fe640 1 --2- 172.21.15.76:0/2018283623 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd541600b0 0x7fdd541f66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:19:37.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.185+0000 7fdd52ffd640 1 --2- 172.21.15.76:0/2018283623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd540ca750 0x7fdd540ce090 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdd4400e870 tx=0x7fdd4400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:37.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.185+0000 7fdd3b7fe640 1 -- 172.21.15.76:0/2018283623 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd44017070 con 0x7fdd540ca750 2024-04-25T19:19:37.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.185+0000 7fdd3b7fe640 1 -- 172.21.15.76:0/2018283623 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd4400c940 con 0x7fdd540ca750 2024-04-25T19:19:37.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.185+0000 7fdd59669640 1 -- 172.21.15.76:0/2018283623 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd541f7330 con 0x7fdd540ca750 2024-04-25T19:19:37.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.185+0000 7fdd3b7fe640 1 -- 172.21.15.76:0/2018283623 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4400cd30 con 0x7fdd540ca750 2024-04-25T19:19:37.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.185+0000 7fdd59669640 1 -- 172.21.15.76:0/2018283623 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd541f75d0 con 0x7fdd540ca750 2024-04-25T19:19:37.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.186+0000 7fdd59669640 1 -- 172.21.15.76:0/2018283623 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd540ce7a0 con 0x7fdd540ca750 2024-04-25T19:19:37.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.187+0000 7fdd3b7fe640 1 -- 172.21.15.76:0/2018283623 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdd4401a070 con 0x7fdd540ca750 2024-04-25T19:19:37.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.187+0000 7fdd3b7fe640 1 --2- 172.21.15.76:0/2018283623 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fdd400744b0 0x7fdd400769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.187+0000 7fdd3b7fe640 1 -- 172.21.15.76:0/2018283623 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fdd44095060 con 0x7fdd540ca750 2024-04-25T19:19:37.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.188+0000 7fdd527fc640 1 --2- 172.21.15.76:0/2018283623 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fdd400744b0 0x7fdd400769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.188+0000 7fdd527fc640 1 --2- 172.21.15.76:0/2018283623 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fdd400744b0 0x7fdd400769c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdd540cbf80 tx=0x7fdd48009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:37.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.192+0000 7fdd3b7fe640 1 -- 172.21.15.76:0/2018283623 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdd4405f890 con 0x7fdd540ca750 2024-04-25T19:19:37.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.369+0000 7fdd59669640 1 -- 172.21.15.76:0/2018283623 --> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fdd5406e410 con 0x7fdd400744b0 2024-04-25T19:19:37.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.369+0000 7fdd3b7fe640 1 -- 172.21.15.76:0/2018283623 <== mgr.4109 v2:172.21.15.105:6832/2585775531 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fdd5406e410 con 0x7fdd400744b0 2024-04-25T19:19:37.371 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T19:19:37.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.371+0000 7fdd59669640 1 -- 172.21.15.76:0/2018283623 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fdd400744b0 msgr2=0x7fdd400769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.371+0000 7fdd59669640 1 --2- 172.21.15.76:0/2018283623 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fdd400744b0 0x7fdd400769c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdd540cbf80 tx=0x7fdd48009040 comp rx=0 tx=0).stop 2024-04-25T19:19:37.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.371+0000 7fdd59669640 1 -- 172.21.15.76:0/2018283623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd540ca750 msgr2=0x7fdd540ce090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.371+0000 7fdd59669640 1 --2- 172.21.15.76:0/2018283623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd540ca750 0x7fdd540ce090 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdd4400e870 tx=0x7fdd4400ed30 comp rx=0 tx=0).stop 2024-04-25T19:19:37.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.372+0000 7fdd59669640 1 -- 172.21.15.76:0/2018283623 shutdown_connections 2024-04-25T19:19:37.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.372+0000 7fdd59669640 1 --2- 172.21.15.76:0/2018283623 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fdd400744b0 0x7fdd400769c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.372+0000 7fdd59669640 1 --2- 172.21.15.76:0/2018283623 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fdd541600b0 0x7fdd541f66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.372+0000 7fdd59669640 1 --2- 172.21.15.76:0/2018283623 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdd540cce00 0x7fdd5414e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.372+0000 7fdd59669640 1 --2- 172.21.15.76:0/2018283623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd540ca750 0x7fdd540ce090 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.372+0000 7fdd59669640 1 -- 172.21.15.76:0/2018283623 >> 172.21.15.76:0/2018283623 conn(0x7fdd54068b10 msgr2=0x7fdd540c55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:37.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.372+0000 7fdd59669640 1 -- 172.21.15.76:0/2018283623 shutdown_connections 2024-04-25T19:19:37.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.372+0000 7fdd59669640 1 -- 172.21.15.76:0/2018283623 wait complete. 2024-04-25T19:19:37.384 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T19:19:37.384 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T19:19:37.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.495+0000 7f02bf68c640 1 Processor -- start 2024-04-25T19:19:37.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.495+0000 7f02bf68c640 1 -- start start 2024-04-25T19:19:37.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.497+0000 7f02bf68c640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f02b8157110 0x7f02b815ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.497+0000 7f02bf68c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02b80cf760 con 0x7f02b806f3a0 2024-04-25T19:19:37.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.497+0000 7f02bf68c640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02b815d3c0 con 0x7f02b80cc870 2024-04-25T19:19:37.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.497+0000 7f02bf68c640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f02b80cebc0 con 0x7f02b8157110 2024-04-25T19:19:37.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.497+0000 7f02bd428640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f02b806f3a0 0x7f02b80cf050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35832/0 (socket says 172.21.15.76:35832) 2024-04-25T19:19:37.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.497+0000 7f02bd428640 1 -- 172.21.15.76:0/4185455417 learned_addr learned my addr 172.21.15.76:0/4185455417 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:37.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.497+0000 7f02bdc29640 1 --2- 172.21.15.76:0/4185455417 >> v2:172.21.15.105:3300/0 conn(0x7f02b8157110 0x7f02b815ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.497+0000 7f02affff640 1 -- 172.21.15.76:0/4185455417 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2370661891 0 0) 0x7f02b80cf760 con 0x7f02b806f3a0 2024-04-25T19:19:37.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.497+0000 7f02affff640 1 -- 172.21.15.76:0/4185455417 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02a4003520 con 0x7f02b806f3a0 2024-04-25T19:19:37.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.497+0000 7f02affff640 1 -- 172.21.15.76:0/4185455417 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3187070267 0 0) 0x7f02a4003520 con 0x7f02b806f3a0 2024-04-25T19:19:37.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.497+0000 7f02affff640 1 -- 172.21.15.76:0/4185455417 >> v2:172.21.15.105:3300/0 conn(0x7f02b8157110 msgr2=0x7f02b815ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:37.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.497+0000 7f02affff640 1 --2- 172.21.15.76:0/4185455417 >> v2:172.21.15.105:3300/0 conn(0x7f02b8157110 0x7f02b815ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.497+0000 7f02affff640 1 -- 172.21.15.76:0/4185455417 >> v1:172.21.15.105:6789/0 conn(0x7f02b80cc870 legacy=0x7f02b80d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.497+0000 7f02affff640 1 -- 172.21.15.76:0/4185455417 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b815d6a0 con 0x7f02b806f3a0 2024-04-25T19:19:37.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.497+0000 7f02affff640 1 -- 172.21.15.76:0/4185455417 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f02b4001670 con 0x7f02b806f3a0 2024-04-25T19:19:37.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.497+0000 7f02affff640 1 -- 172.21.15.76:0/4185455417 >> v1:172.21.15.76:6789/0 conn(0x7f02b806f3a0 legacy=0x7f02b80cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.497+0000 7f02affff640 1 --2- 172.21.15.76:0/4185455417 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02a40039b0 0x7f02a4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.497+0000 7f02affff640 1 --2- 172.21.15.76:0/4185455417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02a4004800 0x7f02a4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.497+0000 7f02affff640 1 --2- 172.21.15.76:0/4185455417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02a4005270 0x7f02a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.497+0000 7f02affff640 1 -- 172.21.15.76:0/4185455417 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f02b80cebc0 con 0x7f02a4005270 2024-04-25T19:19:37.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.497+0000 7f02affff640 1 -- 172.21.15.76:0/4185455417 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f02a400c180 con 0x7f02a4004800 2024-04-25T19:19:37.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.497+0000 7f02affff640 1 -- 172.21.15.76:0/4185455417 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f02a400c380 con 0x7f02a40039b0 2024-04-25T19:19:37.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.498+0000 7f02bdc29640 1 --2- 172.21.15.76:0/4185455417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02a4005270 0x7f02a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.498+0000 7f02bd428640 1 --2- 172.21.15.76:0/4185455417 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02a40039b0 0x7f02a4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.498+0000 7f02bcc27640 1 --2- 172.21.15.76:0/4185455417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02a4004800 0x7f02a4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.498+0000 7f02bdc29640 1 -- 172.21.15.76:0/4185455417 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02a40039b0 msgr2=0x7f02a4003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:37.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.498+0000 7f02bdc29640 1 --2- 172.21.15.76:0/4185455417 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02a40039b0 0x7f02a4003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.498+0000 7f02bdc29640 1 -- 172.21.15.76:0/4185455417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02a4004800 msgr2=0x7f02a4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:37.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.498+0000 7f02bdc29640 1 --2- 172.21.15.76:0/4185455417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02a4004800 0x7f02a4004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.498+0000 7f02bdc29640 1 -- 172.21.15.76:0/4185455417 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a4003520 con 0x7f02a4005270 2024-04-25T19:19:37.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.498+0000 7f02bcc27640 1 --2- 172.21.15.76:0/4185455417 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02a4004800 0x7f02a4004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:19:37.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.498+0000 7f02bdc29640 1 -- 172.21.15.76:0/4185455417 reap_dead start 2024-04-25T19:19:37.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.498+0000 7f02bdc29640 1 --2- 172.21.15.76:0/4185455417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02a4005270 0x7f02a400b9c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009fc0 tx=0x7f02a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf921e22c5ac0486 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:37.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.499+0000 7f02affff640 1 -- 172.21.15.76:0/4185455417 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b0035020 con 0x7f02a4005270 2024-04-25T19:19:37.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.499+0000 7f02affff640 1 -- 172.21.15.76:0/4185455417 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02b0003b90 con 0x7f02a4005270 2024-04-25T19:19:37.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.499+0000 7f02affff640 1 -- 172.21.15.76:0/4185455417 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b0009c10 con 0x7f02a4005270 2024-04-25T19:19:37.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.499+0000 7f02bf68c640 1 -- 172.21.15.76:0/4185455417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02a4005270 msgr2=0x7f02a400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.499+0000 7f02bf68c640 1 --2- 172.21.15.76:0/4185455417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02a4005270 0x7f02a400b9c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009fc0 tx=0x7f02a4005830 comp rx=0 tx=0).stop 2024-04-25T19:19:37.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.499+0000 7f02bf68c640 1 -- 172.21.15.76:0/4185455417 shutdown_connections 2024-04-25T19:19:37.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.499+0000 7f02bf68c640 1 --2- 172.21.15.76:0/4185455417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02a4005270 0x7f02a400b9c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.499+0000 7f02bf68c640 1 -- 172.21.15.76:0/4185455417 >> 172.21.15.76:0/4185455417 conn(0x7f02b8068900 msgr2=0x7f02b8068d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:37.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.499+0000 7f02bf68c640 1 -- 172.21.15.76:0/4185455417 shutdown_connections 2024-04-25T19:19:37.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.500+0000 7f02bf68c640 1 -- 172.21.15.76:0/4185455417 wait complete. 2024-04-25T19:19:37.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.501+0000 7f02bf68c640 1 Processor -- start 2024-04-25T19:19:37.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.501+0000 7f02bf68c640 1 -- start start 2024-04-25T19:19:37.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.502+0000 7f02bf68c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02a4005270 0x7f02b8160b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.502+0000 7f02bf68c640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02b8152480 0x7f02b81f5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.502+0000 7f02bf68c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02b8152890 0x7f02b81f6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.502+0000 7f02bf68c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f02b80ce850 con 0x7f02b8152890 2024-04-25T19:19:37.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.502+0000 7f02bf68c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f02b815dc60 con 0x7f02a4005270 2024-04-25T19:19:37.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.502+0000 7f02bf68c640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f02b80d09a0 con 0x7f02b8152480 2024-04-25T19:19:37.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.503+0000 7f02bdc29640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02b8152890 0x7f02b81f6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.503+0000 7f02bdc29640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02b8152890 0x7f02b81f6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50720/0 (socket says 172.21.15.76:50720) 2024-04-25T19:19:37.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.503+0000 7f02bdc29640 1 -- 172.21.15.76:0/1778048355 learned_addr learned my addr 172.21.15.76:0/1778048355 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:37.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.503+0000 7f02bcc27640 1 --2- 172.21.15.76:0/1778048355 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02b8152480 0x7f02b81f5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.503+0000 7f02bd428640 1 --2- 172.21.15.76:0/1778048355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02a4005270 0x7f02b8160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.503+0000 7f02bdc29640 1 -- 172.21.15.76:0/1778048355 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02b8152480 msgr2=0x7f02b81f5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.503+0000 7f02bdc29640 1 --2- 172.21.15.76:0/1778048355 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02b8152480 0x7f02b81f5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.503+0000 7f02bdc29640 1 -- 172.21.15.76:0/1778048355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02a4005270 msgr2=0x7f02b8160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.503+0000 7f02bdc29640 1 --2- 172.21.15.76:0/1778048355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02a4005270 0x7f02b8160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.503+0000 7f02bdc29640 1 -- 172.21.15.76:0/1778048355 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b8161070 con 0x7f02b8152890 2024-04-25T19:19:37.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.503+0000 7f02bcc27640 1 --2- 172.21.15.76:0/1778048355 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f02b8152480 0x7f02b81f5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:19:37.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.503+0000 7f02bd428640 1 --2- 172.21.15.76:0/1778048355 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02a4005270 0x7f02b8160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:19:37.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.503+0000 7f02bdc29640 1 --2- 172.21.15.76:0/1778048355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02b8152890 0x7f02b81f6180 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f02b000e200 tx=0x7f02b0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:37.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.503+0000 7f02adffb640 1 -- 172.21.15.76:0/1778048355 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b003f030 con 0x7f02b8152890 2024-04-25T19:19:37.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.503+0000 7f02adffb640 1 -- 172.21.15.76:0/1778048355 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02b0003990 con 0x7f02b8152890 2024-04-25T19:19:37.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.503+0000 7f02bf68c640 1 -- 172.21.15.76:0/1778048355 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b815eec0 con 0x7f02b8152890 2024-04-25T19:19:37.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.503+0000 7f02adffb640 1 -- 172.21.15.76:0/1778048355 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b0035070 con 0x7f02b8152890 2024-04-25T19:19:37.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.503+0000 7f02bf68c640 1 -- 172.21.15.76:0/1778048355 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b815e350 con 0x7f02b8152890 2024-04-25T19:19:37.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.504+0000 7f02bf68c640 1 -- 172.21.15.76:0/1778048355 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f02b806a390 con 0x7f02b8152890 2024-04-25T19:19:37.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.505+0000 7f02adffb640 1 -- 172.21.15.76:0/1778048355 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f02b0042030 con 0x7f02b8152890 2024-04-25T19:19:37.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.506+0000 7f02adffb640 1 --2- 172.21.15.76:0/1778048355 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f029c0744b0 0x7f029c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.506+0000 7f02adffb640 1 -- 172.21.15.76:0/1778048355 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f02b00b5f60 con 0x7f02b8152890 2024-04-25T19:19:37.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.506+0000 7f02adffb640 1 --2- 172.21.15.76:0/1778048355 >> [v2:172.21.15.76:6824/3418257163,v1:172.21.15.76:6825/3418257163] conn(0x7f029c07c880 0x7f029c07ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.506+0000 7f02adffb640 1 -- 172.21.15.76:0/1778048355 --> [v2:172.21.15.76:6824/3418257163,v1:172.21.15.76:6825/3418257163] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f02b0032ea0 con 0x7f029c07c880 2024-04-25T19:19:37.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.506+0000 7f02adffb640 1 -- 172.21.15.76:0/1778048355 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f02b00b63f0 con 0x7f02b8152890 2024-04-25T19:19:37.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.506+0000 7f02bd428640 1 --2- 172.21.15.76:0/1778048355 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f029c0744b0 0x7f029c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.506+0000 7f02bcc27640 1 --2- 172.21.15.76:0/1778048355 >> [v2:172.21.15.76:6824/3418257163,v1:172.21.15.76:6825/3418257163] conn(0x7f029c07c880 0x7f029c07ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.506+0000 7f02bcc27640 1 --2- 172.21.15.76:0/1778048355 >> [v2:172.21.15.76:6824/3418257163,v1:172.21.15.76:6825/3418257163] conn(0x7f029c07c880 0x7f029c07ed50 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:37.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.506+0000 7f02bd428640 1 --2- 172.21.15.76:0/1778048355 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f029c0744b0 0x7f029c0769c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f02b40015d0 tx=0x7f02b4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:37.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.507+0000 7f02adffb640 1 -- 172.21.15.76:0/1778048355 <== osd.3 v2:172.21.15.76:6824/3418257163 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f02b0032ea0 con 0x7f029c07c880 2024-04-25T19:19:37.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.520+0000 7f02bf68c640 1 -- 172.21.15.76:0/1778048355 --> [v2:172.21.15.76:6824/3418257163,v1:172.21.15.76:6825/3418257163] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f02b80b7e70 con 0x7f029c07c880 2024-04-25T19:19:37.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.520+0000 7f02adffb640 1 -- 172.21.15.76:0/1778048355 <== osd.3 v2:172.21.15.76:6824/3418257163 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f02b80b7e70 con 0x7f029c07c880 2024-04-25T19:19:37.522 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:19:37.522 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T19:19:37.522 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:19:37.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.521+0000 7f02bf68c640 1 -- 172.21.15.76:0/1778048355 >> [v2:172.21.15.76:6824/3418257163,v1:172.21.15.76:6825/3418257163] conn(0x7f029c07c880 msgr2=0x7f029c07ed50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.521+0000 7f02bf68c640 1 --2- 172.21.15.76:0/1778048355 >> [v2:172.21.15.76:6824/3418257163,v1:172.21.15.76:6825/3418257163] conn(0x7f029c07c880 0x7f029c07ed50 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.521+0000 7f02bf68c640 1 -- 172.21.15.76:0/1778048355 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f029c0744b0 msgr2=0x7f029c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.521+0000 7f02bf68c640 1 --2- 172.21.15.76:0/1778048355 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f029c0744b0 0x7f029c0769c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f02b40015d0 tx=0x7f02b4009040 comp rx=0 tx=0).stop 2024-04-25T19:19:37.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.521+0000 7f02bf68c640 1 -- 172.21.15.76:0/1778048355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02b8152890 msgr2=0x7f02b81f6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.521+0000 7f02bf68c640 1 --2- 172.21.15.76:0/1778048355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02b8152890 0x7f02b81f6180 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f02b000e200 tx=0x7f02b0009750 comp rx=0 tx=0).stop 2024-04-25T19:19:37.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.521+0000 7f02bdc29640 1 -- 172.21.15.76:0/1778048355 reap_dead start 2024-04-25T19:19:37.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.521+0000 7f02bf68c640 1 -- 172.21.15.76:0/1778048355 shutdown_connections 2024-04-25T19:19:37.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.521+0000 7f02bf68c640 1 -- 172.21.15.76:0/1778048355 >> 172.21.15.76:0/1778048355 conn(0x7f02b8068900 msgr2=0x7f02b8075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:37.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.521+0000 7f02bf68c640 1 -- 172.21.15.76:0/1778048355 shutdown_connections 2024-04-25T19:19:37.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.521+0000 7f02bf68c640 1 -- 172.21.15.76:0/1778048355 wait complete. 2024-04-25T19:19:37.533 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T19:19:37.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.641+0000 7f956fd55640 1 Processor -- start 2024-04-25T19:19:37.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.641+0000 7f956fd55640 1 -- start start 2024-04-25T19:19:37.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.642+0000 7f956fd55640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f956806e0c0 0x7f956806e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.642+0000 7f956fd55640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f956815d250 con 0x7f95681609c0 2024-04-25T19:19:37.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.642+0000 7f956fd55640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f956815d510 con 0x7f956815cd10 2024-04-25T19:19:37.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.642+0000 7f956fd55640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f956815d7f0 con 0x7f956806e0c0 2024-04-25T19:19:37.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.642+0000 7f956e2f2640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f95681609c0 0x7f9568072ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35844/0 (socket says 172.21.15.76:35844) 2024-04-25T19:19:37.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.642+0000 7f956e2f2640 1 -- 172.21.15.76:0/812872903 learned_addr learned my addr 172.21.15.76:0/812872903 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:37.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.643+0000 7f956d2f0640 1 --2- 172.21.15.76:0/812872903 >> v2:172.21.15.105:3300/0 conn(0x7f956806e0c0 0x7f956806e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.643+0000 7f956caef640 1 -- 172.21.15.76:0/812872903 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3255323569 0 0) 0x7f956815d250 con 0x7f95681609c0 2024-04-25T19:19:37.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.643+0000 7f956caef640 1 -- 172.21.15.76:0/812872903 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9554003520 con 0x7f95681609c0 2024-04-25T19:19:37.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.643+0000 7f956d2f0640 1 -- 172.21.15.76:0/812872903 >> v1:172.21.15.105:6789/0 conn(0x7f956815cd10 legacy=0x7f9568156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.643+0000 7f956d2f0640 1 -- 172.21.15.76:0/812872903 >> v1:172.21.15.76:6789/0 conn(0x7f95681609c0 legacy=0x7f9568072ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.643+0000 7f956d2f0640 1 -- 172.21.15.76:0/812872903 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956815dad0 con 0x7f956806e0c0 2024-04-25T19:19:37.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.644+0000 7f956d2f0640 1 --2- 172.21.15.76:0/812872903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f956806e0c0 0x7f956806e4d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f95580097a0 tx=0x7f955802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffebc49f9fc9d73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:37.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.644+0000 7f956caef640 1 -- 172.21.15.76:0/812872903 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9558037070 con 0x7f956806e0c0 2024-04-25T19:19:37.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.644+0000 7f956caef640 1 -- 172.21.15.76:0/812872903 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9558032440 con 0x7f956806e0c0 2024-04-25T19:19:37.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.644+0000 7f956caef640 1 -- 172.21.15.76:0/812872903 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95580327f0 con 0x7f956806e0c0 2024-04-25T19:19:37.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.645+0000 7f956fd55640 1 -- 172.21.15.76:0/812872903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f956806e0c0 msgr2=0x7f956806e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.645+0000 7f956fd55640 1 --2- 172.21.15.76:0/812872903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f956806e0c0 0x7f956806e4d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f95580097a0 tx=0x7f955802a230 comp rx=0 tx=0).stop 2024-04-25T19:19:37.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.645+0000 7f956fd55640 1 -- 172.21.15.76:0/812872903 shutdown_connections 2024-04-25T19:19:37.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.645+0000 7f956fd55640 1 --2- 172.21.15.76:0/812872903 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f956806e0c0 0x7f956806e4d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.645+0000 7f956fd55640 1 -- 172.21.15.76:0/812872903 >> 172.21.15.76:0/812872903 conn(0x7f9568070b00 msgr2=0x7f9568070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:37.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.645+0000 7f956fd55640 1 -- 172.21.15.76:0/812872903 shutdown_connections 2024-04-25T19:19:37.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.645+0000 7f956fd55640 1 -- 172.21.15.76:0/812872903 wait complete. 2024-04-25T19:19:37.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.646+0000 7f956fd55640 1 Processor -- start 2024-04-25T19:19:37.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.646+0000 7f956fd55640 1 -- start start 2024-04-25T19:19:37.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.648+0000 7f956fd55640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f956806e0c0 0x7f95681f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.648+0000 7f956fd55640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f956815cd10 0x7f9568153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.648+0000 7f956fd55640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95681609c0 0x7f9568149bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.648+0000 7f956fd55640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f95680c5390 con 0x7f956815cd10 2024-04-25T19:19:37.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.648+0000 7f956fd55640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f95680c5210 con 0x7f95681609c0 2024-04-25T19:19:37.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.648+0000 7f956fd55640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f95680731e0 con 0x7f956806e0c0 2024-04-25T19:19:37.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.648+0000 7f956d2f0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f956815cd10 0x7f9568153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.648+0000 7f956d2f0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f956815cd10 0x7f9568153dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50732/0 (socket says 172.21.15.76:50732) 2024-04-25T19:19:37.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.648+0000 7f956d2f0640 1 -- 172.21.15.76:0/1225886388 learned_addr learned my addr 172.21.15.76:0/1225886388 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:37.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.648+0000 7f956e2f2640 1 --2- 172.21.15.76:0/1225886388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95681609c0 0x7f9568149bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.648+0000 7f956daf1640 1 --2- 172.21.15.76:0/1225886388 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f956806e0c0 0x7f95681f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.648+0000 7f956d2f0640 1 -- 172.21.15.76:0/1225886388 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f956806e0c0 msgr2=0x7f95681f34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.648+0000 7f956d2f0640 1 --2- 172.21.15.76:0/1225886388 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f956806e0c0 0x7f95681f34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.648+0000 7f956d2f0640 1 -- 172.21.15.76:0/1225886388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95681609c0 msgr2=0x7f9568149bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.648+0000 7f956d2f0640 1 --2- 172.21.15.76:0/1225886388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95681609c0 0x7f9568149bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.648+0000 7f956d2f0640 1 -- 172.21.15.76:0/1225886388 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9568154300 con 0x7f956815cd10 2024-04-25T19:19:37.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.648+0000 7f956d2f0640 1 --2- 172.21.15.76:0/1225886388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f956815cd10 0x7f9568153dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f95580097a0 tx=0x7f95580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:37.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.648+0000 7f955e7fc640 1 -- 172.21.15.76:0/1225886388 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9558037050 con 0x7f956815cd10 2024-04-25T19:19:37.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.648+0000 7f955e7fc640 1 -- 172.21.15.76:0/1225886388 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9558038040 con 0x7f956815cd10 2024-04-25T19:19:37.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.648+0000 7f955e7fc640 1 -- 172.21.15.76:0/1225886388 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9558004470 con 0x7f956815cd10 2024-04-25T19:19:37.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.648+0000 7f956fd55640 1 -- 172.21.15.76:0/1225886388 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9568153000 con 0x7f956815cd10 2024-04-25T19:19:37.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.648+0000 7f956fd55640 1 -- 172.21.15.76:0/1225886388 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95681535c0 con 0x7f956815cd10 2024-04-25T19:19:37.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.649+0000 7f956fd55640 1 -- 172.21.15.76:0/1225886388 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f956806a390 con 0x7f956815cd10 2024-04-25T19:19:37.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.650+0000 7f955e7fc640 1 -- 172.21.15.76:0/1225886388 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f955803c6d0 con 0x7f956815cd10 2024-04-25T19:19:37.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.650+0000 7f955e7fc640 1 --2- 172.21.15.76:0/1225886388 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f954c0744b0 0x7f954c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.650+0000 7f955e7fc640 1 -- 172.21.15.76:0/1225886388 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f95580b5c40 con 0x7f956815cd10 2024-04-25T19:19:37.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.651+0000 7f956daf1640 1 --2- 172.21.15.76:0/1225886388 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f954c0744b0 0x7f954c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.651+0000 7f956daf1640 1 --2- 172.21.15.76:0/1225886388 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f954c0744b0 0x7f954c0769c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9564001500 tx=0x7f9564009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:37.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.655+0000 7f955e7fc640 1 -- 172.21.15.76:0/1225886388 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f95580802d0 con 0x7f956815cd10 2024-04-25T19:19:37.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.837+0000 7f956fd55640 1 -- 172.21.15.76:0/1225886388 --> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f956815e160 con 0x7f954c0744b0 2024-04-25T19:19:37.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.838+0000 7f955e7fc640 1 -- 172.21.15.76:0/1225886388 <== mgr.4109 v2:172.21.15.105:6832/2585775531 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f956815e160 con 0x7f954c0744b0 2024-04-25T19:19:37.839 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T19:19:37.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.840+0000 7f956fd55640 1 -- 172.21.15.76:0/1225886388 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f954c0744b0 msgr2=0x7f954c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.840+0000 7f956fd55640 1 --2- 172.21.15.76:0/1225886388 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f954c0744b0 0x7f954c0769c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9564001500 tx=0x7f9564009040 comp rx=0 tx=0).stop 2024-04-25T19:19:37.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.840+0000 7f956fd55640 1 -- 172.21.15.76:0/1225886388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f956815cd10 msgr2=0x7f9568153dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.840+0000 7f956fd55640 1 --2- 172.21.15.76:0/1225886388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f956815cd10 0x7f9568153dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f95580097a0 tx=0x7f95580046c0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.840+0000 7f956fd55640 1 -- 172.21.15.76:0/1225886388 shutdown_connections 2024-04-25T19:19:37.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.840+0000 7f956fd55640 1 --2- 172.21.15.76:0/1225886388 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f954c0744b0 0x7f954c0769c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.840+0000 7f956fd55640 1 --2- 172.21.15.76:0/1225886388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f95681609c0 0x7f9568149bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.840+0000 7f956fd55640 1 --2- 172.21.15.76:0/1225886388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f956815cd10 0x7f9568153dc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.840+0000 7f956fd55640 1 --2- 172.21.15.76:0/1225886388 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f956806e0c0 0x7f95681f34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.840+0000 7f956fd55640 1 -- 172.21.15.76:0/1225886388 >> 172.21.15.76:0/1225886388 conn(0x7f9568070b00 msgr2=0x7f95680b5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:37.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.840+0000 7f956fd55640 1 -- 172.21.15.76:0/1225886388 shutdown_connections 2024-04-25T19:19:37.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.840+0000 7f956fd55640 1 -- 172.21.15.76:0/1225886388 wait complete. 2024-04-25T19:19:37.853 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T19:19:37.853 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T19:19:37.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.966+0000 7f6288f2e640 1 Processor -- start 2024-04-25T19:19:37.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.966+0000 7f6288f2e640 1 -- start start 2024-04-25T19:19:37.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.967+0000 7f6288f2e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62840d0810 0x7f628406f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.967+0000 7f6288f2e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f62840cea70 0x7f62840cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.967+0000 7f6288f2e640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f628406e160 0x7f628406e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.967+0000 7f6288f2e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f628415ce80 con 0x7f62840cea70 2024-04-25T19:19:37.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.967+0000 7f6288f2e640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f628415d160 con 0x7f628406e160 2024-04-25T19:19:37.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.967+0000 7f6288f2e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f628415d440 con 0x7f62840d0810 2024-04-25T19:19:37.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.967+0000 7f627bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f62840cea70 0x7f62840cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.967+0000 7f627bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f62840cea70 0x7f62840cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50746/0 (socket says 172.21.15.76:50746) 2024-04-25T19:19:37.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.967+0000 7f627bfff640 1 -- 172.21.15.76:0/4074349366 learned_addr learned my addr 172.21.15.76:0/4074349366 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:37.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.967+0000 7f6282d9d640 1 --2- 172.21.15.76:0/4074349366 >> v2:172.21.15.105:3300/0 conn(0x7f628406e160 0x7f628406e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.967+0000 7f628259c640 1 --2- 172.21.15.76:0/4074349366 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62840d0810 0x7f628406f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.968+0000 7f627bfff640 1 -- 172.21.15.76:0/4074349366 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62840d0810 msgr2=0x7f628406f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:37.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.968+0000 7f627bfff640 1 --2- 172.21.15.76:0/4074349366 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62840d0810 0x7f628406f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.968+0000 7f627bfff640 1 -- 172.21.15.76:0/4074349366 >> v2:172.21.15.105:3300/0 conn(0x7f628406e160 msgr2=0x7f628406e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:37.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.968+0000 7f627bfff640 1 --2- 172.21.15.76:0/4074349366 >> v2:172.21.15.105:3300/0 conn(0x7f628406e160 0x7f628406e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.968+0000 7f627bfff640 1 -- 172.21.15.76:0/4074349366 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628415d720 con 0x7f62840cea70 2024-04-25T19:19:37.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.968+0000 7f6282d9d640 1 --2- 172.21.15.76:0/4074349366 >> v2:172.21.15.105:3300/0 conn(0x7f628406e160 0x7f628406e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:19:37.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.968+0000 7f627bfff640 1 --2- 172.21.15.76:0/4074349366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62840cea70 0x7f62840cee80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6284074d10 tx=0x7f626c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1161585fd69942a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:37.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.968+0000 7f6281d9b640 1 -- 172.21.15.76:0/4074349366 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626c037070 con 0x7f62840cea70 2024-04-25T19:19:37.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.969+0000 7f6281d9b640 1 -- 172.21.15.76:0/4074349366 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f626c032440 con 0x7f62840cea70 2024-04-25T19:19:37.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.969+0000 7f6281d9b640 1 -- 172.21.15.76:0/4074349366 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626c032810 con 0x7f62840cea70 2024-04-25T19:19:37.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.969+0000 7f6288f2e640 1 -- 172.21.15.76:0/4074349366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62840cea70 msgr2=0x7f62840cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.969+0000 7f6288f2e640 1 --2- 172.21.15.76:0/4074349366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62840cea70 0x7f62840cee80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6284074d10 tx=0x7f626c02a1e0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.969+0000 7f6288f2e640 1 -- 172.21.15.76:0/4074349366 shutdown_connections 2024-04-25T19:19:37.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.969+0000 7f6288f2e640 1 --2- 172.21.15.76:0/4074349366 >> v2:172.21.15.105:3300/0 conn(0x7f628406e160 0x7f628406e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.969+0000 7f6288f2e640 1 --2- 172.21.15.76:0/4074349366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62840cea70 0x7f62840cee80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.969+0000 7f6288f2e640 1 --2- 172.21.15.76:0/4074349366 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62840d0810 0x7f628406f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.969+0000 7f6288f2e640 1 -- 172.21.15.76:0/4074349366 >> 172.21.15.76:0/4074349366 conn(0x7f62840b3c00 msgr2=0x7f62840b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:37.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.969+0000 7f6288f2e640 1 -- 172.21.15.76:0/4074349366 shutdown_connections 2024-04-25T19:19:37.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.969+0000 7f6288f2e640 1 -- 172.21.15.76:0/4074349366 wait complete. 2024-04-25T19:19:37.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.971+0000 7f6288f2e640 1 Processor -- start 2024-04-25T19:19:37.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.971+0000 7f6288f2e640 1 -- start start 2024-04-25T19:19:37.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.972+0000 7f6288f2e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f628406e160 0x7f62841f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.972+0000 7f6288f2e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62840cea70 0x7f6284158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.972+0000 7f6288f2e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62840d0810 0x7f62841562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.972+0000 7f6288f2e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f628406eab0 con 0x7f62840d0810 2024-04-25T19:19:37.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.972+0000 7f6288f2e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f628406dc40 con 0x7f628406e160 2024-04-25T19:19:37.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.972+0000 7f6288f2e640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f628415cb70 con 0x7f62840cea70 2024-04-25T19:19:37.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.973+0000 7f6282d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62840d0810 0x7f62841562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.973+0000 7f6282d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62840d0810 0x7f62841562d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50756/0 (socket says 172.21.15.76:50756) 2024-04-25T19:19:37.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.973+0000 7f6282d9d640 1 -- 172.21.15.76:0/878617326 learned_addr learned my addr 172.21.15.76:0/878617326 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:37.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.973+0000 7f627bfff640 1 --2- 172.21.15.76:0/878617326 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62840cea70 0x7f6284158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.973+0000 7f628259c640 1 --2- 172.21.15.76:0/878617326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f628406e160 0x7f62841f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.973+0000 7f6282d9d640 1 -- 172.21.15.76:0/878617326 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62840cea70 msgr2=0x7f6284158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.973+0000 7f6282d9d640 1 --2- 172.21.15.76:0/878617326 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f62840cea70 0x7f6284158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.973+0000 7f6282d9d640 1 -- 172.21.15.76:0/878617326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f628406e160 msgr2=0x7f62841f6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.973+0000 7f6282d9d640 1 --2- 172.21.15.76:0/878617326 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f628406e160 0x7f62841f6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.973+0000 7f6282d9d640 1 -- 172.21.15.76:0/878617326 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6284160370 con 0x7f62840d0810 2024-04-25T19:19:37.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.973+0000 7f6282d9d640 1 --2- 172.21.15.76:0/878617326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62840d0810 0x7f62841562d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f627c007d80 tx=0x7f627c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:37.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.973+0000 7f627affd640 1 -- 172.21.15.76:0/878617326 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c01a070 con 0x7f62840d0810 2024-04-25T19:19:37.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.973+0000 7f627affd640 1 -- 172.21.15.76:0/878617326 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f627c003990 con 0x7f62840d0810 2024-04-25T19:19:37.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.973+0000 7f6288f2e640 1 -- 172.21.15.76:0/878617326 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6284160650 con 0x7f62840d0810 2024-04-25T19:19:37.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.973+0000 7f627affd640 1 -- 172.21.15.76:0/878617326 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c003d60 con 0x7f62840d0810 2024-04-25T19:19:37.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.973+0000 7f6288f2e640 1 -- 172.21.15.76:0/878617326 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62841f69a0 con 0x7f62840d0810 2024-04-25T19:19:37.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.974+0000 7f6288f2e640 1 -- 172.21.15.76:0/878617326 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f62841f7400 con 0x7f62840d0810 2024-04-25T19:19:37.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.975+0000 7f627affd640 1 -- 172.21.15.76:0/878617326 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f627c01d070 con 0x7f62840d0810 2024-04-25T19:19:37.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.975+0000 7f627affd640 1 --2- 172.21.15.76:0/878617326 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f62640744e0 0x7f62640769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.975+0000 7f627affd640 1 -- 172.21.15.76:0/878617326 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f627c095410 con 0x7f62840d0810 2024-04-25T19:19:37.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.976+0000 7f628259c640 1 --2- 172.21.15.76:0/878617326 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f62640744e0 0x7f62640769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.976+0000 7f627affd640 1 --2- 172.21.15.76:0/878617326 >> [v2:172.21.15.105:6800/2125002468,v1:172.21.15.105:6801/2125002468] conn(0x7f626407c8b0 0x7f626407ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:37.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.976+0000 7f627affd640 1 -- 172.21.15.76:0/878617326 --> [v2:172.21.15.105:6800/2125002468,v1:172.21.15.105:6801/2125002468] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f626407f450 con 0x7f626407c8b0 2024-04-25T19:19:37.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.976+0000 7f627affd640 1 -- 172.21.15.76:0/878617326 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f627c09c050 con 0x7f62840d0810 2024-04-25T19:19:37.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.976+0000 7f627bfff640 1 --2- 172.21.15.76:0/878617326 >> [v2:172.21.15.105:6800/2125002468,v1:172.21.15.105:6801/2125002468] conn(0x7f626407c8b0 0x7f626407ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:37.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.976+0000 7f628259c640 1 --2- 172.21.15.76:0/878617326 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f62640744e0 0x7f62640769f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f62740079a0 tx=0x7f6274008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:37.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.976+0000 7f627bfff640 1 --2- 172.21.15.76:0/878617326 >> [v2:172.21.15.105:6800/2125002468,v1:172.21.15.105:6801/2125002468] conn(0x7f626407c8b0 0x7f626407ed80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:37.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.977+0000 7f627affd640 1 -- 172.21.15.76:0/878617326 <== osd.4 v2:172.21.15.105:6800/2125002468 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f626407f450 con 0x7f626407c8b0 2024-04-25T19:19:37.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.990+0000 7f6288f2e640 1 -- 172.21.15.76:0/878617326 --> [v2:172.21.15.105:6800/2125002468,v1:172.21.15.105:6801/2125002468] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6284160120 con 0x7f626407c8b0 2024-04-25T19:19:37.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.991+0000 7f627affd640 1 -- 172.21.15.76:0/878617326 <== osd.4 v2:172.21.15.105:6800/2125002468 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6284160120 con 0x7f626407c8b0 2024-04-25T19:19:37.992 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:19:37.992 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T19:19:37.992 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:19:37.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.991+0000 7f6288f2e640 1 -- 172.21.15.76:0/878617326 >> [v2:172.21.15.105:6800/2125002468,v1:172.21.15.105:6801/2125002468] conn(0x7f626407c8b0 msgr2=0x7f626407ed80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.991+0000 7f6288f2e640 1 --2- 172.21.15.76:0/878617326 >> [v2:172.21.15.105:6800/2125002468,v1:172.21.15.105:6801/2125002468] conn(0x7f626407c8b0 0x7f626407ed80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.991+0000 7f6288f2e640 1 -- 172.21.15.76:0/878617326 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f62640744e0 msgr2=0x7f62640769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.991+0000 7f6288f2e640 1 --2- 172.21.15.76:0/878617326 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f62640744e0 0x7f62640769f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f62740079a0 tx=0x7f6274008040 comp rx=0 tx=0).stop 2024-04-25T19:19:37.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.991+0000 7f6288f2e640 1 -- 172.21.15.76:0/878617326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62840d0810 msgr2=0x7f62841562d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:37.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.991+0000 7f6288f2e640 1 --2- 172.21.15.76:0/878617326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62840d0810 0x7f62841562d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f627c007d80 tx=0x7f627c00d4a0 comp rx=0 tx=0).stop 2024-04-25T19:19:37.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.991+0000 7f6282d9d640 1 -- 172.21.15.76:0/878617326 reap_dead start 2024-04-25T19:19:37.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.991+0000 7f6288f2e640 1 -- 172.21.15.76:0/878617326 shutdown_connections 2024-04-25T19:19:37.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.991+0000 7f6288f2e640 1 -- 172.21.15.76:0/878617326 >> 172.21.15.76:0/878617326 conn(0x7f62840b3c00 msgr2=0x7f62840c9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:37.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.991+0000 7f6288f2e640 1 -- 172.21.15.76:0/878617326 shutdown_connections 2024-04-25T19:19:37.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:37.991+0000 7f6288f2e640 1 -- 172.21.15.76:0/878617326 wait complete. 2024-04-25T19:19:38.003 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T19:19:38.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.111+0000 7ff90a692640 1 Processor -- start 2024-04-25T19:19:38.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.111+0000 7ff90a692640 1 -- start start 2024-04-25T19:19:38.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.113+0000 7ff90a692640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff9040ccb70 0x7ff9040ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:38.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.113+0000 7ff90a692640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff90415dd00 con 0x7ff9040d0b10 2024-04-25T19:19:38.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.113+0000 7ff90a692640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff90415cea0 con 0x7ff904157200 2024-04-25T19:19:38.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.113+0000 7ff90a692640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff9040d0460 con 0x7ff9040ccb70 2024-04-25T19:19:38.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.113+0000 7ff908c2f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff9040ccb70 0x7ff9040ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:38.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.113+0000 7ff9037fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff9040d0b10 0x7ff9040cee90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35850/0 (socket says 172.21.15.76:35850) 2024-04-25T19:19:38.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.113+0000 7ff9037fe640 1 -- 172.21.15.76:0/4094367140 learned_addr learned my addr 172.21.15.76:0/4094367140 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:38.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.113+0000 7ff908c2f640 1 -- 172.21.15.76:0/4094367140 >> v1:172.21.15.105:6789/0 conn(0x7ff904157200 legacy=0x7ff9040cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.113+0000 7ff908c2f640 1 -- 172.21.15.76:0/4094367140 >> v1:172.21.15.76:6789/0 conn(0x7ff9040d0b10 legacy=0x7ff9040cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.113+0000 7ff908c2f640 1 -- 172.21.15.76:0/4094367140 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff90415d180 con 0x7ff9040ccb70 2024-04-25T19:19:38.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.113+0000 7ff902ffd640 1 -- 172.21.15.76:0/4094367140 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2344940534 0 0) 0x7ff90415dd00 con 0x7ff9040d0b10 2024-04-25T19:19:38.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.114+0000 7ff908c2f640 1 --2- 172.21.15.76:0/4094367140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9040ccb70 0x7ff9040ccf80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc009d90 tx=0x7ff8fc02a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b445f538e8b7d395 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:38.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.114+0000 7ff902ffd640 1 -- 172.21.15.76:0/4094367140 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc037070 con 0x7ff9040ccb70 2024-04-25T19:19:38.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.114+0000 7ff902ffd640 1 -- 172.21.15.76:0/4094367140 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8fc007740 con 0x7ff9040ccb70 2024-04-25T19:19:38.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.114+0000 7ff902ffd640 1 -- 172.21.15.76:0/4094367140 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc007b10 con 0x7ff9040ccb70 2024-04-25T19:19:38.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.115+0000 7ff90a692640 1 -- 172.21.15.76:0/4094367140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9040ccb70 msgr2=0x7ff9040ccf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.115+0000 7ff90a692640 1 --2- 172.21.15.76:0/4094367140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9040ccb70 0x7ff9040ccf80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc009d90 tx=0x7ff8fc02a780 comp rx=0 tx=0).stop 2024-04-25T19:19:38.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.115+0000 7ff90a692640 1 -- 172.21.15.76:0/4094367140 shutdown_connections 2024-04-25T19:19:38.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.115+0000 7ff90a692640 1 --2- 172.21.15.76:0/4094367140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9040ccb70 0x7ff9040ccf80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:38.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.115+0000 7ff90a692640 1 -- 172.21.15.76:0/4094367140 >> 172.21.15.76:0/4094367140 conn(0x7ff9040d2370 msgr2=0x7ff9040d2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:38.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.115+0000 7ff90a692640 1 -- 172.21.15.76:0/4094367140 shutdown_connections 2024-04-25T19:19:38.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.115+0000 7ff90a692640 1 -- 172.21.15.76:0/4094367140 wait complete. 2024-04-25T19:19:38.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.116+0000 7ff90a692640 1 Processor -- start 2024-04-25T19:19:38.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.117+0000 7ff90a692640 1 -- start start 2024-04-25T19:19:38.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.118+0000 7ff90a692640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9040ccb70 0x7ff9041f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:38.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.118+0000 7ff90a692640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9040d0b10 0x7ff9041e9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:38.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.118+0000 7ff90a692640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff904157200 0x7ff904158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:38.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.118+0000 7ff90a692640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff90415da20 con 0x7ff9040d0b10 2024-04-25T19:19:38.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.118+0000 7ff90a692640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff90415d740 con 0x7ff904157200 2024-04-25T19:19:38.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.118+0000 7ff90a692640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff9040cd4c0 con 0x7ff9040ccb70 2024-04-25T19:19:38.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.118+0000 7ff9037fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9040d0b10 0x7ff9041e9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:38.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.118+0000 7ff9037fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9040d0b10 0x7ff9041e9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50786/0 (socket says 172.21.15.76:50786) 2024-04-25T19:19:38.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.118+0000 7ff9037fe640 1 -- 172.21.15.76:0/2060755686 learned_addr learned my addr 172.21.15.76:0/2060755686 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:38.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.118+0000 7ff908c2f640 1 --2- 172.21.15.76:0/2060755686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff904157200 0x7ff904158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:38.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.118+0000 7ff903fff640 1 --2- 172.21.15.76:0/2060755686 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9040ccb70 0x7ff9041f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:38.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.118+0000 7ff9037fe640 1 -- 172.21.15.76:0/2060755686 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9040ccb70 msgr2=0x7ff9041f2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.118+0000 7ff9037fe640 1 --2- 172.21.15.76:0/2060755686 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9040ccb70 0x7ff9041f2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:38.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.118+0000 7ff9037fe640 1 -- 172.21.15.76:0/2060755686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff904157200 msgr2=0x7ff904158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.118+0000 7ff9037fe640 1 --2- 172.21.15.76:0/2060755686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff904157200 0x7ff904158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:38.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.118+0000 7ff9037fe640 1 -- 172.21.15.76:0/2060755686 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9041e6db0 con 0x7ff9040d0b10 2024-04-25T19:19:38.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.118+0000 7ff9037fe640 1 --2- 172.21.15.76:0/2060755686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9040d0b10 0x7ff9041e9bc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800c3c0 tx=0x7ff8f800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:38.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.118+0000 7ff900ff9640 1 -- 172.21.15.76:0/2060755686 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f80074e0 con 0x7ff9040d0b10 2024-04-25T19:19:38.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.119+0000 7ff900ff9640 1 -- 172.21.15.76:0/2060755686 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8f80047e0 con 0x7ff9040d0b10 2024-04-25T19:19:38.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.119+0000 7ff900ff9640 1 -- 172.21.15.76:0/2060755686 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f8004bd0 con 0x7ff9040d0b10 2024-04-25T19:19:38.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.119+0000 7ff90a692640 1 -- 172.21.15.76:0/2060755686 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9040d0200 con 0x7ff9040d0b10 2024-04-25T19:19:38.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.119+0000 7ff90a692640 1 -- 172.21.15.76:0/2060755686 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9041e5d30 con 0x7ff9040d0b10 2024-04-25T19:19:38.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.120+0000 7ff90a692640 1 -- 172.21.15.76:0/2060755686 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff90406a390 con 0x7ff9040d0b10 2024-04-25T19:19:38.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.120+0000 7ff900ff9640 1 -- 172.21.15.76:0/2060755686 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff8f8011070 con 0x7ff9040d0b10 2024-04-25T19:19:38.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.121+0000 7ff900ff9640 1 --2- 172.21.15.76:0/2060755686 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff8f00744b0 0x7ff8f00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:38.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.121+0000 7ff900ff9640 1 -- 172.21.15.76:0/2060755686 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7ff8f8094c40 con 0x7ff9040d0b10 2024-04-25T19:19:38.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.121+0000 7ff903fff640 1 --2- 172.21.15.76:0/2060755686 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff8f00744b0 0x7ff8f00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:38.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.122+0000 7ff903fff640 1 --2- 172.21.15.76:0/2060755686 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff8f00744b0 0x7ff8f00769c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4000f60 tx=0x7ff8f4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:38.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.125+0000 7ff900ff9640 1 -- 172.21.15.76:0/2060755686 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff8f805ee20 con 0x7ff9040d0b10 2024-04-25T19:19:38.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.301+0000 7ff90a692640 1 -- 172.21.15.76:0/2060755686 --> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7ff9040b7ed0 con 0x7ff8f00744b0 2024-04-25T19:19:38.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.302+0000 7ff900ff9640 1 -- 172.21.15.76:0/2060755686 <== mgr.4109 v2:172.21.15.105:6832/2585775531 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff9040b7ed0 con 0x7ff8f00744b0 2024-04-25T19:19:38.303 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T19:19:38.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.304+0000 7ff90a692640 1 -- 172.21.15.76:0/2060755686 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff8f00744b0 msgr2=0x7ff8f00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.304+0000 7ff90a692640 1 --2- 172.21.15.76:0/2060755686 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff8f00744b0 0x7ff8f00769c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4000f60 tx=0x7ff8f4009040 comp rx=0 tx=0).stop 2024-04-25T19:19:38.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.304+0000 7ff90a692640 1 -- 172.21.15.76:0/2060755686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9040d0b10 msgr2=0x7ff9041e9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.304+0000 7ff90a692640 1 --2- 172.21.15.76:0/2060755686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9040d0b10 0x7ff9041e9bc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800c3c0 tx=0x7ff8f800c880 comp rx=0 tx=0).stop 2024-04-25T19:19:38.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.304+0000 7ff90a692640 1 -- 172.21.15.76:0/2060755686 shutdown_connections 2024-04-25T19:19:38.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.304+0000 7ff90a692640 1 --2- 172.21.15.76:0/2060755686 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff8f00744b0 0x7ff8f00769c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:38.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.304+0000 7ff90a692640 1 --2- 172.21.15.76:0/2060755686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff904157200 0x7ff904158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:38.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.304+0000 7ff90a692640 1 --2- 172.21.15.76:0/2060755686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9040d0b10 0x7ff9041e9bc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:38.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.304+0000 7ff90a692640 1 --2- 172.21.15.76:0/2060755686 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff9040ccb70 0x7ff9041f2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:38.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.304+0000 7ff90a692640 1 -- 172.21.15.76:0/2060755686 >> 172.21.15.76:0/2060755686 conn(0x7ff9040d2370 msgr2=0x7ff90406f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:38.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.304+0000 7ff90a692640 1 -- 172.21.15.76:0/2060755686 shutdown_connections 2024-04-25T19:19:38.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.304+0000 7ff90a692640 1 -- 172.21.15.76:0/2060755686 wait complete. 2024-04-25T19:19:38.317 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T19:19:38.317 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T19:19:38.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.427+0000 7fe0acc51640 1 Processor -- start 2024-04-25T19:19:38.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.427+0000 7fe0acc51640 1 -- start start 2024-04-25T19:19:38.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.428+0000 7fe0acc51640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe0a80c9330 0x7fe0a80c4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:38.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.428+0000 7fe0acc51640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0a80c8cc0 con 0x7fe0a8074280 2024-04-25T19:19:38.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.428+0000 7fe0acc51640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0a80c9b70 con 0x7fe0a8074690 2024-04-25T19:19:38.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.428+0000 7fe0acc51640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe0a80c4ea0 con 0x7fe0a80c9330 2024-04-25T19:19:38.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.429+0000 7fe0a659c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe0a80c9330 0x7fe0a80c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:38.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.429+0000 7fe0a5d9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe0a8074280 0x7fe0a806f000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35860/0 (socket says 172.21.15.76:35860) 2024-04-25T19:19:38.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.429+0000 7fe0a5d9b640 1 -- 172.21.15.76:0/2980809605 learned_addr learned my addr 172.21.15.76:0/2980809605 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:38.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.429+0000 7fe0a659c640 1 -- 172.21.15.76:0/2980809605 >> v1:172.21.15.105:6789/0 conn(0x7fe0a8074690 legacy=0x7fe0a8152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:38.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.429+0000 7fe0a659c640 1 -- 172.21.15.76:0/2980809605 >> v1:172.21.15.76:6789/0 conn(0x7fe0a8074280 legacy=0x7fe0a806f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.429+0000 7fe0a559a640 1 -- 172.21.15.76:0/2980809605 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 966780285 0 0) 0x7fe0a80c8cc0 con 0x7fe0a8074280 2024-04-25T19:19:38.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.429+0000 7fe0a659c640 1 -- 172.21.15.76:0/2980809605 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a80c8720 con 0x7fe0a80c9330 2024-04-25T19:19:38.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.430+0000 7fe0a659c640 1 --2- 172.21.15.76:0/2980809605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0a80c9330 0x7fe0a80c4960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe0a80c9d40 tx=0x7fe0a002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa3dc3be51078abc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:38.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.430+0000 7fe0a559a640 1 -- 172.21.15.76:0/2980809605 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0007740 con 0x7fe0a80c9330 2024-04-25T19:19:38.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.430+0000 7fe0a559a640 1 -- 172.21.15.76:0/2980809605 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0a00078e0 con 0x7fe0a80c9330 2024-04-25T19:19:38.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.430+0000 7fe0a559a640 1 -- 172.21.15.76:0/2980809605 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0007cb0 con 0x7fe0a80c9330 2024-04-25T19:19:38.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.430+0000 7fe0acc51640 1 -- 172.21.15.76:0/2980809605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0a80c9330 msgr2=0x7fe0a80c4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.430+0000 7fe0acc51640 1 --2- 172.21.15.76:0/2980809605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0a80c9330 0x7fe0a80c4960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe0a80c9d40 tx=0x7fe0a002a730 comp rx=0 tx=0).stop 2024-04-25T19:19:38.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.430+0000 7fe0acc51640 1 -- 172.21.15.76:0/2980809605 shutdown_connections 2024-04-25T19:19:38.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.430+0000 7fe0acc51640 1 --2- 172.21.15.76:0/2980809605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0a80c9330 0x7fe0a80c4960 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:38.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.430+0000 7fe0acc51640 1 -- 172.21.15.76:0/2980809605 >> 172.21.15.76:0/2980809605 conn(0x7fe0a80ca820 msgr2=0x7fe0a80cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:38.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.431+0000 7fe0acc51640 1 -- 172.21.15.76:0/2980809605 shutdown_connections 2024-04-25T19:19:38.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.431+0000 7fe0acc51640 1 -- 172.21.15.76:0/2980809605 wait complete. 2024-04-25T19:19:38.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.432+0000 7fe0acc51640 1 Processor -- start 2024-04-25T19:19:38.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.432+0000 7fe0acc51640 1 -- start start 2024-04-25T19:19:38.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.434+0000 7fe0acc51640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0a8074280 0x7fe0a81f0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:38.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.434+0000 7fe0acc51640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0a8074690 0x7fe0a814a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:38.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.434+0000 7fe0acc51640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe0a80c9330 0x7fe0a8146e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:38.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.434+0000 7fe0acc51640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a8073df0 con 0x7fe0a8074690 2024-04-25T19:19:38.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.434+0000 7fe0acc51640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a806f710 con 0x7fe0a8074280 2024-04-25T19:19:38.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.434+0000 7fe0acc51640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fe0a80c5330 con 0x7fe0a80c9330 2024-04-25T19:19:38.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.434+0000 7fe0a5d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0a8074690 0x7fe0a814a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:38.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.434+0000 7fe0a5d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0a8074690 0x7fe0a814a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50810/0 (socket says 172.21.15.76:50810) 2024-04-25T19:19:38.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.434+0000 7fe0a5d9b640 1 -- 172.21.15.76:0/2916686965 learned_addr learned my addr 172.21.15.76:0/2916686965 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:38.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.434+0000 7fe0a659c640 1 --2- 172.21.15.76:0/2916686965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0a8074280 0x7fe0a81f0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:38.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.434+0000 7fe0a6d9d640 1 --2- 172.21.15.76:0/2916686965 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe0a80c9330 0x7fe0a8146e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:38.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.434+0000 7fe0a5d9b640 1 -- 172.21.15.76:0/2916686965 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe0a80c9330 msgr2=0x7fe0a8146e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.434+0000 7fe0a5d9b640 1 --2- 172.21.15.76:0/2916686965 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fe0a80c9330 0x7fe0a8146e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:38.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.434+0000 7fe0a5d9b640 1 -- 172.21.15.76:0/2916686965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0a8074280 msgr2=0x7fe0a81f0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.434+0000 7fe0a5d9b640 1 --2- 172.21.15.76:0/2916686965 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0a8074280 0x7fe0a81f0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:38.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.434+0000 7fe0a5d9b640 1 -- 172.21.15.76:0/2916686965 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a81e3b30 con 0x7fe0a8074690 2024-04-25T19:19:38.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.434+0000 7fe0a5d9b640 1 --2- 172.21.15.76:0/2916686965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0a8074690 0x7fe0a814a710 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe09c00c460 tx=0x7fe09c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:38.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.434+0000 7fe096ffd640 1 -- 172.21.15.76:0/2916686965 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09c018070 con 0x7fe0a8074690 2024-04-25T19:19:38.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.434+0000 7fe096ffd640 1 -- 172.21.15.76:0/2916686965 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe09c007500 con 0x7fe0a8074690 2024-04-25T19:19:38.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.434+0000 7fe096ffd640 1 -- 172.21.15.76:0/2916686965 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09c004a20 con 0x7fe0a8074690 2024-04-25T19:19:38.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.434+0000 7fe0acc51640 1 -- 172.21.15.76:0/2916686965 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0a81e3e10 con 0x7fe0a8074690 2024-04-25T19:19:38.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.434+0000 7fe0acc51640 1 -- 172.21.15.76:0/2916686965 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0a81f0150 con 0x7fe0a8074690 2024-04-25T19:19:38.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.435+0000 7fe0acc51640 1 -- 172.21.15.76:0/2916686965 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe0a8073930 con 0x7fe0a8074690 2024-04-25T19:19:38.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.436+0000 7fe096ffd640 1 -- 172.21.15.76:0/2916686965 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe09c01b070 con 0x7fe0a8074690 2024-04-25T19:19:38.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.436+0000 7fe096ffd640 1 --2- 172.21.15.76:0/2916686965 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe0880744b0 0x7fe0880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:38.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.436+0000 7fe096ffd640 1 -- 172.21.15.76:0/2916686965 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fe09c0944e0 con 0x7fe0a8074690 2024-04-25T19:19:38.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.437+0000 7fe096ffd640 1 --2- 172.21.15.76:0/2916686965 >> [v2:172.21.15.105:6817/3413103121,v1:172.21.15.105:6819/3413103121] conn(0x7fe08807c880 0x7fe08807ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:38.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.437+0000 7fe096ffd640 1 -- 172.21.15.76:0/2916686965 --> [v2:172.21.15.105:6817/3413103121,v1:172.21.15.105:6819/3413103121] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe08807f420 con 0x7fe08807c880 2024-04-25T19:19:38.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.437+0000 7fe096ffd640 1 -- 172.21.15.76:0/2916686965 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe09c09b050 con 0x7fe0a8074690 2024-04-25T19:19:38.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.437+0000 7fe0a659c640 1 --2- 172.21.15.76:0/2916686965 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe0880744b0 0x7fe0880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:38.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.437+0000 7fe0a6d9d640 1 --2- 172.21.15.76:0/2916686965 >> [v2:172.21.15.105:6817/3413103121,v1:172.21.15.105:6819/3413103121] conn(0x7fe08807c880 0x7fe08807ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:38.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.437+0000 7fe0a659c640 1 --2- 172.21.15.76:0/2916686965 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe0880744b0 0x7fe0880769c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0034000 tx=0x7fe0a002f000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:38.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.437+0000 7fe0a6d9d640 1 --2- 172.21.15.76:0/2916686965 >> [v2:172.21.15.105:6817/3413103121,v1:172.21.15.105:6819/3413103121] conn(0x7fe08807c880 0x7fe08807ed50 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:38.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.438+0000 7fe096ffd640 1 -- 172.21.15.76:0/2916686965 <== osd.5 v2:172.21.15.105:6817/3413103121 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe08807f420 con 0x7fe08807c880 2024-04-25T19:19:38.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.451+0000 7fe0acc51640 1 -- 172.21.15.76:0/2916686965 --> [v2:172.21.15.105:6817/3413103121,v1:172.21.15.105:6819/3413103121] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe0a81e38e0 con 0x7fe08807c880 2024-04-25T19:19:38.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.452+0000 7fe096ffd640 1 -- 172.21.15.76:0/2916686965 <== osd.5 v2:172.21.15.105:6817/3413103121 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe0a81e38e0 con 0x7fe08807c880 2024-04-25T19:19:38.453 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:19:38.453 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T19:19:38.453 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:19:38.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.452+0000 7fe0acc51640 1 -- 172.21.15.76:0/2916686965 >> [v2:172.21.15.105:6817/3413103121,v1:172.21.15.105:6819/3413103121] conn(0x7fe08807c880 msgr2=0x7fe08807ed50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.452+0000 7fe0acc51640 1 --2- 172.21.15.76:0/2916686965 >> [v2:172.21.15.105:6817/3413103121,v1:172.21.15.105:6819/3413103121] conn(0x7fe08807c880 0x7fe08807ed50 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:38.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.452+0000 7fe0acc51640 1 -- 172.21.15.76:0/2916686965 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe0880744b0 msgr2=0x7fe0880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.452+0000 7fe0acc51640 1 --2- 172.21.15.76:0/2916686965 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fe0880744b0 0x7fe0880769c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0034000 tx=0x7fe0a002f000 comp rx=0 tx=0).stop 2024-04-25T19:19:38.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.452+0000 7fe0acc51640 1 -- 172.21.15.76:0/2916686965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0a8074690 msgr2=0x7fe0a814a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.452+0000 7fe0acc51640 1 --2- 172.21.15.76:0/2916686965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0a8074690 0x7fe0a814a710 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe09c00c460 tx=0x7fe09c00c920 comp rx=0 tx=0).stop 2024-04-25T19:19:38.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.452+0000 7fe0a6d9d640 1 -- 172.21.15.76:0/2916686965 reap_dead start 2024-04-25T19:19:38.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.452+0000 7fe0acc51640 1 -- 172.21.15.76:0/2916686965 shutdown_connections 2024-04-25T19:19:38.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.452+0000 7fe0acc51640 1 -- 172.21.15.76:0/2916686965 >> 172.21.15.76:0/2916686965 conn(0x7fe0a80ca820 msgr2=0x7fe0a80cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:38.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.452+0000 7fe0acc51640 1 -- 172.21.15.76:0/2916686965 shutdown_connections 2024-04-25T19:19:38.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.452+0000 7fe0acc51640 1 -- 172.21.15.76:0/2916686965 wait complete. 2024-04-25T19:19:38.465 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T19:19:38.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.572+0000 7f5fae30d640 1 Processor -- start 2024-04-25T19:19:38.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.572+0000 7f5fae30d640 1 -- start start 2024-04-25T19:19:38.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.574+0000 7f5fae30d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f5fa80d0b10 0x7f5fa80cee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:38.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.574+0000 7f5fae30d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5fa80cf3d0 0x7f5fa80ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:38.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.574+0000 7f5fae30d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fa815dd00 con 0x7f5fa8157200 2024-04-25T19:19:38.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.574+0000 7f5fae30d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5fa80d0460 con 0x7f5fa80cf3d0 2024-04-25T19:19:38.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.574+0000 7f5fae30d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f5fa815cea0 con 0x7f5fa80d0b10 2024-04-25T19:19:38.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.574+0000 7f5fa7fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f5fa8157200 0x7f5fa80cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35870/0 (socket says 172.21.15.76:35870) 2024-04-25T19:19:38.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.574+0000 7f5fac8aa640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5fa80cf3d0 0x7f5fa80ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:38.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.574+0000 7f5fa7fff640 1 -- 172.21.15.76:0/4150135395 learned_addr learned my addr 172.21.15.76:0/4150135395 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:38.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.574+0000 7f5fa77fe640 1 --2- 172.21.15.76:0/4150135395 >> v2:172.21.15.105:3300/0 conn(0x7f5fa80d0b10 0x7f5fa80cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:38.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.574+0000 7f5fac8aa640 1 -- 172.21.15.76:0/4150135395 >> v2:172.21.15.105:3300/0 conn(0x7f5fa80d0b10 msgr2=0x7f5fa80cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:38.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.574+0000 7f5fac8aa640 1 --2- 172.21.15.76:0/4150135395 >> v2:172.21.15.105:3300/0 conn(0x7f5fa80d0b10 0x7f5fa80cee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:38.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.574+0000 7f5fac8aa640 1 -- 172.21.15.76:0/4150135395 >> v1:172.21.15.76:6789/0 conn(0x7f5fa8157200 legacy=0x7f5fa80cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.574+0000 7f5fac8aa640 1 -- 172.21.15.76:0/4150135395 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa815d180 con 0x7f5fa80cf3d0 2024-04-25T19:19:38.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.574+0000 7f5fa6ffd640 1 -- 172.21.15.76:0/4150135395 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1514016535 0 0) 0x7f5fa815dd00 con 0x7f5fa8157200 2024-04-25T19:19:38.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.574+0000 7f5fa77fe640 1 --2- 172.21.15.76:0/4150135395 >> v2:172.21.15.105:3300/0 conn(0x7f5fa80d0b10 0x7f5fa80cee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:19:38.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.575+0000 7f5fac8aa640 1 --2- 172.21.15.76:0/4150135395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fa80cf3d0 0x7f5fa80ccb70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5fa808fd80 tx=0x7f5fa002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1affa31c9e859d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:38.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.575+0000 7f5fa6ffd640 1 -- 172.21.15.76:0/4150135395 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0007740 con 0x7f5fa80cf3d0 2024-04-25T19:19:38.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.575+0000 7f5fa6ffd640 1 -- 172.21.15.76:0/4150135395 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa00078e0 con 0x7f5fa80cf3d0 2024-04-25T19:19:38.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.575+0000 7f5fa6ffd640 1 -- 172.21.15.76:0/4150135395 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0007cb0 con 0x7f5fa80cf3d0 2024-04-25T19:19:38.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.576+0000 7f5fae30d640 1 -- 172.21.15.76:0/4150135395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fa80cf3d0 msgr2=0x7f5fa80ccb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.576+0000 7f5fae30d640 1 --2- 172.21.15.76:0/4150135395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fa80cf3d0 0x7f5fa80ccb70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5fa808fd80 tx=0x7f5fa002a710 comp rx=0 tx=0).stop 2024-04-25T19:19:38.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.576+0000 7f5fae30d640 1 -- 172.21.15.76:0/4150135395 shutdown_connections 2024-04-25T19:19:38.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.576+0000 7f5fae30d640 1 --2- 172.21.15.76:0/4150135395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fa80cf3d0 0x7f5fa80ccb70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:38.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.576+0000 7f5fae30d640 1 --2- 172.21.15.76:0/4150135395 >> v2:172.21.15.105:3300/0 conn(0x7f5fa80d0b10 0x7f5fa80cee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:38.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.576+0000 7f5fae30d640 1 -- 172.21.15.76:0/4150135395 >> 172.21.15.76:0/4150135395 conn(0x7f5fa80d2370 msgr2=0x7f5fa80d2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:38.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.576+0000 7f5fae30d640 1 -- 172.21.15.76:0/4150135395 shutdown_connections 2024-04-25T19:19:38.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.576+0000 7f5fae30d640 1 -- 172.21.15.76:0/4150135395 wait complete. 2024-04-25T19:19:38.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.577+0000 7f5fae30d640 1 Processor -- start 2024-04-25T19:19:38.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.578+0000 7f5fae30d640 1 -- start start 2024-04-25T19:19:38.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.579+0000 7f5fae30d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fa80cf3d0 0x7f5fa80b6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:38.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.579+0000 7f5fae30d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fa80d0b10 0x7f5fa81e4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:38.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.579+0000 7f5fae30d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fa8157200 0x7f5fa81e5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:38.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.579+0000 7f5fae30d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa815da20 con 0x7f5fa8157200 2024-04-25T19:19:38.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.579+0000 7f5fae30d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa815d740 con 0x7f5fa80cf3d0 2024-04-25T19:19:38.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.579+0000 7f5fae30d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f5fa80d0f20 con 0x7f5fa80d0b10 2024-04-25T19:19:38.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.579+0000 7f5fac8aa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fa8157200 0x7f5fa81e5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:38.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.579+0000 7f5fac8aa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fa8157200 0x7f5fa81e5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50826/0 (socket says 172.21.15.76:50826) 2024-04-25T19:19:38.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.579+0000 7f5fac8aa640 1 -- 172.21.15.76:0/2726370518 learned_addr learned my addr 172.21.15.76:0/2726370518 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:38.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.579+0000 7f5fa77fe640 1 --2- 172.21.15.76:0/2726370518 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fa80d0b10 0x7f5fa81e4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:38.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.579+0000 7f5fa7fff640 1 --2- 172.21.15.76:0/2726370518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fa80cf3d0 0x7f5fa80b6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:38.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.579+0000 7f5fac8aa640 1 -- 172.21.15.76:0/2726370518 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fa80d0b10 msgr2=0x7f5fa81e4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.579+0000 7f5fac8aa640 1 --2- 172.21.15.76:0/2726370518 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fa80d0b10 0x7f5fa81e4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:38.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.579+0000 7f5fac8aa640 1 -- 172.21.15.76:0/2726370518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fa80cf3d0 msgr2=0x7f5fa80b6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.579+0000 7f5fac8aa640 1 --2- 172.21.15.76:0/2726370518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fa80cf3d0 0x7f5fa80b6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:38.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.579+0000 7f5fac8aa640 1 -- 172.21.15.76:0/2726370518 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa81e5a80 con 0x7f5fa8157200 2024-04-25T19:19:38.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.579+0000 7f5fac8aa640 1 --2- 172.21.15.76:0/2726370518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fa8157200 0x7f5fa81e5540 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5fa002ad00 tx=0x7f5fa0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:38.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.579+0000 7f5fa4ff9640 1 -- 172.21.15.76:0/2726370518 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa003f070 con 0x7f5fa8157200 2024-04-25T19:19:38.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.580+0000 7f5fa4ff9640 1 -- 172.21.15.76:0/2726370518 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa0031bc0 con 0x7f5fa8157200 2024-04-25T19:19:38.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.580+0000 7f5fae30d640 1 -- 172.21.15.76:0/2726370518 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fa81e3a20 con 0x7f5fa8157200 2024-04-25T19:19:38.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.580+0000 7f5fa4ff9640 1 -- 172.21.15.76:0/2726370518 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0044460 con 0x7f5fa8157200 2024-04-25T19:19:38.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.580+0000 7f5fae30d640 1 -- 172.21.15.76:0/2726370518 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fa80b7560 con 0x7f5fa8157200 2024-04-25T19:19:38.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.581+0000 7f5fae30d640 1 -- 172.21.15.76:0/2726370518 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fa8069a50 con 0x7f5fa8157200 2024-04-25T19:19:38.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.581+0000 7f5fa4ff9640 1 -- 172.21.15.76:0/2726370518 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5fa003b410 con 0x7f5fa8157200 2024-04-25T19:19:38.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.582+0000 7f5fa4ff9640 1 --2- 172.21.15.76:0/2726370518 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5f940744b0 0x7f5f940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:38.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.582+0000 7f5fa4ff9640 1 -- 172.21.15.76:0/2726370518 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f5fa00b6bc0 con 0x7f5fa8157200 2024-04-25T19:19:38.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.582+0000 7f5fa7fff640 1 --2- 172.21.15.76:0/2726370518 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5f940744b0 0x7f5f940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:38.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.582+0000 7f5fa7fff640 1 --2- 172.21.15.76:0/2726370518 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5f940744b0 0x7f5f940769c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5f98000f60 tx=0x7f5f98007c00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:38.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.586+0000 7f5fa4ff9640 1 -- 172.21.15.76:0/2726370518 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5fa8069a50 con 0x7f5fa8157200 2024-04-25T19:19:38.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.762+0000 7f5fae30d640 1 -- 172.21.15.76:0/2726370518 --> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f5fa8156df0 con 0x7f5f940744b0 2024-04-25T19:19:38.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.763+0000 7f5fa4ff9640 1 -- 172.21.15.76:0/2726370518 <== mgr.4109 v2:172.21.15.105:6832/2585775531 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5fa8156df0 con 0x7f5f940744b0 2024-04-25T19:19:38.765 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T19:19:38.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.765+0000 7f5fae30d640 1 -- 172.21.15.76:0/2726370518 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5f940744b0 msgr2=0x7f5f940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.765+0000 7f5fae30d640 1 --2- 172.21.15.76:0/2726370518 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5f940744b0 0x7f5f940769c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5f98000f60 tx=0x7f5f98007c00 comp rx=0 tx=0).stop 2024-04-25T19:19:38.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.765+0000 7f5fae30d640 1 -- 172.21.15.76:0/2726370518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fa8157200 msgr2=0x7f5fa81e5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.765+0000 7f5fae30d640 1 --2- 172.21.15.76:0/2726370518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fa8157200 0x7f5fa81e5540 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5fa002ad00 tx=0x7f5fa0004a40 comp rx=0 tx=0).stop 2024-04-25T19:19:38.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.765+0000 7f5fae30d640 1 -- 172.21.15.76:0/2726370518 shutdown_connections 2024-04-25T19:19:38.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.765+0000 7f5fae30d640 1 --2- 172.21.15.76:0/2726370518 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f5f940744b0 0x7f5f940769c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:38.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.765+0000 7f5fae30d640 1 --2- 172.21.15.76:0/2726370518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5fa8157200 0x7f5fa81e5540 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:38.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.765+0000 7f5fae30d640 1 --2- 172.21.15.76:0/2726370518 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f5fa80d0b10 0x7f5fa81e4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:38.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.765+0000 7f5fae30d640 1 --2- 172.21.15.76:0/2726370518 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fa80cf3d0 0x7f5fa80b6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:38.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.765+0000 7f5fae30d640 1 -- 172.21.15.76:0/2726370518 >> 172.21.15.76:0/2726370518 conn(0x7f5fa80d2370 msgr2=0x7f5fa808f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:38.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.765+0000 7f5fae30d640 1 -- 172.21.15.76:0/2726370518 shutdown_connections 2024-04-25T19:19:38.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.765+0000 7f5fae30d640 1 -- 172.21.15.76:0/2726370518 wait complete. 2024-04-25T19:19:38.778 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T19:19:38.778 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T19:19:38.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.889+0000 7f9da6ee2640 1 Processor -- start 2024-04-25T19:19:38.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.889+0000 7f9da6ee2640 1 -- start start 2024-04-25T19:19:38.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.890+0000 7f9da6ee2640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9da015cae0 0x7f9da0156c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:38.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.890+0000 7f9da6ee2640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9da0074650 0x7f9da00c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:38.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.890+0000 7f9da6ee2640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9da00c7670 con 0x7f9da00c95c0 2024-04-25T19:19:38.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.890+0000 7f9da6ee2640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f9da015d040 con 0x7f9da015cae0 2024-04-25T19:19:38.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.890+0000 7f9da6ee2640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f9da015d320 con 0x7f9da0074650 2024-04-25T19:19:38.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.890+0000 7f9da547f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9da0074650 0x7f9da00c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:38.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.890+0000 7f9d9ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9da015cae0 0x7f9da0156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:38.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.890+0000 7f9da547f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9da0074650 0x7f9da00c7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:50172/0 (socket says 172.21.15.76:50172) 2024-04-25T19:19:38.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.890+0000 7f9d9ffff640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f9da015cae0 0x7f9da0156c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:37076/0 (socket says 172.21.15.76:37076) 2024-04-25T19:19:38.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.890+0000 7f9da547f640 1 -- 172.21.15.76:0/1011181040 learned_addr learned my addr 172.21.15.76:0/1011181040 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:38.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.891+0000 7f9d9ffff640 1 -- 172.21.15.76:0/1011181040 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9da0074650 msgr2=0x7f9da00c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:38.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.891+0000 7f9d9ffff640 1 --2- 172.21.15.76:0/1011181040 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9da0074650 0x7f9da00c7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:38.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.891+0000 7f9d9ffff640 1 -- 172.21.15.76:0/1011181040 >> v1:172.21.15.105:6789/0 conn(0x7f9da00c95c0 legacy=0x7f9da00cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.891+0000 7f9d9ffff640 1 -- 172.21.15.76:0/1011181040 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da015d600 con 0x7f9da015cae0 2024-04-25T19:19:38.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.891+0000 7f9da547f640 1 --2- 172.21.15.76:0/1011181040 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9da0074650 0x7f9da00c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:19:38.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.891+0000 7f9d9ffff640 1 --2- 172.21.15.76:0/1011181040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da015cae0 0x7f9da0156c10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9d900097a0 tx=0x7f9d9002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c801416af4da0793 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:38.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.892+0000 7f9d9f7fe640 1 -- 172.21.15.76:0/1011181040 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90037070 con 0x7f9da015cae0 2024-04-25T19:19:38.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.892+0000 7f9d9f7fe640 1 -- 172.21.15.76:0/1011181040 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d90032440 con 0x7f9da015cae0 2024-04-25T19:19:38.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.892+0000 7f9d9f7fe640 1 -- 172.21.15.76:0/1011181040 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90032810 con 0x7f9da015cae0 2024-04-25T19:19:38.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.892+0000 7f9da6ee2640 1 -- 172.21.15.76:0/1011181040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da015cae0 msgr2=0x7f9da0156c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.892+0000 7f9da6ee2640 1 --2- 172.21.15.76:0/1011181040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da015cae0 0x7f9da0156c10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9d900097a0 tx=0x7f9d9002a180 comp rx=0 tx=0).stop 2024-04-25T19:19:38.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.892+0000 7f9da6ee2640 1 -- 172.21.15.76:0/1011181040 shutdown_connections 2024-04-25T19:19:38.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.892+0000 7f9da6ee2640 1 --2- 172.21.15.76:0/1011181040 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9da0074650 0x7f9da00c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:38.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.892+0000 7f9da6ee2640 1 --2- 172.21.15.76:0/1011181040 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da015cae0 0x7f9da0156c10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:38.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.892+0000 7f9da6ee2640 1 -- 172.21.15.76:0/1011181040 >> 172.21.15.76:0/1011181040 conn(0x7f9da0071a30 msgr2=0x7f9da0071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:38.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.892+0000 7f9da6ee2640 1 -- 172.21.15.76:0/1011181040 shutdown_connections 2024-04-25T19:19:38.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.892+0000 7f9da6ee2640 1 -- 172.21.15.76:0/1011181040 wait complete. 2024-04-25T19:19:38.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.894+0000 7f9da6ee2640 1 Processor -- start 2024-04-25T19:19:38.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.894+0000 7f9da6ee2640 1 -- start start 2024-04-25T19:19:38.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.895+0000 7f9da6ee2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9da0074650 0x7f9da00ce580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:38.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.895+0000 7f9da6ee2640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9da00c95c0 0x7f9da01f3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:38.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.895+0000 7f9da6ee2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da015cae0 0x7f9da00cd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:38.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.895+0000 7f9da6ee2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9da00c70a0 con 0x7f9da0074650 2024-04-25T19:19:38.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.895+0000 7f9da6ee2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9da00c78e0 con 0x7f9da015cae0 2024-04-25T19:19:38.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.895+0000 7f9da6ee2640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f9da00c7220 con 0x7f9da00c95c0 2024-04-25T19:19:38.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.895+0000 7f9da4c7e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9da0074650 0x7f9da00ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:38.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.895+0000 7f9da4c7e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9da0074650 0x7f9da00ce580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50830/0 (socket says 172.21.15.76:50830) 2024-04-25T19:19:38.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.896+0000 7f9da4c7e640 1 -- 172.21.15.76:0/1020040817 learned_addr learned my addr 172.21.15.76:0/1020040817 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:38.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.896+0000 7f9d9ffff640 1 --2- 172.21.15.76:0/1020040817 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9da00c95c0 0x7f9da01f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:38.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.896+0000 7f9da547f640 1 --2- 172.21.15.76:0/1020040817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da015cae0 0x7f9da00cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:38.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.896+0000 7f9da4c7e640 1 -- 172.21.15.76:0/1020040817 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9da00c95c0 msgr2=0x7f9da01f3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.896+0000 7f9da4c7e640 1 --2- 172.21.15.76:0/1020040817 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9da00c95c0 0x7f9da01f3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:38.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.896+0000 7f9da4c7e640 1 -- 172.21.15.76:0/1020040817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da015cae0 msgr2=0x7f9da00cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.896+0000 7f9da4c7e640 1 --2- 172.21.15.76:0/1020040817 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9da015cae0 0x7f9da00cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:38.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.896+0000 7f9da4c7e640 1 -- 172.21.15.76:0/1020040817 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da00ccf20 con 0x7f9da0074650 2024-04-25T19:19:38.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.896+0000 7f9da4c7e640 1 --2- 172.21.15.76:0/1020040817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9da0074650 0x7f9da00ce580 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800c470 tx=0x7f9d8800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:38.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.896+0000 7f9d9d7fa640 1 -- 172.21.15.76:0/1020040817 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d880052d0 con 0x7f9da0074650 2024-04-25T19:19:38.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.896+0000 7f9d9d7fa640 1 -- 172.21.15.76:0/1020040817 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d88005490 con 0x7f9da0074650 2024-04-25T19:19:38.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.896+0000 7f9d9d7fa640 1 -- 172.21.15.76:0/1020040817 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d88007260 con 0x7f9da0074650 2024-04-25T19:19:38.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.896+0000 7f9da6ee2640 1 -- 172.21.15.76:0/1020040817 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da014ee10 con 0x7f9da0074650 2024-04-25T19:19:38.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.896+0000 7f9da6ee2640 1 -- 172.21.15.76:0/1020040817 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da014f3d0 con 0x7f9da0074650 2024-04-25T19:19:38.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.897+0000 7f9da6ee2640 1 -- 172.21.15.76:0/1020040817 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9da006cfe0 con 0x7f9da0074650 2024-04-25T19:19:38.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.898+0000 7f9d9d7fa640 1 -- 172.21.15.76:0/1020040817 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9d88007400 con 0x7f9da0074650 2024-04-25T19:19:38.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.898+0000 7f9d9d7fa640 1 --2- 172.21.15.76:0/1020040817 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9d800744b0 0x7f9d800769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:38.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.898+0000 7f9d9d7fa640 1 -- 172.21.15.76:0/1020040817 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f9d88094dd0 con 0x7f9da0074650 2024-04-25T19:19:38.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.899+0000 7f9d9d7fa640 1 --2- 172.21.15.76:0/1020040817 >> [v2:172.21.15.105:6808/2721832954,v1:172.21.15.105:6809/2721832954] conn(0x7f9d8007c880 0x7f9d8007ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:38.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.899+0000 7f9d9d7fa640 1 -- 172.21.15.76:0/1020040817 --> [v2:172.21.15.105:6808/2721832954,v1:172.21.15.105:6809/2721832954] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9d8007f420 con 0x7f9d8007c880 2024-04-25T19:19:38.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.899+0000 7f9d9d7fa640 1 -- 172.21.15.76:0/1020040817 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9d88095260 con 0x7f9da0074650 2024-04-25T19:19:38.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.899+0000 7f9d9ffff640 1 --2- 172.21.15.76:0/1020040817 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9d800744b0 0x7f9d800769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:38.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.899+0000 7f9da547f640 1 --2- 172.21.15.76:0/1020040817 >> [v2:172.21.15.105:6808/2721832954,v1:172.21.15.105:6809/2721832954] conn(0x7f9d8007c880 0x7f9d8007ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:38.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.899+0000 7f9d9ffff640 1 --2- 172.21.15.76:0/1020040817 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9d800744b0 0x7f9d800769c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9d90034000 tx=0x7f9d9002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:38.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.899+0000 7f9da547f640 1 --2- 172.21.15.76:0/1020040817 >> [v2:172.21.15.105:6808/2721832954,v1:172.21.15.105:6809/2721832954] conn(0x7f9d8007c880 0x7f9d8007ed50 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:38.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.900+0000 7f9d9d7fa640 1 -- 172.21.15.76:0/1020040817 <== osd.6 v2:172.21.15.105:6808/2721832954 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9d8007f420 con 0x7f9d8007c880 2024-04-25T19:19:38.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.913+0000 7f9da6ee2640 1 -- 172.21.15.76:0/1020040817 --> [v2:172.21.15.105:6808/2721832954,v1:172.21.15.105:6809/2721832954] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9da0150100 con 0x7f9d8007c880 2024-04-25T19:19:38.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.913+0000 7f9d9d7fa640 1 -- 172.21.15.76:0/1020040817 <== osd.6 v2:172.21.15.105:6808/2721832954 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9da0150100 con 0x7f9d8007c880 2024-04-25T19:19:38.915 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:19:38.915 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T19:19:38.915 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:19:38.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.914+0000 7f9da6ee2640 1 -- 172.21.15.76:0/1020040817 >> [v2:172.21.15.105:6808/2721832954,v1:172.21.15.105:6809/2721832954] conn(0x7f9d8007c880 msgr2=0x7f9d8007ed50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.914+0000 7f9da6ee2640 1 --2- 172.21.15.76:0/1020040817 >> [v2:172.21.15.105:6808/2721832954,v1:172.21.15.105:6809/2721832954] conn(0x7f9d8007c880 0x7f9d8007ed50 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:38.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.914+0000 7f9da6ee2640 1 -- 172.21.15.76:0/1020040817 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9d800744b0 msgr2=0x7f9d800769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.914+0000 7f9da6ee2640 1 --2- 172.21.15.76:0/1020040817 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9d800744b0 0x7f9d800769c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9d90034000 tx=0x7f9d9002e000 comp rx=0 tx=0).stop 2024-04-25T19:19:38.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.914+0000 7f9da6ee2640 1 -- 172.21.15.76:0/1020040817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9da0074650 msgr2=0x7f9da00ce580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:38.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.914+0000 7f9da6ee2640 1 --2- 172.21.15.76:0/1020040817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9da0074650 0x7f9da00ce580 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800c470 tx=0x7f9d8800c930 comp rx=0 tx=0).stop 2024-04-25T19:19:38.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.914+0000 7f9da547f640 1 -- 172.21.15.76:0/1020040817 reap_dead start 2024-04-25T19:19:38.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.914+0000 7f9da6ee2640 1 -- 172.21.15.76:0/1020040817 shutdown_connections 2024-04-25T19:19:38.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.914+0000 7f9da6ee2640 1 -- 172.21.15.76:0/1020040817 >> 172.21.15.76:0/1020040817 conn(0x7f9da0071a30 msgr2=0x7f9da015b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:38.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.914+0000 7f9da6ee2640 1 -- 172.21.15.76:0/1020040817 shutdown_connections 2024-04-25T19:19:38.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:38.914+0000 7f9da6ee2640 1 -- 172.21.15.76:0/1020040817 wait complete. 2024-04-25T19:19:38.927 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T19:19:39.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.034+0000 7ff5e6e30640 1 Processor -- start 2024-04-25T19:19:39.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.034+0000 7ff5e6e30640 1 -- start start 2024-04-25T19:19:39.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.036+0000 7ff5e6e30640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5e00c7750 0x7ff5e0153590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.036+0000 7ff5e6e30640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff5e015cd80 0x7ff5e015d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.036+0000 7ff5e6e30640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5e00c8020 con 0x7ff5e00c7b60 2024-04-25T19:19:39.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.036+0000 7ff5e6e30640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff5e01598a0 con 0x7ff5e015cd80 2024-04-25T19:19:39.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.036+0000 7ff5e6e30640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e0159b40 con 0x7ff5e00c7750 2024-04-25T19:19:39.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.036+0000 7ff5e53cd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff5e015cd80 0x7ff5e015d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.036+0000 7ff5dffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff5e00c7b60 0x7ff5e00cdd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35884/0 (socket says 172.21.15.76:35884) 2024-04-25T19:19:39.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.036+0000 7ff5e53cd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff5e015cd80 0x7ff5e015d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50838/0 (socket says 172.21.15.76:50838) 2024-04-25T19:19:39.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.036+0000 7ff5dffff640 1 -- 172.21.15.76:0/494389577 learned_addr learned my addr 172.21.15.76:0/494389577 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:39.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.036+0000 7ff5e4bcc640 1 --2- 172.21.15.76:0/494389577 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5e00c7750 0x7ff5e0153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.036+0000 7ff5e53cd640 1 -- 172.21.15.76:0/494389577 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5e00c7750 msgr2=0x7ff5e0153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:39.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.036+0000 7ff5e53cd640 1 --2- 172.21.15.76:0/494389577 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5e00c7750 0x7ff5e0153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.036+0000 7ff5e53cd640 1 -- 172.21.15.76:0/494389577 >> v1:172.21.15.76:6789/0 conn(0x7ff5e00c7b60 legacy=0x7ff5e00cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:39.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.036+0000 7ff5df7fe640 1 -- 172.21.15.76:0/494389577 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 962051915 0 0) 0x7ff5e00c8020 con 0x7ff5e00c7b60 2024-04-25T19:19:39.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.036+0000 7ff5e53cd640 1 -- 172.21.15.76:0/494389577 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e0159e20 con 0x7ff5e015cd80 2024-04-25T19:19:39.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.037+0000 7ff5e53cd640 1 --2- 172.21.15.76:0/494389577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5e015cd80 0x7ff5e015d190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8009d90 tx=0x7ff5d802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e97df888f5dd8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:39.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.037+0000 7ff5df7fe640 1 -- 172.21.15.76:0/494389577 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8007740 con 0x7ff5e015cd80 2024-04-25T19:19:39.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.037+0000 7ff5df7fe640 1 -- 172.21.15.76:0/494389577 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d80078e0 con 0x7ff5e015cd80 2024-04-25T19:19:39.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.037+0000 7ff5df7fe640 1 -- 172.21.15.76:0/494389577 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8007cb0 con 0x7ff5e015cd80 2024-04-25T19:19:39.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.038+0000 7ff5e6e30640 1 -- 172.21.15.76:0/494389577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5e015cd80 msgr2=0x7ff5e015d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:39.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.038+0000 7ff5e6e30640 1 --2- 172.21.15.76:0/494389577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5e015cd80 0x7ff5e015d190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8009d90 tx=0x7ff5d802a680 comp rx=0 tx=0).stop 2024-04-25T19:19:39.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.038+0000 7ff5e6e30640 1 -- 172.21.15.76:0/494389577 shutdown_connections 2024-04-25T19:19:39.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.038+0000 7ff5e6e30640 1 --2- 172.21.15.76:0/494389577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5e015cd80 0x7ff5e015d190 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.038+0000 7ff5e6e30640 1 --2- 172.21.15.76:0/494389577 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5e00c7750 0x7ff5e0153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.038+0000 7ff5e6e30640 1 -- 172.21.15.76:0/494389577 >> 172.21.15.76:0/494389577 conn(0x7ff5e0070800 msgr2=0x7ff5e0070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:39.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.038+0000 7ff5e6e30640 1 -- 172.21.15.76:0/494389577 shutdown_connections 2024-04-25T19:19:39.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.038+0000 7ff5e6e30640 1 -- 172.21.15.76:0/494389577 wait complete. 2024-04-25T19:19:39.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.039+0000 7ff5e6e30640 1 Processor -- start 2024-04-25T19:19:39.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.039+0000 7ff5e6e30640 1 -- start start 2024-04-25T19:19:39.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.041+0000 7ff5e6e30640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5e00c7750 0x7ff5e0150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.041+0000 7ff5e6e30640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5e015cd80 0x7ff5e01e3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.041+0000 7ff5e6e30640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5e0152a70 0x7ff5e0156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.041+0000 7ff5e6e30640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e015d6d0 con 0x7ff5e015cd80 2024-04-25T19:19:39.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.041+0000 7ff5e6e30640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e015a3e0 con 0x7ff5e0152a70 2024-04-25T19:19:39.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.041+0000 7ff5e6e30640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e0153ad0 con 0x7ff5e00c7750 2024-04-25T19:19:39.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.041+0000 7ff5dffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5e015cd80 0x7ff5e01e3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.041+0000 7ff5dffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5e015cd80 0x7ff5e01e3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50852/0 (socket says 172.21.15.76:50852) 2024-04-25T19:19:39.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.041+0000 7ff5dffff640 1 -- 172.21.15.76:0/183221720 learned_addr learned my addr 172.21.15.76:0/183221720 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:39.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.041+0000 7ff5e53cd640 1 --2- 172.21.15.76:0/183221720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5e0152a70 0x7ff5e0156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.041+0000 7ff5e4bcc640 1 --2- 172.21.15.76:0/183221720 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5e00c7750 0x7ff5e0150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.041+0000 7ff5dffff640 1 -- 172.21.15.76:0/183221720 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5e00c7750 msgr2=0x7ff5e0150ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:39.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.041+0000 7ff5dffff640 1 --2- 172.21.15.76:0/183221720 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5e00c7750 0x7ff5e0150ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.041+0000 7ff5dffff640 1 -- 172.21.15.76:0/183221720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5e0152a70 msgr2=0x7ff5e0156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:39.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.041+0000 7ff5dffff640 1 --2- 172.21.15.76:0/183221720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5e0152a70 0x7ff5e0156a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.041+0000 7ff5dffff640 1 -- 172.21.15.76:0/183221720 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e01f20f0 con 0x7ff5e015cd80 2024-04-25T19:19:39.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.041+0000 7ff5e53cd640 1 --2- 172.21.15.76:0/183221720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5e0152a70 0x7ff5e0156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:19:39.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.041+0000 7ff5e4bcc640 1 --2- 172.21.15.76:0/183221720 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5e00c7750 0x7ff5e0150ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:19:39.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.041+0000 7ff5dffff640 1 --2- 172.21.15.76:0/183221720 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5e015cd80 0x7ff5e01e3340 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff5e006fd10 tx=0x7ff5d400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:39.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.041+0000 7ff5dd7fa640 1 -- 172.21.15.76:0/183221720 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d40074e0 con 0x7ff5e015cd80 2024-04-25T19:19:39.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.041+0000 7ff5dd7fa640 1 -- 172.21.15.76:0/183221720 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d4007680 con 0x7ff5e015cd80 2024-04-25T19:19:39.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.041+0000 7ff5e6e30640 1 -- 172.21.15.76:0/183221720 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e014e880 con 0x7ff5e015cd80 2024-04-25T19:19:39.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.042+0000 7ff5dd7fa640 1 -- 172.21.15.76:0/183221720 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4004a00 con 0x7ff5e015cd80 2024-04-25T19:19:39.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.042+0000 7ff5e6e30640 1 -- 172.21.15.76:0/183221720 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e014e5a0 con 0x7ff5e015cd80 2024-04-25T19:19:39.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.042+0000 7ff5e6e30640 1 -- 172.21.15.76:0/183221720 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e006a390 con 0x7ff5e015cd80 2024-04-25T19:19:39.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.043+0000 7ff5dd7fa640 1 -- 172.21.15.76:0/183221720 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff5d4011070 con 0x7ff5e015cd80 2024-04-25T19:19:39.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.043+0000 7ff5dd7fa640 1 --2- 172.21.15.76:0/183221720 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff5cc0744e0 0x7ff5cc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.044+0000 7ff5dd7fa640 1 -- 172.21.15.76:0/183221720 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7ff5d4094d40 con 0x7ff5e015cd80 2024-04-25T19:19:39.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.044+0000 7ff5e4bcc640 1 --2- 172.21.15.76:0/183221720 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff5cc0744e0 0x7ff5cc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.044+0000 7ff5e4bcc640 1 --2- 172.21.15.76:0/183221720 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff5cc0744e0 0x7ff5cc0769f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0006fd0 tx=0x7ff5d0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:39.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.048+0000 7ff5dd7fa640 1 -- 172.21.15.76:0/183221720 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff5d405f570 con 0x7ff5e015cd80 2024-04-25T19:19:39.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.225+0000 7ff5e6e30640 1 -- 172.21.15.76:0/183221720 --> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7ff5e01f2300 con 0x7ff5cc0744e0 2024-04-25T19:19:39.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.225+0000 7ff5dd7fa640 1 -- 172.21.15.76:0/183221720 <== mgr.4109 v2:172.21.15.105:6832/2585775531 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff5e01f2300 con 0x7ff5cc0744e0 2024-04-25T19:19:39.227 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T19:19:39.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.227+0000 7ff5e6e30640 1 -- 172.21.15.76:0/183221720 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff5cc0744e0 msgr2=0x7ff5cc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:39.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.227+0000 7ff5e6e30640 1 --2- 172.21.15.76:0/183221720 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff5cc0744e0 0x7ff5cc0769f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0006fd0 tx=0x7ff5d0008040 comp rx=0 tx=0).stop 2024-04-25T19:19:39.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.227+0000 7ff5e6e30640 1 -- 172.21.15.76:0/183221720 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5e015cd80 msgr2=0x7ff5e01e3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:39.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.227+0000 7ff5e6e30640 1 --2- 172.21.15.76:0/183221720 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5e015cd80 0x7ff5e01e3340 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff5e006fd10 tx=0x7ff5d400c820 comp rx=0 tx=0).stop 2024-04-25T19:19:39.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.227+0000 7ff5e6e30640 1 -- 172.21.15.76:0/183221720 shutdown_connections 2024-04-25T19:19:39.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.227+0000 7ff5e6e30640 1 --2- 172.21.15.76:0/183221720 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7ff5cc0744e0 0x7ff5cc0769f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.227+0000 7ff5e6e30640 1 --2- 172.21.15.76:0/183221720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff5e0152a70 0x7ff5e0156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.227+0000 7ff5e6e30640 1 --2- 172.21.15.76:0/183221720 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5e015cd80 0x7ff5e01e3340 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.227+0000 7ff5e6e30640 1 --2- 172.21.15.76:0/183221720 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7ff5e00c7750 0x7ff5e0150ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.227+0000 7ff5e6e30640 1 -- 172.21.15.76:0/183221720 >> 172.21.15.76:0/183221720 conn(0x7ff5e0070800 msgr2=0x7ff5e0070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:39.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.227+0000 7ff5e6e30640 1 -- 172.21.15.76:0/183221720 shutdown_connections 2024-04-25T19:19:39.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.227+0000 7ff5e6e30640 1 -- 172.21.15.76:0/183221720 wait complete. 2024-04-25T19:19:39.240 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T19:19:39.240 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T19:19:39.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.350+0000 7fb12770d640 1 Processor -- start 2024-04-25T19:19:39.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.350+0000 7fb12770d640 1 -- start start 2024-04-25T19:19:39.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.351+0000 7fb12770d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1200c9730 0x7fb12015cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.351+0000 7fb12770d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fb120157020 0x7fb12006c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.351+0000 7fb12770d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb120160140 con 0x7fb120156c10 2024-04-25T19:19:39.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.351+0000 7fb12770d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fb1201603e0 con 0x7fb120157020 2024-04-25T19:19:39.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.351+0000 7fb12770d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb120160680 con 0x7fb1200c9730 2024-04-25T19:19:39.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.351+0000 7fb124ca8640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb120156c10 0x7fb120071a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:35888/0 (socket says 172.21.15.76:35888) 2024-04-25T19:19:39.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.351+0000 7fb124ca8640 1 -- 172.21.15.76:0/2040032765 learned_addr learned my addr 172.21.15.76:0/2040032765 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:39.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.352+0000 7fb125caa640 1 --2- 172.21.15.76:0/2040032765 >> v2:172.21.15.105:3300/0 conn(0x7fb120157020 0x7fb12006c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.352+0000 7fb1254a9640 1 --2- 172.21.15.76:0/2040032765 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1200c9730 0x7fb12015cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.352+0000 7fb117fff640 1 -- 172.21.15.76:0/2040032765 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3267929409 0 0) 0x7fb120160140 con 0x7fb120156c10 2024-04-25T19:19:39.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.352+0000 7fb117fff640 1 -- 172.21.15.76:0/2040032765 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb10c003520 con 0x7fb120156c10 2024-04-25T19:19:39.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.352+0000 7fb117fff640 1 -- 172.21.15.76:0/2040032765 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1455928820 0 0) 0x7fb10c003520 con 0x7fb120156c10 2024-04-25T19:19:39.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.352+0000 7fb117fff640 1 -- 172.21.15.76:0/2040032765 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1200c9730 msgr2=0x7fb12015cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:39.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.352+0000 7fb117fff640 1 --2- 172.21.15.76:0/2040032765 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1200c9730 0x7fb12015cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.352+0000 7fb117fff640 1 -- 172.21.15.76:0/2040032765 >> v2:172.21.15.105:3300/0 conn(0x7fb120157020 msgr2=0x7fb12006c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:39.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.352+0000 7fb117fff640 1 --2- 172.21.15.76:0/2040032765 >> v2:172.21.15.105:3300/0 conn(0x7fb120157020 0x7fb12006c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.352+0000 7fb117fff640 1 -- 172.21.15.76:0/2040032765 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb120071560 con 0x7fb120156c10 2024-04-25T19:19:39.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.352+0000 7fb117fff640 1 -- 172.21.15.76:0/2040032765 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7fb110001670 con 0x7fb120156c10 2024-04-25T19:19:39.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.352+0000 7fb117fff640 1 -- 172.21.15.76:0/2040032765 >> v1:172.21.15.76:6789/0 conn(0x7fb120156c10 legacy=0x7fb120071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:39.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.352+0000 7fb117fff640 1 --2- 172.21.15.76:0/2040032765 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1200c9730 0x7fb10c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.352+0000 7fb117fff640 1 --2- 172.21.15.76:0/2040032765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb10c003f30 0x7fb10c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.352+0000 7fb117fff640 1 --2- 172.21.15.76:0/2040032765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb10c004d40 0x7fb10c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.352+0000 7fb117fff640 1 -- 172.21.15.76:0/2040032765 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb1201603e0 con 0x7fb10c003f30 2024-04-25T19:19:39.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.352+0000 7fb117fff640 1 -- 172.21.15.76:0/2040032765 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb120160680 con 0x7fb10c004d40 2024-04-25T19:19:39.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.352+0000 7fb117fff640 1 -- 172.21.15.76:0/2040032765 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb10c00ba10 con 0x7fb1200c9730 2024-04-25T19:19:39.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.352+0000 7fb124ca8640 1 --2- 172.21.15.76:0/2040032765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb10c003f30 0x7fb10c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.352+0000 7fb1254a9640 1 --2- 172.21.15.76:0/2040032765 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1200c9730 0x7fb10c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.352+0000 7fb125caa640 1 --2- 172.21.15.76:0/2040032765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb10c004d40 0x7fb10c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.352+0000 7fb124ca8640 1 -- 172.21.15.76:0/2040032765 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1200c9730 msgr2=0x7fb10c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:39.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.352+0000 7fb124ca8640 1 --2- 172.21.15.76:0/2040032765 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb1200c9730 0x7fb10c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.352+0000 7fb124ca8640 1 -- 172.21.15.76:0/2040032765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb10c004d40 msgr2=0x7fb10c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:39.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.352+0000 7fb124ca8640 1 --2- 172.21.15.76:0/2040032765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb10c004d40 0x7fb10c005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.352+0000 7fb124ca8640 1 -- 172.21.15.76:0/2040032765 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10c003520 con 0x7fb10c003f30 2024-04-25T19:19:39.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.353+0000 7fb124ca8640 1 --2- 172.21.15.76:0/2040032765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb10c003f30 0x7fb10c004800 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb11000c210 tx=0x7fb10c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a614f3213a7ae216 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:39.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.353+0000 7fb117fff640 1 -- 172.21.15.76:0/2040032765 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb110007d30 con 0x7fb10c003f30 2024-04-25T19:19:39.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.353+0000 7fb117fff640 1 -- 172.21.15.76:0/2040032765 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb110005ad0 con 0x7fb10c003f30 2024-04-25T19:19:39.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.353+0000 7fb117fff640 1 -- 172.21.15.76:0/2040032765 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb110007070 con 0x7fb10c003f30 2024-04-25T19:19:39.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.354+0000 7fb12770d640 1 -- 172.21.15.76:0/2040032765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb10c003f30 msgr2=0x7fb10c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:39.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.354+0000 7fb12770d640 1 --2- 172.21.15.76:0/2040032765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb10c003f30 0x7fb10c004800 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb11000c210 tx=0x7fb10c005830 comp rx=0 tx=0).stop 2024-04-25T19:19:39.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.354+0000 7fb125caa640 1 -- 172.21.15.76:0/2040032765 reap_dead start 2024-04-25T19:19:39.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.354+0000 7fb12770d640 1 -- 172.21.15.76:0/2040032765 shutdown_connections 2024-04-25T19:19:39.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.354+0000 7fb12770d640 1 -- 172.21.15.76:0/2040032765 >> 172.21.15.76:0/2040032765 conn(0x7fb12006d6d0 msgr2=0x7fb12006dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:39.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.354+0000 7fb12770d640 1 -- 172.21.15.76:0/2040032765 shutdown_connections 2024-04-25T19:19:39.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.354+0000 7fb12770d640 1 -- 172.21.15.76:0/2040032765 wait complete. 2024-04-25T19:19:39.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.356+0000 7fb12770d640 1 Processor -- start 2024-04-25T19:19:39.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.356+0000 7fb12770d640 1 -- start start 2024-04-25T19:19:39.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.357+0000 7fb12770d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb10c003f30 0x7fb1200c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.357+0000 7fb12770d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1200cbc30 0x7fb12015f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.357+0000 7fb12770d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb12015bdf0 0x7fb120148600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.357+0000 7fb12770d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb120160f00 con 0x7fb1200cbc30 2024-04-25T19:19:39.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.357+0000 7fb12770d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb120160c20 con 0x7fb12015bdf0 2024-04-25T19:19:39.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.357+0000 7fb12770d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fb120072170 con 0x7fb10c003f30 2024-04-25T19:19:39.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.357+0000 7fb124ca8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1200cbc30 0x7fb12015f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.357+0000 7fb124ca8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1200cbc30 0x7fb12015f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50870/0 (socket says 172.21.15.76:50870) 2024-04-25T19:19:39.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.357+0000 7fb124ca8640 1 -- 172.21.15.76:0/327743750 learned_addr learned my addr 172.21.15.76:0/327743750 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:39.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.357+0000 7fb1254a9640 1 --2- 172.21.15.76:0/327743750 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb10c003f30 0x7fb1200c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.357+0000 7fb125caa640 1 --2- 172.21.15.76:0/327743750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb12015bdf0 0x7fb120148600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.357+0000 7fb124ca8640 1 -- 172.21.15.76:0/327743750 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb10c003f30 msgr2=0x7fb1200c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:39.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.357+0000 7fb124ca8640 1 --2- 172.21.15.76:0/327743750 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fb10c003f30 0x7fb1200c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.357+0000 7fb124ca8640 1 -- 172.21.15.76:0/327743750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb12015bdf0 msgr2=0x7fb120148600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:39.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.357+0000 7fb124ca8640 1 --2- 172.21.15.76:0/327743750 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb12015bdf0 0x7fb120148600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.357+0000 7fb124ca8640 1 -- 172.21.15.76:0/327743750 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1200ca860 con 0x7fb1200cbc30 2024-04-25T19:19:39.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.358+0000 7fb124ca8640 1 --2- 172.21.15.76:0/327743750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1200cbc30 0x7fb12015f0c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb11000c330 tx=0x7fb1200c5d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:39.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.358+0000 7fb115ffb640 1 -- 172.21.15.76:0/327743750 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb110042070 con 0x7fb1200cbc30 2024-04-25T19:19:39.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.358+0000 7fb115ffb640 1 -- 172.21.15.76:0/327743750 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb110007480 con 0x7fb1200cbc30 2024-04-25T19:19:39.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.358+0000 7fb12770d640 1 -- 172.21.15.76:0/327743750 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1201f2d10 con 0x7fb1200cbc30 2024-04-25T19:19:39.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.358+0000 7fb115ffb640 1 -- 172.21.15.76:0/327743750 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb110001c70 con 0x7fb1200cbc30 2024-04-25T19:19:39.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.358+0000 7fb12770d640 1 -- 172.21.15.76:0/327743750 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1201f32d0 con 0x7fb1200cbc30 2024-04-25T19:19:39.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.359+0000 7fb12770d640 1 -- 172.21.15.76:0/327743750 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb12006a390 con 0x7fb1200cbc30 2024-04-25T19:19:39.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.359+0000 7fb115ffb640 1 -- 172.21.15.76:0/327743750 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb110045070 con 0x7fb1200cbc30 2024-04-25T19:19:39.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.360+0000 7fb115ffb640 1 --2- 172.21.15.76:0/327743750 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb1040744b0 0x7fb1040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.360+0000 7fb115ffb640 1 -- 172.21.15.76:0/327743750 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fb1100b57d0 con 0x7fb1200cbc30 2024-04-25T19:19:39.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.360+0000 7fb115ffb640 1 --2- 172.21.15.76:0/327743750 >> [v2:172.21.15.105:6816/548350430,v1:172.21.15.105:6818/548350430] conn(0x7fb10407c880 0x7fb10407ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.360+0000 7fb115ffb640 1 -- 172.21.15.76:0/327743750 --> [v2:172.21.15.105:6816/548350430,v1:172.21.15.105:6818/548350430] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb110005ad0 con 0x7fb10407c880 2024-04-25T19:19:39.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.360+0000 7fb1254a9640 1 --2- 172.21.15.76:0/327743750 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb1040744b0 0x7fb1040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.360+0000 7fb115ffb640 1 -- 172.21.15.76:0/327743750 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb1100ba2c0 con 0x7fb1200cbc30 2024-04-25T19:19:39.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.360+0000 7fb125caa640 1 --2- 172.21.15.76:0/327743750 >> [v2:172.21.15.105:6816/548350430,v1:172.21.15.105:6818/548350430] conn(0x7fb10407c880 0x7fb10407ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.361+0000 7fb1254a9640 1 --2- 172.21.15.76:0/327743750 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb1040744b0 0x7fb1040769c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb11c009aa0 tx=0x7fb11c009190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:39.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.361+0000 7fb125caa640 1 --2- 172.21.15.76:0/327743750 >> [v2:172.21.15.105:6816/548350430,v1:172.21.15.105:6818/548350430] conn(0x7fb10407c880 0x7fb10407ed50 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:39.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.362+0000 7fb115ffb640 1 -- 172.21.15.76:0/327743750 <== osd.7 v2:172.21.15.105:6816/548350430 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb110005ad0 con 0x7fb10407c880 2024-04-25T19:19:39.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.374+0000 7fb12770d640 1 -- 172.21.15.76:0/327743750 --> [v2:172.21.15.105:6816/548350430,v1:172.21.15.105:6818/548350430] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb1200ca610 con 0x7fb10407c880 2024-04-25T19:19:39.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.375+0000 7fb115ffb640 1 -- 172.21.15.76:0/327743750 <== osd.7 v2:172.21.15.105:6816/548350430 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb1200ca610 con 0x7fb10407c880 2024-04-25T19:19:39.377 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-04-25T19:19:39.377 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T19:19:39.377 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-04-25T19:19:39.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.376+0000 7fb12770d640 1 -- 172.21.15.76:0/327743750 >> [v2:172.21.15.105:6816/548350430,v1:172.21.15.105:6818/548350430] conn(0x7fb10407c880 msgr2=0x7fb10407ed50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:39.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.376+0000 7fb12770d640 1 --2- 172.21.15.76:0/327743750 >> [v2:172.21.15.105:6816/548350430,v1:172.21.15.105:6818/548350430] conn(0x7fb10407c880 0x7fb10407ed50 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.376+0000 7fb12770d640 1 -- 172.21.15.76:0/327743750 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb1040744b0 msgr2=0x7fb1040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:39.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.376+0000 7fb12770d640 1 --2- 172.21.15.76:0/327743750 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fb1040744b0 0x7fb1040769c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb11c009aa0 tx=0x7fb11c009190 comp rx=0 tx=0).stop 2024-04-25T19:19:39.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.376+0000 7fb12770d640 1 -- 172.21.15.76:0/327743750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1200cbc30 msgr2=0x7fb12015f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:39.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.376+0000 7fb12770d640 1 --2- 172.21.15.76:0/327743750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1200cbc30 0x7fb12015f0c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb11000c330 tx=0x7fb1200c5d10 comp rx=0 tx=0).stop 2024-04-25T19:19:39.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.376+0000 7fb125caa640 1 -- 172.21.15.76:0/327743750 reap_dead start 2024-04-25T19:19:39.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.376+0000 7fb12770d640 1 -- 172.21.15.76:0/327743750 shutdown_connections 2024-04-25T19:19:39.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.376+0000 7fb12770d640 1 -- 172.21.15.76:0/327743750 >> 172.21.15.76:0/327743750 conn(0x7fb12006d6d0 msgr2=0x7fb1200ce3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:39.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.376+0000 7fb12770d640 1 -- 172.21.15.76:0/327743750 shutdown_connections 2024-04-25T19:19:39.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.376+0000 7fb12770d640 1 -- 172.21.15.76:0/327743750 wait complete. 2024-04-25T19:19:39.388 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T19:19:39.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.496+0000 7f3279d1f640 1 Processor -- start 2024-04-25T19:19:39.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.496+0000 7f3279d1f640 1 -- start start 2024-04-25T19:19:39.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.497+0000 7f3279d1f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3274156f60 0x7f32740cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.497+0000 7f3279d1f640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f32740c95c0 0x7f32740cf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.497+0000 7f3279d1f640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3274073c30 con 0x7f32740c9ea0 2024-04-25T19:19:39.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.497+0000 7f3279d1f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f327415d730 con 0x7f3274156f60 2024-04-25T19:19:39.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.497+0000 7f3279d1f640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f32740746b0 con 0x7f32740c95c0 2024-04-25T19:19:39.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.497+0000 7f32737fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3274156f60 0x7f32740cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.497+0000 7f32737fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3274156f60 0x7f32740cefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52378/0 (socket says 172.21.15.76:52378) 2024-04-25T19:19:39.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.497+0000 7f32737fe640 1 -- 172.21.15.76:0/3584771885 learned_addr learned my addr 172.21.15.76:0/3584771885 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:39.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.497+0000 7f3272ffd640 1 --2- 172.21.15.76:0/3584771885 >> v2:172.21.15.105:3300/0 conn(0x7f32740c95c0 0x7f32740cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.497+0000 7f32737fe640 1 -- 172.21.15.76:0/3584771885 >> v2:172.21.15.105:3300/0 conn(0x7f32740c95c0 msgr2=0x7f32740cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:39.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.497+0000 7f32737fe640 1 --2- 172.21.15.76:0/3584771885 >> v2:172.21.15.105:3300/0 conn(0x7f32740c95c0 0x7f32740cf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.497+0000 7f32737fe640 1 -- 172.21.15.76:0/3584771885 >> v1:172.21.15.105:6789/0 conn(0x7f32740c9ea0 legacy=0x7f32740c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:39.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.498+0000 7f32737fe640 1 -- 172.21.15.76:0/3584771885 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3274073940 con 0x7f3274156f60 2024-04-25T19:19:39.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.498+0000 7f3272ffd640 1 --2- 172.21.15.76:0/3584771885 >> v2:172.21.15.105:3300/0 conn(0x7f32740c95c0 0x7f32740cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:19:39.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.498+0000 7f32737fe640 1 --2- 172.21.15.76:0/3584771885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3274156f60 0x7f32740cefe0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f32740c5bf0 tx=0x7f326402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a356db5da95ef643 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:39.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.498+0000 7f32727fc640 1 -- 172.21.15.76:0/3584771885 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3264037070 con 0x7f3274156f60 2024-04-25T19:19:39.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.498+0000 7f32727fc640 1 -- 172.21.15.76:0/3584771885 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3264032440 con 0x7f3274156f60 2024-04-25T19:19:39.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.498+0000 7f32727fc640 1 -- 172.21.15.76:0/3584771885 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3264032810 con 0x7f3274156f60 2024-04-25T19:19:39.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.499+0000 7f3279d1f640 1 -- 172.21.15.76:0/3584771885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3274156f60 msgr2=0x7f32740cefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:39.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.499+0000 7f3279d1f640 1 --2- 172.21.15.76:0/3584771885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3274156f60 0x7f32740cefe0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f32740c5bf0 tx=0x7f326402a230 comp rx=0 tx=0).stop 2024-04-25T19:19:39.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.499+0000 7f3279d1f640 1 -- 172.21.15.76:0/3584771885 shutdown_connections 2024-04-25T19:19:39.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.499+0000 7f3279d1f640 1 --2- 172.21.15.76:0/3584771885 >> v2:172.21.15.105:3300/0 conn(0x7f32740c95c0 0x7f32740cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.499+0000 7f3279d1f640 1 --2- 172.21.15.76:0/3584771885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3274156f60 0x7f32740cefe0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.499+0000 7f3279d1f640 1 -- 172.21.15.76:0/3584771885 >> 172.21.15.76:0/3584771885 conn(0x7f3274071a30 msgr2=0x7f3274071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:39.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.499+0000 7f3279d1f640 1 -- 172.21.15.76:0/3584771885 shutdown_connections 2024-04-25T19:19:39.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.499+0000 7f3279d1f640 1 -- 172.21.15.76:0/3584771885 wait complete. 2024-04-25T19:19:39.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.501+0000 7f3279d1f640 1 Processor -- start 2024-04-25T19:19:39.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.501+0000 7f3279d1f640 1 -- start start 2024-04-25T19:19:39.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.502+0000 7f3279d1f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32740c95c0 0x7f327414c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.502+0000 7f3279d1f640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f32740c9ea0 0x7f32740cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.502+0000 7f3279d1f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3274156f60 0x7f327414f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.502+0000 7f3279d1f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f32740740e0 con 0x7f3274156f60 2024-04-25T19:19:39.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.502+0000 7f3279d1f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f327415da10 con 0x7f32740c95c0 2024-04-25T19:19:39.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.502+0000 7f3279d1f640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f32740c7ea0 con 0x7f32740c9ea0 2024-04-25T19:19:39.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.502+0000 7f3273fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3274156f60 0x7f327414f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.502+0000 7f3273fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3274156f60 0x7f327414f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52392/0 (socket says 172.21.15.76:52392) 2024-04-25T19:19:39.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.502+0000 7f3273fff640 1 -- 172.21.15.76:0/3971691670 learned_addr learned my addr 172.21.15.76:0/3971691670 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:39.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.503+0000 7f32737fe640 1 --2- 172.21.15.76:0/3971691670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32740c95c0 0x7f327414c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.503+0000 7f3272ffd640 1 --2- 172.21.15.76:0/3971691670 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f32740c9ea0 0x7f32740cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.503+0000 7f3273fff640 1 -- 172.21.15.76:0/3971691670 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f32740c9ea0 msgr2=0x7f32740cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:39.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.503+0000 7f3273fff640 1 --2- 172.21.15.76:0/3971691670 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f32740c9ea0 0x7f32740cbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.503+0000 7f3273fff640 1 -- 172.21.15.76:0/3971691670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32740c95c0 msgr2=0x7f327414c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:39.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.503+0000 7f3273fff640 1 --2- 172.21.15.76:0/3971691670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32740c95c0 0x7f327414c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.503+0000 7f3273fff640 1 -- 172.21.15.76:0/3971691670 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32741f7390 con 0x7f3274156f60 2024-04-25T19:19:39.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.503+0000 7f32737fe640 1 --2- 172.21.15.76:0/3971691670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32740c95c0 0x7f327414c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:19:39.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.503+0000 7f3273fff640 1 --2- 172.21.15.76:0/3971691670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3274156f60 0x7f327414f1e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f326c00ba10 tx=0x7f326c00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:39.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.503+0000 7f325bfff640 1 -- 172.21.15.76:0/3971691670 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c019070 con 0x7f3274156f60 2024-04-25T19:19:39.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.503+0000 7f325bfff640 1 -- 172.21.15.76:0/3971691670 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f326c00cb40 con 0x7f3274156f60 2024-04-25T19:19:39.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.503+0000 7f3279d1f640 1 -- 172.21.15.76:0/3971691670 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f327414c240 con 0x7f3274156f60 2024-04-25T19:19:39.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.503+0000 7f325bfff640 1 -- 172.21.15.76:0/3971691670 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c014440 con 0x7f3274156f60 2024-04-25T19:19:39.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.503+0000 7f3279d1f640 1 -- 172.21.15.76:0/3971691670 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f327414da90 con 0x7f3274156f60 2024-04-25T19:19:39.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.504+0000 7f3279d1f640 1 -- 172.21.15.76:0/3971691670 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3274155b60 con 0x7f3274156f60 2024-04-25T19:19:39.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.505+0000 7f325bfff640 1 -- 172.21.15.76:0/3971691670 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f326c01c070 con 0x7f3274156f60 2024-04-25T19:19:39.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.505+0000 7f325bfff640 1 --2- 172.21.15.76:0/3971691670 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f32600744b0 0x7f32600769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.505+0000 7f325bfff640 1 -- 172.21.15.76:0/3971691670 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f326c094e90 con 0x7f3274156f60 2024-04-25T19:19:39.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.506+0000 7f32737fe640 1 --2- 172.21.15.76:0/3971691670 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f32600744b0 0x7f32600769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.506+0000 7f32737fe640 1 --2- 172.21.15.76:0/3971691670 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f32600744b0 0x7f32600769c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3264034000 tx=0x7f326402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:39.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.510+0000 7f325bfff640 1 -- 172.21.15.76:0/3971691670 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3274155b60 con 0x7f3274156f60 2024-04-25T19:19:39.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.686+0000 7f3279d1f640 1 -- 172.21.15.76:0/3971691670 --> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f327414bff0 con 0x7f32600744b0 2024-04-25T19:19:39.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.687+0000 7f325bfff640 1 -- 172.21.15.76:0/3971691670 <== mgr.4109 v2:172.21.15.105:6832/2585775531 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f327414bff0 con 0x7f32600744b0 2024-04-25T19:19:39.688 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T19:19:39.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.689+0000 7f3279d1f640 1 -- 172.21.15.76:0/3971691670 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f32600744b0 msgr2=0x7f32600769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:39.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.689+0000 7f3279d1f640 1 --2- 172.21.15.76:0/3971691670 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f32600744b0 0x7f32600769c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3264034000 tx=0x7f326402e000 comp rx=0 tx=0).stop 2024-04-25T19:19:39.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.689+0000 7f3279d1f640 1 -- 172.21.15.76:0/3971691670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3274156f60 msgr2=0x7f327414f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:39.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.689+0000 7f3279d1f640 1 --2- 172.21.15.76:0/3971691670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3274156f60 0x7f327414f1e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f326c00ba10 tx=0x7f326c00bed0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.689+0000 7f3279d1f640 1 -- 172.21.15.76:0/3971691670 shutdown_connections 2024-04-25T19:19:39.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.689+0000 7f3279d1f640 1 --2- 172.21.15.76:0/3971691670 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f32600744b0 0x7f32600769c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.689+0000 7f3279d1f640 1 --2- 172.21.15.76:0/3971691670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3274156f60 0x7f327414f1e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.689+0000 7f3279d1f640 1 --2- 172.21.15.76:0/3971691670 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f32740c9ea0 0x7f32740cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.689+0000 7f3279d1f640 1 --2- 172.21.15.76:0/3971691670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32740c95c0 0x7f327414c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.689+0000 7f3279d1f640 1 -- 172.21.15.76:0/3971691670 >> 172.21.15.76:0/3971691670 conn(0x7f3274071a30 msgr2=0x7f327415cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:39.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.689+0000 7f3279d1f640 1 -- 172.21.15.76:0/3971691670 shutdown_connections 2024-04-25T19:19:39.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.689+0000 7f3279d1f640 1 -- 172.21.15.76:0/3971691670 wait complete. 2024-04-25T19:19:39.701 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T19:19:39.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.808+0000 7f254d19d640 1 Processor -- start 2024-04-25T19:19:39.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.808+0000 7f254d19d640 1 -- start start 2024-04-25T19:19:39.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.810+0000 7f254d19d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f25480714c0 0x7f2548154a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.810+0000 7f254d19d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2548154f80 con 0x7f254806d120 2024-04-25T19:19:39.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.810+0000 7f254d19d640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f254806c7b0 con 0x7f25480c8440 2024-04-25T19:19:39.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.810+0000 7f254d19d640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f25480cb490 con 0x7f25480714c0 2024-04-25T19:19:39.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.810+0000 7f253ffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f254806d120 0x7f254815e480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:34218/0 (socket says 172.21.15.76:34218) 2024-04-25T19:19:39.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.810+0000 7f253ffff640 1 -- 172.21.15.76:0/2980840998 learned_addr learned my addr 172.21.15.76:0/2980840998 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:19:39.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.810+0000 7f2546d9d640 1 --2- 172.21.15.76:0/2980840998 >> v2:172.21.15.105:3300/0 conn(0x7f25480714c0 0x7f2548154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.810+0000 7f254659c640 1 -- 172.21.15.76:0/2980840998 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3595414357 0 0) 0x7f2548154f80 con 0x7f254806d120 2024-04-25T19:19:39.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.810+0000 7f254659c640 1 -- 172.21.15.76:0/2980840998 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2534003520 con 0x7f254806d120 2024-04-25T19:19:39.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.810+0000 7f254659c640 1 -- 172.21.15.76:0/2980840998 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1442213726 0 0) 0x7f2534003520 con 0x7f254806d120 2024-04-25T19:19:39.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.810+0000 7f254659c640 1 -- 172.21.15.76:0/2980840998 >> v2:172.21.15.105:3300/0 conn(0x7f25480714c0 msgr2=0x7f2548154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:39.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.810+0000 7f254659c640 1 --2- 172.21.15.76:0/2980840998 >> v2:172.21.15.105:3300/0 conn(0x7f25480714c0 0x7f2548154a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.810+0000 7f254659c640 1 -- 172.21.15.76:0/2980840998 >> v1:172.21.15.105:6789/0 conn(0x7f25480c8440 legacy=0x7f2548165170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:39.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.810+0000 7f254659c640 1 -- 172.21.15.76:0/2980840998 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25480ca230 con 0x7f254806d120 2024-04-25T19:19:39.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.810+0000 7f254659c640 1 -- 172.21.15.76:0/2980840998 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2162533571 0 0) 0x7f2530001670 con 0x7f254806d120 2024-04-25T19:19:39.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.810+0000 7f254659c640 1 -- 172.21.15.76:0/2980840998 >> v1:172.21.15.76:6789/0 conn(0x7f254806d120 legacy=0x7f254815e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:39.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.810+0000 7f254659c640 1 --2- 172.21.15.76:0/2980840998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25340039b0 0x7f2534003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.811+0000 7f254659c640 1 --2- 172.21.15.76:0/2980840998 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2534004800 0x7f2534004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.811+0000 7f254659c640 1 --2- 172.21.15.76:0/2980840998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25340051c0 0x7f253400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.811+0000 7f254659c640 1 -- 172.21.15.76:0/2980840998 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f25480cb490 con 0x7f25340039b0 2024-04-25T19:19:39.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.811+0000 7f254659c640 1 -- 172.21.15.76:0/2980840998 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f253400c150 con 0x7f25340051c0 2024-04-25T19:19:39.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.811+0000 7f254659c640 1 -- 172.21.15.76:0/2980840998 --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f253400c350 con 0x7f2534004800 2024-04-25T19:19:39.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.811+0000 7f2546d9d640 1 --2- 172.21.15.76:0/2980840998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25340039b0 0x7f2534003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.811+0000 7f253ffff640 1 --2- 172.21.15.76:0/2980840998 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2534004800 0x7f2534004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.811+0000 7f254759e640 1 --2- 172.21.15.76:0/2980840998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25340051c0 0x7f253400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.811+0000 7f2546d9d640 1 -- 172.21.15.76:0/2980840998 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2534004800 msgr2=0x7f2534004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:39.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.811+0000 7f2546d9d640 1 --2- 172.21.15.76:0/2980840998 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f2534004800 0x7f2534004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.811+0000 7f2546d9d640 1 -- 172.21.15.76:0/2980840998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25340051c0 msgr2=0x7f253400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:19:39.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.811+0000 7f2546d9d640 1 --2- 172.21.15.76:0/2980840998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25340051c0 0x7f253400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.811+0000 7f2546d9d640 1 -- 172.21.15.76:0/2980840998 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2534003520 con 0x7f25340039b0 2024-04-25T19:19:39.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.811+0000 7f254759e640 1 --2- 172.21.15.76:0/2980840998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25340051c0 0x7f253400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:19:39.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.811+0000 7f254759e640 1 -- 172.21.15.76:0/2980840998 reap_dead start 2024-04-25T19:19:39.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.812+0000 7f2546d9d640 1 --2- 172.21.15.76:0/2980840998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25340039b0 0x7f2534003e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2538009ac0 tx=0x7f2534005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=542fae6f3c0cd7af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:39.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.812+0000 7f254659c640 1 -- 172.21.15.76:0/2980840998 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538036070 con 0x7f25340039b0 2024-04-25T19:19:39.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.812+0000 7f254659c640 1 -- 172.21.15.76:0/2980840998 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2538031400 con 0x7f25340039b0 2024-04-25T19:19:39.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.812+0000 7f254659c640 1 -- 172.21.15.76:0/2980840998 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538009690 con 0x7f25340039b0 2024-04-25T19:19:39.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.812+0000 7f254d19d640 1 -- 172.21.15.76:0/2980840998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25340039b0 msgr2=0x7f2534003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:39.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.812+0000 7f254d19d640 1 --2- 172.21.15.76:0/2980840998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25340039b0 0x7f2534003e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2538009ac0 tx=0x7f2534005830 comp rx=0 tx=0).stop 2024-04-25T19:19:39.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.812+0000 7f254d19d640 1 -- 172.21.15.76:0/2980840998 shutdown_connections 2024-04-25T19:19:39.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.812+0000 7f254d19d640 1 --2- 172.21.15.76:0/2980840998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25340039b0 0x7f2534003e20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.812+0000 7f254d19d640 1 -- 172.21.15.76:0/2980840998 >> 172.21.15.76:0/2980840998 conn(0x7f254806d6d0 msgr2=0x7f254806dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:39.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.812+0000 7f254d19d640 1 -- 172.21.15.76:0/2980840998 shutdown_connections 2024-04-25T19:19:39.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.812+0000 7f254d19d640 1 -- 172.21.15.76:0/2980840998 wait complete. 2024-04-25T19:19:39.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.814+0000 7f254d19d640 1 Processor -- start 2024-04-25T19:19:39.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.814+0000 7f254d19d640 1 -- start start 2024-04-25T19:19:39.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.816+0000 7f254d19d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25340039b0 0x7f2548152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.816+0000 7f254d19d640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25480c65f0 0x7f25481e76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.816+0000 7f254d19d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f254815cc10 0x7f254815d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.816+0000 7f254d19d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f254815df60 con 0x7f254815cc10 2024-04-25T19:19:39.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.816+0000 7f254d19d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f25480ca7f0 con 0x7f25340039b0 2024-04-25T19:19:39.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.816+0000 7f254d19d640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f25480724b0 con 0x7f25480c65f0 2024-04-25T19:19:39.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.816+0000 7f254759e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f254815cc10 0x7f254815d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.816+0000 7f254759e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f254815cc10 0x7f254815d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52422/0 (socket says 172.21.15.76:52422) 2024-04-25T19:19:39.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.816+0000 7f254759e640 1 -- 172.21.15.76:0/273760981 learned_addr learned my addr 172.21.15.76:0/273760981 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:19:39.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.816+0000 7f2546d9d640 1 --2- 172.21.15.76:0/273760981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25340039b0 0x7f2548152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.816+0000 7f254759e640 1 -- 172.21.15.76:0/273760981 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25480c65f0 msgr2=0x7f25481e76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:39.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.816+0000 7f253ffff640 1 --2- 172.21.15.76:0/273760981 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25480c65f0 0x7f25481e76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.816+0000 7f254759e640 1 --2- 172.21.15.76:0/273760981 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25480c65f0 0x7f25481e76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.816+0000 7f254759e640 1 -- 172.21.15.76:0/273760981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25340039b0 msgr2=0x7f2548152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:39.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.816+0000 7f254759e640 1 --2- 172.21.15.76:0/273760981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25340039b0 0x7f2548152000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:39.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.816+0000 7f254759e640 1 -- 172.21.15.76:0/273760981 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2548158db0 con 0x7f254815cc10 2024-04-25T19:19:39.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.816+0000 7f2546d9d640 1 --2- 172.21.15.76:0/273760981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25340039b0 0x7f2548152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:19:39.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.816+0000 7f254759e640 1 --2- 172.21.15.76:0/273760981 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f254815cc10 0x7f254815d020 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2548070b80 tx=0x7f254000d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:39.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.816+0000 7f253f7fe640 1 -- 172.21.15.76:0/273760981 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2540018070 con 0x7f254815cc10 2024-04-25T19:19:39.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.816+0000 7f253f7fe640 1 -- 172.21.15.76:0/273760981 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2540003da0 con 0x7f254815cc10 2024-04-25T19:19:39.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.816+0000 7f254d19d640 1 -- 172.21.15.76:0/273760981 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2548152540 con 0x7f254815cc10 2024-04-25T19:19:39.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.816+0000 7f253f7fe640 1 -- 172.21.15.76:0/273760981 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25400052b0 con 0x7f254815cc10 2024-04-25T19:19:39.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.816+0000 7f254d19d640 1 -- 172.21.15.76:0/273760981 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2548152750 con 0x7f254815cc10 2024-04-25T19:19:39.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.817+0000 7f254d19d640 1 -- 172.21.15.76:0/273760981 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f254806a390 con 0x7f254815cc10 2024-04-25T19:19:39.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.818+0000 7f253f7fe640 1 -- 172.21.15.76:0/273760981 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f254001b070 con 0x7f254815cc10 2024-04-25T19:19:39.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.818+0000 7f253f7fe640 1 --2- 172.21.15.76:0/273760981 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f25280744e0 0x7f25280769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:19:39.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.819+0000 7f2546d9d640 1 --2- 172.21.15.76:0/273760981 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f25280744e0 0x7f25280769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:19:39.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.819+0000 7f253f7fe640 1 -- 172.21.15.76:0/273760981 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f25400948f0 con 0x7f254815cc10 2024-04-25T19:19:39.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.819+0000 7f2546d9d640 1 --2- 172.21.15.76:0/273760981 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f25280744e0 0x7f25280769f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2538033000 tx=0x7f2538008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:19:39.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.823+0000 7f253f7fe640 1 -- 172.21.15.76:0/273760981 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f254005f120 con 0x7f254815cc10 2024-04-25T19:19:39.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:39.996+0000 7f254d19d640 1 -- 172.21.15.76:0/273760981 --> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f25480c9bf0 con 0x7f25280744e0 2024-04-25T19:19:40.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:40.003+0000 7f253f7fe640 1 -- 172.21.15.76:0/273760981 <== mgr.4109 v2:172.21.15.105:6832/2585775531 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251561 (secure 0 0 0) 0x7f25480c9bf0 con 0x7f25280744e0 2024-04-25T19:19:40.007 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:19:40.008 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-04-25T19:19:40.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:40.006+0000 7f254d19d640 1 -- 172.21.15.76:0/273760981 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f25280744e0 msgr2=0x7f25280769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:40.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:40.006+0000 7f254d19d640 1 --2- 172.21.15.76:0/273760981 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f25280744e0 0x7f25280769f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2538033000 tx=0x7f2538008030 comp rx=0 tx=0).stop 2024-04-25T19:19:40.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:40.006+0000 7f254d19d640 1 -- 172.21.15.76:0/273760981 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f254815cc10 msgr2=0x7f254815d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:19:40.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:40.006+0000 7f254d19d640 1 --2- 172.21.15.76:0/273760981 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f254815cc10 0x7f254815d020 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2548070b80 tx=0x7f254000d9f0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:40.006+0000 7f254d19d640 1 -- 172.21.15.76:0/273760981 shutdown_connections 2024-04-25T19:19:40.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:40.006+0000 7f254d19d640 1 --2- 172.21.15.76:0/273760981 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f25280744e0 0x7f25280769f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:40.006+0000 7f254d19d640 1 --2- 172.21.15.76:0/273760981 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f254815cc10 0x7f254815d020 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:40.006+0000 7f254d19d640 1 --2- 172.21.15.76:0/273760981 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f25480c65f0 0x7f25481e76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:40.006+0000 7f254d19d640 1 --2- 172.21.15.76:0/273760981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f25340039b0 0x7f2548152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:19:40.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:40.006+0000 7f254d19d640 1 -- 172.21.15.76:0/273760981 >> 172.21.15.76:0/273760981 conn(0x7f254806d6d0 msgr2=0x7f25480ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:19:40.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:40.006+0000 7f254d19d640 1 -- 172.21.15.76:0/273760981 shutdown_connections 2024-04-25T19:19:40.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:19:40.006+0000 7f254d19d640 1 -- 172.21.15.76:0/273760981 wait complete. 2024-04-25T19:19:40.019 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-04-25T19:19:39.898430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":815835,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":70,"num_read_kb":59,"num_write":9670,"num_write_kb":25167794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10119,"ondisk_log_size":10119,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":306184,"kb_used_data":2696,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754668536,"statfs":{"total":773094113280,"available":772780580864,"internally_reserved":0,"allocated":2760704,"data_stored":2334030,"data_compressed":64838,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":310693424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":672,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004059"},"pg_stats":[{"pgid":"3.3f","version":"23'146","reported_seq":164,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.307942+0000","last_change":"2024-04-25T19:15:19.433569+0000","last_active":"2024-04-25T19:19:33.307942+0000","last_peered":"2024-04-25T19:19:33.307942+0000","last_clean":"2024-04-25T19:19:33.307942+0000","last_became_active":"2024-04-25T19:15:19.431332+0000","last_became_peered":"2024-04-25T19:15:19.431332+0000","last_unstale":"2024-04-25T19:19:33.307942+0000","last_undegraded":"2024-04-25T19:19:33.307942+0000","last_fullsized":"2024-04-25T19:19:33.307942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:57:01.213834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'130","reported_seq":149,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.303922+0000","last_change":"2024-04-25T19:15:19.418695+0000","last_active":"2024-04-25T19:19:33.303922+0000","last_peered":"2024-04-25T19:19:33.303922+0000","last_clean":"2024-04-25T19:19:33.303922+0000","last_became_active":"2024-04-25T19:15:19.418235+0000","last_became_peered":"2024-04-25T19:15:19.418235+0000","last_unstale":"2024-04-25T19:19:33.303922+0000","last_undegraded":"2024-04-25T19:19:33.303922+0000","last_fullsized":"2024-04-25T19:19:33.303922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'151","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.294871+0000","last_change":"2024-04-25T19:15:19.416983+0000","last_active":"2024-04-25T19:19:33.294871+0000","last_peered":"2024-04-25T19:19:33.294871+0000","last_clean":"2024-04-25T19:19:33.294871+0000","last_became_active":"2024-04-25T19:15:19.416443+0000","last_became_peered":"2024-04-25T19:15:19.416443+0000","last_unstale":"2024-04-25T19:19:33.294871+0000","last_undegraded":"2024-04-25T19:19:33.294871+0000","last_fullsized":"2024-04-25T19:19:33.294871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'173","reported_seq":191,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.315372+0000","last_change":"2024-04-25T19:15:19.429058+0000","last_active":"2024-04-25T19:19:33.315372+0000","last_peered":"2024-04-25T19:19:33.315372+0000","last_clean":"2024-04-25T19:19:33.315372+0000","last_became_active":"2024-04-25T19:15:19.428419+0000","last_became_peered":"2024-04-25T19:15:19.428419+0000","last_unstale":"2024-04-25T19:19:33.315372+0000","last_undegraded":"2024-04-25T19:19:33.315372+0000","last_fullsized":"2024-04-25T19:19:33.315372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:23:56.817616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'148","reported_seq":165,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.291531+0000","last_change":"2024-04-25T19:15:19.427615+0000","last_active":"2024-04-25T19:19:33.291531+0000","last_peered":"2024-04-25T19:19:33.291531+0000","last_clean":"2024-04-25T19:19:33.291531+0000","last_became_active":"2024-04-25T19:15:19.426968+0000","last_became_peered":"2024-04-25T19:15:19.426968+0000","last_unstale":"2024-04-25T19:19:33.291531+0000","last_undegraded":"2024-04-25T19:19:33.291531+0000","last_fullsized":"2024-04-25T19:19:33.291531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:42:00.686712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'135","reported_seq":152,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.320601+0000","last_change":"2024-04-25T19:15:19.417602+0000","last_active":"2024-04-25T19:19:33.320601+0000","last_peered":"2024-04-25T19:19:33.320601+0000","last_clean":"2024-04-25T19:19:33.320601+0000","last_became_active":"2024-04-25T19:15:19.417292+0000","last_became_peered":"2024-04-25T19:15:19.417292+0000","last_unstale":"2024-04-25T19:19:33.320601+0000","last_undegraded":"2024-04-25T19:19:33.320601+0000","last_fullsized":"2024-04-25T19:19:33.320601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:37:43.583809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'164","reported_seq":181,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.298724+0000","last_change":"2024-04-25T19:15:19.428180+0000","last_active":"2024-04-25T19:19:33.298724+0000","last_peered":"2024-04-25T19:19:33.298724+0000","last_clean":"2024-04-25T19:19:33.298724+0000","last_became_active":"2024-04-25T19:15:19.427852+0000","last_became_peered":"2024-04-25T19:15:19.427852+0000","last_unstale":"2024-04-25T19:19:33.298724+0000","last_undegraded":"2024-04-25T19:19:33.298724+0000","last_fullsized":"2024-04-25T19:19:33.298724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:51.300880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'186","reported_seq":204,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.314701+0000","last_change":"2024-04-25T19:15:19.418253+0000","last_active":"2024-04-25T19:19:33.314701+0000","last_peered":"2024-04-25T19:19:33.314701+0000","last_clean":"2024-04-25T19:19:33.314701+0000","last_became_active":"2024-04-25T19:15:19.417183+0000","last_became_peered":"2024-04-25T19:15:19.417183+0000","last_unstale":"2024-04-25T19:19:33.314701+0000","last_undegraded":"2024-04-25T19:19:33.314701+0000","last_fullsized":"2024-04-25T19:19:33.314701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'152","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.295790+0000","last_change":"2024-04-25T19:15:19.431671+0000","last_active":"2024-04-25T19:19:33.295790+0000","last_peered":"2024-04-25T19:19:33.295790+0000","last_clean":"2024-04-25T19:19:33.295790+0000","last_became_active":"2024-04-25T19:15:19.431240+0000","last_became_peered":"2024-04-25T19:15:19.431240+0000","last_unstale":"2024-04-25T19:19:33.295790+0000","last_undegraded":"2024-04-25T19:19:33.295790+0000","last_fullsized":"2024-04-25T19:19:33.295790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:08:22.759462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'156","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.299566+0000","last_change":"2024-04-25T19:16:17.555086+0000","last_active":"2024-04-25T19:19:33.299566+0000","last_peered":"2024-04-25T19:19:33.299566+0000","last_clean":"2024-04-25T19:19:33.299566+0000","last_became_active":"2024-04-25T19:16:17.554771+0000","last_became_peered":"2024-04-25T19:16:17.554771+0000","last_unstale":"2024-04-25T19:19:33.299566+0000","last_undegraded":"2024-04-25T19:19:33.299566+0000","last_fullsized":"2024-04-25T19:19:33.299566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:22:35.638155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'189","reported_seq":206,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.296465+0000","last_change":"2024-04-25T19:15:19.417596+0000","last_active":"2024-04-25T19:19:33.296465+0000","last_peered":"2024-04-25T19:19:33.296465+0000","last_clean":"2024-04-25T19:19:33.296465+0000","last_became_active":"2024-04-25T19:15:19.417230+0000","last_became_peered":"2024-04-25T19:15:19.417230+0000","last_unstale":"2024-04-25T19:19:33.296465+0000","last_undegraded":"2024-04-25T19:19:33.296465+0000","last_fullsized":"2024-04-25T19:19:33.296465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:28:12.387651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'142","reported_seq":159,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.283123+0000","last_change":"2024-04-25T19:15:19.420564+0000","last_active":"2024-04-25T19:19:33.283123+0000","last_peered":"2024-04-25T19:19:33.283123+0000","last_clean":"2024-04-25T19:19:33.283123+0000","last_became_active":"2024-04-25T19:15:19.420044+0000","last_became_peered":"2024-04-25T19:15:19.420044+0000","last_unstale":"2024-04-25T19:19:33.283123+0000","last_undegraded":"2024-04-25T19:19:33.283123+0000","last_fullsized":"2024-04-25T19:19:33.283123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:04.066030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'151","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.304472+0000","last_change":"2024-04-25T19:15:19.415500+0000","last_active":"2024-04-25T19:19:33.304472+0000","last_peered":"2024-04-25T19:19:33.304472+0000","last_clean":"2024-04-25T19:19:33.304472+0000","last_became_active":"2024-04-25T19:15:19.414007+0000","last_became_peered":"2024-04-25T19:15:19.414007+0000","last_unstale":"2024-04-25T19:19:33.304472+0000","last_undegraded":"2024-04-25T19:19:33.304472+0000","last_fullsized":"2024-04-25T19:19:33.304472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'149","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.306150+0000","last_change":"2024-04-25T19:15:19.423884+0000","last_active":"2024-04-25T19:19:33.306150+0000","last_peered":"2024-04-25T19:19:33.306150+0000","last_clean":"2024-04-25T19:19:33.306150+0000","last_became_active":"2024-04-25T19:15:19.422575+0000","last_became_peered":"2024-04-25T19:15:19.422575+0000","last_unstale":"2024-04-25T19:19:33.306150+0000","last_undegraded":"2024-04-25T19:19:33.306150+0000","last_fullsized":"2024-04-25T19:19:33.306150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'151","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.289389+0000","last_change":"2024-04-25T19:15:19.424267+0000","last_active":"2024-04-25T19:19:33.289389+0000","last_peered":"2024-04-25T19:19:33.289389+0000","last_clean":"2024-04-25T19:19:33.289389+0000","last_became_active":"2024-04-25T19:15:19.422953+0000","last_became_peered":"2024-04-25T19:15:19.422953+0000","last_unstale":"2024-04-25T19:19:33.289389+0000","last_undegraded":"2024-04-25T19:19:33.289389+0000","last_fullsized":"2024-04-25T19:19:33.289389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'147","reported_seq":159,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.296467+0000","last_change":"2024-04-25T19:16:17.801933+0000","last_active":"2024-04-25T19:19:33.296467+0000","last_peered":"2024-04-25T19:19:33.296467+0000","last_clean":"2024-04-25T19:19:33.296467+0000","last_became_active":"2024-04-25T19:16:17.801500+0000","last_became_peered":"2024-04-25T19:16:17.801500+0000","last_unstale":"2024-04-25T19:19:33.296467+0000","last_undegraded":"2024-04-25T19:19:33.296467+0000","last_fullsized":"2024-04-25T19:19:33.296467+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"23'146","reported_seq":163,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.314263+0000","last_change":"2024-04-25T19:15:19.434476+0000","last_active":"2024-04-25T19:19:33.314263+0000","last_peered":"2024-04-25T19:19:33.314263+0000","last_clean":"2024-04-25T19:19:33.314263+0000","last_became_active":"2024-04-25T19:15:19.433836+0000","last_became_peered":"2024-04-25T19:15:19.433836+0000","last_unstale":"2024-04-25T19:19:33.314263+0000","last_undegraded":"2024-04-25T19:19:33.314263+0000","last_fullsized":"2024-04-25T19:19:33.314263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:57:13.234829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"23'157","reported_seq":175,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.294160+0000","last_change":"2024-04-25T19:15:19.415521+0000","last_active":"2024-04-25T19:19:33.294160+0000","last_peered":"2024-04-25T19:19:33.294160+0000","last_clean":"2024-04-25T19:19:33.294160+0000","last_became_active":"2024-04-25T19:15:19.415177+0000","last_became_peered":"2024-04-25T19:15:19.415177+0000","last_unstale":"2024-04-25T19:19:33.294160+0000","last_undegraded":"2024-04-25T19:19:33.294160+0000","last_fullsized":"2024-04-25T19:19:33.294160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'172","reported_seq":189,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.291002+0000","last_change":"2024-04-25T19:15:19.430291+0000","last_active":"2024-04-25T19:19:33.291002+0000","last_peered":"2024-04-25T19:19:33.291002+0000","last_clean":"2024-04-25T19:19:33.291002+0000","last_became_active":"2024-04-25T19:15:19.427226+0000","last_became_peered":"2024-04-25T19:15:19.427226+0000","last_unstale":"2024-04-25T19:19:33.291002+0000","last_undegraded":"2024-04-25T19:19:33.291002+0000","last_fullsized":"2024-04-25T19:19:33.291002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:38.464211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"23'161","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.299290+0000","last_change":"2024-04-25T19:15:19.421696+0000","last_active":"2024-04-25T19:19:33.299290+0000","last_peered":"2024-04-25T19:19:33.299290+0000","last_clean":"2024-04-25T19:19:33.299290+0000","last_became_active":"2024-04-25T19:15:19.421300+0000","last_became_peered":"2024-04-25T19:15:19.421300+0000","last_unstale":"2024-04-25T19:19:33.299290+0000","last_undegraded":"2024-04-25T19:19:33.299290+0000","last_fullsized":"2024-04-25T19:19:33.299290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:15.226398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'137","reported_seq":155,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.309826+0000","last_change":"2024-04-25T19:15:19.427880+0000","last_active":"2024-04-25T19:19:33.309826+0000","last_peered":"2024-04-25T19:19:33.309826+0000","last_clean":"2024-04-25T19:19:33.309826+0000","last_became_active":"2024-04-25T19:15:19.427579+0000","last_became_peered":"2024-04-25T19:15:19.427579+0000","last_unstale":"2024-04-25T19:19:33.309826+0000","last_undegraded":"2024-04-25T19:19:33.309826+0000","last_fullsized":"2024-04-25T19:19:33.309826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'166","reported_seq":183,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.320719+0000","last_change":"2024-04-25T19:15:19.430409+0000","last_active":"2024-04-25T19:19:33.320719+0000","last_peered":"2024-04-25T19:19:33.320719+0000","last_clean":"2024-04-25T19:19:33.320719+0000","last_became_active":"2024-04-25T19:15:19.428305+0000","last_became_peered":"2024-04-25T19:15:19.428305+0000","last_unstale":"2024-04-25T19:19:33.320719+0000","last_undegraded":"2024-04-25T19:19:33.320719+0000","last_fullsized":"2024-04-25T19:19:33.320719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:10:01.268963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"23'134","reported_seq":152,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.311449+0000","last_change":"2024-04-25T19:15:19.424435+0000","last_active":"2024-04-25T19:19:33.311449+0000","last_peered":"2024-04-25T19:19:33.311449+0000","last_clean":"2024-04-25T19:19:33.311449+0000","last_became_active":"2024-04-25T19:15:19.422763+0000","last_became_peered":"2024-04-25T19:15:19.422763+0000","last_unstale":"2024-04-25T19:19:33.311449+0000","last_undegraded":"2024-04-25T19:19:33.311449+0000","last_fullsized":"2024-04-25T19:19:33.311449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'180","reported_seq":197,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.310454+0000","last_change":"2024-04-25T19:15:19.420275+0000","last_active":"2024-04-25T19:19:33.310454+0000","last_peered":"2024-04-25T19:19:33.310454+0000","last_clean":"2024-04-25T19:19:33.310454+0000","last_became_active":"2024-04-25T19:15:19.419936+0000","last_became_peered":"2024-04-25T19:15:19.419936+0000","last_unstale":"2024-04-25T19:19:33.310454+0000","last_undegraded":"2024-04-25T19:19:33.310454+0000","last_fullsized":"2024-04-25T19:19:33.310454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:55.710620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'134","reported_seq":152,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.307254+0000","last_change":"2024-04-25T19:15:19.423408+0000","last_active":"2024-04-25T19:19:33.307254+0000","last_peered":"2024-04-25T19:19:33.307254+0000","last_clean":"2024-04-25T19:19:33.307254+0000","last_became_active":"2024-04-25T19:15:19.423116+0000","last_became_peered":"2024-04-25T19:15:19.423116+0000","last_unstale":"2024-04-25T19:19:33.307254+0000","last_undegraded":"2024-04-25T19:19:33.307254+0000","last_fullsized":"2024-04-25T19:19:33.307254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'198","reported_seq":215,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.310915+0000","last_change":"2024-04-25T19:15:19.428826+0000","last_active":"2024-04-25T19:19:33.310915+0000","last_peered":"2024-04-25T19:19:33.310915+0000","last_clean":"2024-04-25T19:19:33.310915+0000","last_became_active":"2024-04-25T19:15:19.428474+0000","last_became_peered":"2024-04-25T19:15:19.428474+0000","last_unstale":"2024-04-25T19:19:33.310915+0000","last_undegraded":"2024-04-25T19:19:33.310915+0000","last_fullsized":"2024-04-25T19:19:33.310915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:19.974851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'144","reported_seq":162,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.299922+0000","last_change":"2024-04-25T19:15:19.423353+0000","last_active":"2024-04-25T19:19:33.299922+0000","last_peered":"2024-04-25T19:19:33.299922+0000","last_clean":"2024-04-25T19:19:33.299922+0000","last_became_active":"2024-04-25T19:15:19.422795+0000","last_became_peered":"2024-04-25T19:15:19.422795+0000","last_unstale":"2024-04-25T19:19:33.299922+0000","last_undegraded":"2024-04-25T19:19:33.299922+0000","last_fullsized":"2024-04-25T19:19:33.299922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'133","reported_seq":151,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.310700+0000","last_change":"2024-04-25T19:15:19.421809+0000","last_active":"2024-04-25T19:19:33.310700+0000","last_peered":"2024-04-25T19:19:33.310700+0000","last_clean":"2024-04-25T19:19:33.310700+0000","last_became_active":"2024-04-25T19:15:19.421515+0000","last_became_peered":"2024-04-25T19:15:19.421515+0000","last_unstale":"2024-04-25T19:19:33.310700+0000","last_undegraded":"2024-04-25T19:19:33.310700+0000","last_fullsized":"2024-04-25T19:19:33.310700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'118","reported_seq":137,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.303227+0000","last_change":"2024-04-25T19:15:19.410183+0000","last_active":"2024-04-25T19:19:33.303227+0000","last_peered":"2024-04-25T19:19:33.303227+0000","last_clean":"2024-04-25T19:19:33.303227+0000","last_became_active":"2024-04-25T19:15:19.409857+0000","last_became_peered":"2024-04-25T19:15:19.409857+0000","last_unstale":"2024-04-25T19:19:33.303227+0000","last_undegraded":"2024-04-25T19:19:33.303227+0000","last_fullsized":"2024-04-25T19:19:33.303227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'131","reported_seq":149,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.316128+0000","last_change":"2024-04-25T19:15:19.425524+0000","last_active":"2024-04-25T19:19:33.316128+0000","last_peered":"2024-04-25T19:19:33.316128+0000","last_clean":"2024-04-25T19:19:33.316128+0000","last_became_active":"2024-04-25T19:15:19.424578+0000","last_became_peered":"2024-04-25T19:15:19.424578+0000","last_unstale":"2024-04-25T19:19:33.316128+0000","last_undegraded":"2024-04-25T19:19:33.316128+0000","last_fullsized":"2024-04-25T19:19:33.316128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:02.103879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"23'135","reported_seq":154,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.286940+0000","last_change":"2024-04-25T19:15:19.426720+0000","last_active":"2024-04-25T19:19:33.286940+0000","last_peered":"2024-04-25T19:19:33.286940+0000","last_clean":"2024-04-25T19:19:33.286940+0000","last_became_active":"2024-04-25T19:15:19.426139+0000","last_became_peered":"2024-04-25T19:15:19.426139+0000","last_unstale":"2024-04-25T19:19:33.286940+0000","last_undegraded":"2024-04-25T19:19:33.286940+0000","last_fullsized":"2024-04-25T19:19:33.286940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'150","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.300745+0000","last_change":"2024-04-25T19:15:19.413805+0000","last_active":"2024-04-25T19:19:33.300745+0000","last_peered":"2024-04-25T19:19:33.300745+0000","last_clean":"2024-04-25T19:19:33.300745+0000","last_became_active":"2024-04-25T19:15:19.413455+0000","last_became_peered":"2024-04-25T19:15:19.413455+0000","last_unstale":"2024-04-25T19:19:33.300745+0000","last_undegraded":"2024-04-25T19:19:33.300745+0000","last_fullsized":"2024-04-25T19:19:33.300745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:05:45.008076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'130","reported_seq":147,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.278189+0000","last_change":"2024-04-25T19:15:19.423770+0000","last_active":"2024-04-25T19:19:33.278189+0000","last_peered":"2024-04-25T19:19:33.278189+0000","last_clean":"2024-04-25T19:19:33.278189+0000","last_became_active":"2024-04-25T19:15:19.423406+0000","last_became_peered":"2024-04-25T19:15:19.423406+0000","last_unstale":"2024-04-25T19:19:33.278189+0000","last_undegraded":"2024-04-25T19:19:33.278189+0000","last_fullsized":"2024-04-25T19:19:33.278189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:41:23.669832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"23'7","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.257787+0000","last_change":"2024-04-25T19:15:18.422035+0000","last_active":"2024-04-25T19:19:33.257787+0000","last_peered":"2024-04-25T19:19:33.257787+0000","last_clean":"2024-04-25T19:19:33.257787+0000","last_became_active":"2024-04-25T19:15:18.421658+0000","last_became_peered":"2024-04-25T19:15:18.421658+0000","last_unstale":"2024-04-25T19:19:33.257787+0000","last_undegraded":"2024-04-25T19:19:33.257787+0000","last_fullsized":"2024-04-25T19:19:33.257787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'137","reported_seq":155,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.307676+0000","last_change":"2024-04-25T19:15:19.421805+0000","last_active":"2024-04-25T19:19:33.307676+0000","last_peered":"2024-04-25T19:19:33.307676+0000","last_clean":"2024-04-25T19:19:33.307676+0000","last_became_active":"2024-04-25T19:15:19.421497+0000","last_became_peered":"2024-04-25T19:15:19.421497+0000","last_unstale":"2024-04-25T19:19:33.307676+0000","last_undegraded":"2024-04-25T19:19:33.307676+0000","last_fullsized":"2024-04-25T19:19:33.307676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'7","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.553158+0000","last_change":"2024-04-25T19:15:18.416894+0000","last_active":"2024-04-25T19:16:17.553158+0000","last_peered":"2024-04-25T19:16:17.553158+0000","last_clean":"2024-04-25T19:16:17.553158+0000","last_became_active":"2024-04-25T19:15:18.416539+0000","last_became_peered":"2024-04-25T19:15:18.416539+0000","last_unstale":"2024-04-25T19:16:17.553158+0000","last_undegraded":"2024-04-25T19:16:17.553158+0000","last_fullsized":"2024-04-25T19:16:17.553158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:30.388606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:42.866165+0000","last_change":"2024-04-25T19:15:18.420129+0000","last_active":"2024-04-25T19:17:42.866165+0000","last_peered":"2024-04-25T19:17:42.866165+0000","last_clean":"2024-04-25T19:17:42.866165+0000","last_became_active":"2024-04-25T19:15:18.419732+0000","last_became_peered":"2024-04-25T19:15:18.419732+0000","last_unstale":"2024-04-25T19:17:42.866165+0000","last_undegraded":"2024-04-25T19:17:42.866165+0000","last_fullsized":"2024-04-25T19:17:42.866165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'170","reported_seq":182,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.305379+0000","last_change":"2024-04-25T19:16:17.801934+0000","last_active":"2024-04-25T19:19:33.305379+0000","last_peered":"2024-04-25T19:19:33.305379+0000","last_clean":"2024-04-25T19:19:33.305379+0000","last_became_active":"2024-04-25T19:16:17.801451+0000","last_became_peered":"2024-04-25T19:16:17.801451+0000","last_unstale":"2024-04-25T19:19:33.305379+0000","last_undegraded":"2024-04-25T19:19:33.305379+0000","last_fullsized":"2024-04-25T19:19:33.305379+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"21'3","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.550543+0000","last_change":"2024-04-25T19:15:18.420711+0000","last_active":"2024-04-25T19:16:17.550543+0000","last_peered":"2024-04-25T19:16:17.550543+0000","last_clean":"2024-04-25T19:16:17.550543+0000","last_became_active":"2024-04-25T19:15:18.420362+0000","last_became_peered":"2024-04-25T19:15:18.420362+0000","last_unstale":"2024-04-25T19:16:17.550543+0000","last_undegraded":"2024-04-25T19:16:17.550543+0000","last_fullsized":"2024-04-25T19:16:17.550543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:48.245193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'157","reported_seq":175,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.303530+0000","last_change":"2024-04-25T19:15:19.424356+0000","last_active":"2024-04-25T19:19:33.303530+0000","last_peered":"2024-04-25T19:19:33.303530+0000","last_clean":"2024-04-25T19:19:33.303530+0000","last_became_active":"2024-04-25T19:15:19.423556+0000","last_became_peered":"2024-04-25T19:15:19.423556+0000","last_unstale":"2024-04-25T19:19:33.303530+0000","last_undegraded":"2024-04-25T19:19:33.303530+0000","last_fullsized":"2024-04-25T19:19:33.303530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'9","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.258205+0000","last_change":"2024-04-25T19:15:18.419250+0000","last_active":"2024-04-25T19:19:33.258205+0000","last_peered":"2024-04-25T19:19:33.258205+0000","last_clean":"2024-04-25T19:19:33.258205+0000","last_became_active":"2024-04-25T19:15:18.418927+0000","last_became_peered":"2024-04-25T19:15:18.418927+0000","last_unstale":"2024-04-25T19:19:33.258205+0000","last_undegraded":"2024-04-25T19:19:33.258205+0000","last_fullsized":"2024-04-25T19:19:33.258205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:26.012965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'229","reported_seq":246,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.315211+0000","last_change":"2024-04-25T19:15:19.430117+0000","last_active":"2024-04-25T19:19:33.315211+0000","last_peered":"2024-04-25T19:19:33.315211+0000","last_clean":"2024-04-25T19:19:33.315211+0000","last_became_active":"2024-04-25T19:15:19.428985+0000","last_became_peered":"2024-04-25T19:15:19.428985+0000","last_unstale":"2024-04-25T19:19:33.315211+0000","last_undegraded":"2024-04-25T19:19:33.315211+0000","last_fullsized":"2024-04-25T19:19:33.315211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:36:29.069541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"21'4","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.550765+0000","last_change":"2024-04-25T19:15:18.422931+0000","last_active":"2024-04-25T19:16:17.550765+0000","last_peered":"2024-04-25T19:16:17.550765+0000","last_clean":"2024-04-25T19:16:17.550765+0000","last_became_active":"2024-04-25T19:15:18.422610+0000","last_became_peered":"2024-04-25T19:15:18.422610+0000","last_unstale":"2024-04-25T19:16:17.550765+0000","last_undegraded":"2024-04-25T19:16:17.550765+0000","last_fullsized":"2024-04-25T19:16:17.550765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:30:18.489484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'141","reported_seq":159,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.296292+0000","last_change":"2024-04-25T19:15:19.416109+0000","last_active":"2024-04-25T19:19:33.296292+0000","last_peered":"2024-04-25T19:19:33.296292+0000","last_clean":"2024-04-25T19:19:33.296292+0000","last_became_active":"2024-04-25T19:15:19.415734+0000","last_became_peered":"2024-04-25T19:15:19.415734+0000","last_unstale":"2024-04-25T19:19:33.296292+0000","last_undegraded":"2024-04-25T19:19:33.296292+0000","last_fullsized":"2024-04-25T19:19:33.296292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'10","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:18.685025+0000","last_change":"2024-04-25T19:15:18.428677+0000","last_active":"2024-04-25T19:19:18.685025+0000","last_peered":"2024-04-25T19:19:18.685025+0000","last_clean":"2024-04-25T19:19:18.685025+0000","last_became_active":"2024-04-25T19:15:18.428432+0000","last_became_peered":"2024-04-25T19:15:18.428432+0000","last_unstale":"2024-04-25T19:19:18.685025+0000","last_undegraded":"2024-04-25T19:19:18.685025+0000","last_fullsized":"2024-04-25T19:19:18.685025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'149","reported_seq":166,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.291381+0000","last_change":"2024-04-25T19:15:19.428245+0000","last_active":"2024-04-25T19:19:33.291381+0000","last_peered":"2024-04-25T19:19:33.291381+0000","last_clean":"2024-04-25T19:19:33.291381+0000","last_became_active":"2024-04-25T19:15:19.427941+0000","last_became_peered":"2024-04-25T19:15:19.427941+0000","last_unstale":"2024-04-25T19:19:33.291381+0000","last_undegraded":"2024-04-25T19:19:33.291381+0000","last_fullsized":"2024-04-25T19:19:33.291381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:09.947923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'102","reported_seq":121,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.281607+0000","last_change":"2024-04-25T19:15:18.420881+0000","last_active":"2024-04-25T19:19:33.281607+0000","last_peered":"2024-04-25T19:19:33.281607+0000","last_clean":"2024-04-25T19:19:33.281607+0000","last_became_active":"2024-04-25T19:15:18.420303+0000","last_became_peered":"2024-04-25T19:15:18.420303+0000","last_unstale":"2024-04-25T19:19:33.281607+0000","last_undegraded":"2024-04-25T19:19:33.281607+0000","last_fullsized":"2024-04-25T19:19:33.281607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:39:24.356298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":216794,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103,"num_write_kb":251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'150","reported_seq":162,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.293633+0000","last_change":"2024-04-25T19:16:17.661342+0000","last_active":"2024-04-25T19:19:33.293633+0000","last_peered":"2024-04-25T19:19:33.293633+0000","last_clean":"2024-04-25T19:19:33.293633+0000","last_became_active":"2024-04-25T19:16:17.661023+0000","last_became_peered":"2024-04-25T19:16:17.661023+0000","last_unstale":"2024-04-25T19:19:33.293633+0000","last_undegraded":"2024-04-25T19:19:33.293633+0000","last_fullsized":"2024-04-25T19:19:33.293633+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.13","version":"23'15","reported_seq":43,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:36.690896+0000","last_change":"2024-04-25T19:19:36.690953+0000","last_active":"2024-04-25T19:19:36.690896+0000","last_peered":"2024-04-25T19:19:36.690896+0000","last_clean":"2024-04-25T19:19:36.690896+0000","last_became_active":"2024-04-25T19:15:18.420402+0000","last_became_peered":"2024-04-25T19:15:18.420402+0000","last_unstale":"2024-04-25T19:19:36.690896+0000","last_undegraded":"2024-04-25T19:19:36.690896+0000","last_fullsized":"2024-04-25T19:19:36.690896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'15","last_scrub_stamp":"2024-04-25T19:19:36.690811+0000","last_deep_scrub":"23'15","last_deep_scrub_stamp":"2024-04-25T19:19:36.690811+0000","last_clean_scrub_stamp":"2024-04-25T19:19:36.690811+0000","objects_scrubbed":6,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:58:33.775049+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'157","reported_seq":175,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.309257+0000","last_change":"2024-04-25T19:15:19.424805+0000","last_active":"2024-04-25T19:19:33.309257+0000","last_peered":"2024-04-25T19:19:33.309257+0000","last_clean":"2024-04-25T19:19:33.309257+0000","last_became_active":"2024-04-25T19:15:19.424512+0000","last_became_peered":"2024-04-25T19:15:19.424512+0000","last_unstale":"2024-04-25T19:19:33.309257+0000","last_undegraded":"2024-04-25T19:19:33.309257+0000","last_fullsized":"2024-04-25T19:19:33.309257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.550627+0000","last_change":"2024-04-25T19:15:18.426805+0000","last_active":"2024-04-25T19:16:17.550627+0000","last_peered":"2024-04-25T19:16:17.550627+0000","last_clean":"2024-04-25T19:16:17.550627+0000","last_became_active":"2024-04-25T19:15:18.426472+0000","last_became_peered":"2024-04-25T19:15:18.426472+0000","last_unstale":"2024-04-25T19:16:17.550627+0000","last_undegraded":"2024-04-25T19:16:17.550627+0000","last_fullsized":"2024-04-25T19:16:17.550627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:29.373363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'171","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.294267+0000","last_change":"2024-04-25T19:15:19.428780+0000","last_active":"2024-04-25T19:19:33.294267+0000","last_peered":"2024-04-25T19:19:33.294267+0000","last_clean":"2024-04-25T19:19:33.294267+0000","last_became_active":"2024-04-25T19:15:19.428450+0000","last_became_peered":"2024-04-25T19:15:19.428450+0000","last_unstale":"2024-04-25T19:19:33.294267+0000","last_undegraded":"2024-04-25T19:19:33.294267+0000","last_fullsized":"2024-04-25T19:19:33.294267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:01.203880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"23'7","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:37.863425+0000","last_change":"2024-04-25T19:15:18.417740+0000","last_active":"2024-04-25T19:17:37.863425+0000","last_peered":"2024-04-25T19:17:37.863425+0000","last_clean":"2024-04-25T19:17:37.863425+0000","last_became_active":"2024-04-25T19:15:18.417404+0000","last_became_peered":"2024-04-25T19:15:18.417404+0000","last_unstale":"2024-04-25T19:17:37.863425+0000","last_undegraded":"2024-04-25T19:17:37.863425+0000","last_fullsized":"2024-04-25T19:17:37.863425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:21.737283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'165","reported_seq":182,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.300110+0000","last_change":"2024-04-25T19:15:19.411214+0000","last_active":"2024-04-25T19:19:33.300110+0000","last_peered":"2024-04-25T19:19:33.300110+0000","last_clean":"2024-04-25T19:19:33.300110+0000","last_became_active":"2024-04-25T19:15:19.410814+0000","last_became_peered":"2024-04-25T19:15:19.410814+0000","last_unstale":"2024-04-25T19:19:33.300110+0000","last_undegraded":"2024-04-25T19:19:33.300110+0000","last_fullsized":"2024-04-25T19:19:33.300110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:30:42.430277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'18","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:16.542401+0000","last_change":"2024-04-25T19:15:18.422226+0000","last_active":"2024-04-25T19:19:16.542401+0000","last_peered":"2024-04-25T19:19:16.542401+0000","last_clean":"2024-04-25T19:19:16.542401+0000","last_became_active":"2024-04-25T19:15:18.421900+0000","last_became_peered":"2024-04-25T19:15:18.421900+0000","last_unstale":"2024-04-25T19:19:16.542401+0000","last_undegraded":"2024-04-25T19:19:16.542401+0000","last_fullsized":"2024-04-25T19:19:16.542401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'189","reported_seq":206,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.288672+0000","last_change":"2024-04-25T19:15:19.428885+0000","last_active":"2024-04-25T19:19:33.288672+0000","last_peered":"2024-04-25T19:19:33.288672+0000","last_clean":"2024-04-25T19:19:33.288672+0000","last_became_active":"2024-04-25T19:15:19.428563+0000","last_became_peered":"2024-04-25T19:15:19.428563+0000","last_unstale":"2024-04-25T19:19:33.288672+0000","last_undegraded":"2024-04-25T19:19:33.288672+0000","last_fullsized":"2024-04-25T19:19:33.288672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:43.469189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'2","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.800374+0000","last_change":"2024-04-25T19:15:18.422125+0000","last_active":"2024-04-25T19:16:17.800374+0000","last_peered":"2024-04-25T19:16:17.800374+0000","last_clean":"2024-04-25T19:16:17.800374+0000","last_became_active":"2024-04-25T19:15:18.421777+0000","last_became_peered":"2024-04-25T19:15:18.421777+0000","last_unstale":"2024-04-25T19:16:17.800374+0000","last_undegraded":"2024-04-25T19:16:17.800374+0000","last_fullsized":"2024-04-25T19:16:17.800374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:24.289010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'146","reported_seq":163,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.290404+0000","last_change":"2024-04-25T19:15:19.429675+0000","last_active":"2024-04-25T19:19:33.290404+0000","last_peered":"2024-04-25T19:19:33.290404+0000","last_clean":"2024-04-25T19:19:33.290404+0000","last_became_active":"2024-04-25T19:15:19.429316+0000","last_became_peered":"2024-04-25T19:15:19.429316+0000","last_unstale":"2024-04-25T19:19:33.290404+0000","last_undegraded":"2024-04-25T19:19:33.290404+0000","last_fullsized":"2024-04-25T19:19:33.290404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:09:55.521944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'5","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:32.867322+0000","last_change":"2024-04-25T19:15:18.418661+0000","last_active":"2024-04-25T19:17:32.867322+0000","last_peered":"2024-04-25T19:17:32.867322+0000","last_clean":"2024-04-25T19:17:32.867322+0000","last_became_active":"2024-04-25T19:15:18.418359+0000","last_became_peered":"2024-04-25T19:15:18.418359+0000","last_unstale":"2024-04-25T19:17:32.867322+0000","last_undegraded":"2024-04-25T19:17:32.867322+0000","last_fullsized":"2024-04-25T19:17:32.867322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'138","reported_seq":155,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.295852+0000","last_change":"2024-04-25T19:15:19.434853+0000","last_active":"2024-04-25T19:19:33.295852+0000","last_peered":"2024-04-25T19:19:33.295852+0000","last_clean":"2024-04-25T19:19:33.295852+0000","last_became_active":"2024-04-25T19:15:19.434508+0000","last_became_peered":"2024-04-25T19:15:19.434508+0000","last_unstale":"2024-04-25T19:19:33.295852+0000","last_undegraded":"2024-04-25T19:19:33.295852+0000","last_fullsized":"2024-04-25T19:19:33.295852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:52.965864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'3","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:32.863302+0000","last_change":"2024-04-25T19:15:18.428626+0000","last_active":"2024-04-25T19:17:32.863302+0000","last_peered":"2024-04-25T19:17:32.863302+0000","last_clean":"2024-04-25T19:17:32.863302+0000","last_became_active":"2024-04-25T19:15:18.428332+0000","last_became_peered":"2024-04-25T19:15:18.428332+0000","last_unstale":"2024-04-25T19:17:32.863302+0000","last_undegraded":"2024-04-25T19:17:32.863302+0000","last_fullsized":"2024-04-25T19:17:32.863302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'144","reported_seq":161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.314727+0000","last_change":"2024-04-25T19:15:19.428754+0000","last_active":"2024-04-25T19:19:33.314727+0000","last_peered":"2024-04-25T19:19:33.314727+0000","last_clean":"2024-04-25T19:19:33.314727+0000","last_became_active":"2024-04-25T19:15:19.427992+0000","last_became_peered":"2024-04-25T19:15:19.427992+0000","last_unstale":"2024-04-25T19:19:33.314727+0000","last_undegraded":"2024-04-25T19:19:33.314727+0000","last_fullsized":"2024-04-25T19:19:33.314727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:43:48.443923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"23'7","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.258607+0000","last_change":"2024-04-25T19:15:18.419645+0000","last_active":"2024-04-25T19:19:33.258607+0000","last_peered":"2024-04-25T19:19:33.258607+0000","last_clean":"2024-04-25T19:19:33.258607+0000","last_became_active":"2024-04-25T19:15:18.419286+0000","last_became_peered":"2024-04-25T19:15:18.419286+0000","last_unstale":"2024-04-25T19:19:33.258607+0000","last_undegraded":"2024-04-25T19:19:33.258607+0000","last_fullsized":"2024-04-25T19:19:33.258607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1752,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'151","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.295289+0000","last_change":"2024-04-25T19:15:19.432395+0000","last_active":"2024-04-25T19:19:33.295289+0000","last_peered":"2024-04-25T19:19:33.295289+0000","last_clean":"2024-04-25T19:19:33.295289+0000","last_became_active":"2024-04-25T19:15:19.431739+0000","last_became_peered":"2024-04-25T19:15:19.431739+0000","last_unstale":"2024-04-25T19:19:33.295289+0000","last_undegraded":"2024-04-25T19:19:33.295289+0000","last_fullsized":"2024-04-25T19:19:33.295289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:50:03.964030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":116,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:17.871195+0000","last_change":"2024-04-25T19:15:15.604825+0000","last_active":"2024-04-25T19:17:17.871195+0000","last_peered":"2024-04-25T19:17:17.871195+0000","last_clean":"2024-04-25T19:17:17.871195+0000","last_became_active":"2024-04-25T19:15:15.603730+0000","last_became_peered":"2024-04-25T19:15:15.603730+0000","last_unstale":"2024-04-25T19:17:17.871195+0000","last_undegraded":"2024-04-25T19:17:17.871195+0000","last_fullsized":"2024-04-25T19:17:17.871195+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:14.367890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:14.367890+0000","last_clean_scrub_stamp":"2024-04-25T19:15:14.367890+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:02.411899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'6","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:22.863164+0000","last_change":"2024-04-25T19:15:18.426331+0000","last_active":"2024-04-25T19:17:22.863164+0000","last_peered":"2024-04-25T19:17:22.863164+0000","last_clean":"2024-04-25T19:17:22.863164+0000","last_became_active":"2024-04-25T19:15:18.425993+0000","last_became_peered":"2024-04-25T19:15:18.425993+0000","last_unstale":"2024-04-25T19:17:22.863164+0000","last_undegraded":"2024-04-25T19:17:22.863164+0000","last_fullsized":"2024-04-25T19:17:22.863164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:17.440484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'158","reported_seq":177,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.314922+0000","last_change":"2024-04-25T19:15:19.417337+0000","last_active":"2024-04-25T19:19:33.314922+0000","last_peered":"2024-04-25T19:19:33.314922+0000","last_clean":"2024-04-25T19:19:33.314922+0000","last_became_active":"2024-04-25T19:15:19.416808+0000","last_became_peered":"2024-04-25T19:15:19.416808+0000","last_unstale":"2024-04-25T19:19:33.314922+0000","last_undegraded":"2024-04-25T19:19:33.314922+0000","last_fullsized":"2024-04-25T19:19:33.314922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"21'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:17.871167+0000","last_change":"2024-04-25T19:15:18.426358+0000","last_active":"2024-04-25T19:17:17.871167+0000","last_peered":"2024-04-25T19:17:17.871167+0000","last_clean":"2024-04-25T19:17:17.871167+0000","last_became_active":"2024-04-25T19:15:18.426037+0000","last_became_peered":"2024-04-25T19:15:18.426037+0000","last_unstale":"2024-04-25T19:17:17.871167+0000","last_undegraded":"2024-04-25T19:17:17.871167+0000","last_fullsized":"2024-04-25T19:17:17.871167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:26:28.945667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'131","reported_seq":150,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.297072+0000","last_change":"2024-04-25T19:15:19.426219+0000","last_active":"2024-04-25T19:19:33.297072+0000","last_peered":"2024-04-25T19:19:33.297072+0000","last_clean":"2024-04-25T19:19:33.297072+0000","last_became_active":"2024-04-25T19:15:19.425626+0000","last_became_peered":"2024-04-25T19:15:19.425626+0000","last_unstale":"2024-04-25T19:19:33.297072+0000","last_undegraded":"2024-04-25T19:19:33.297072+0000","last_fullsized":"2024-04-25T19:19:33.297072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'12","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.257430+0000","last_change":"2024-04-25T19:15:18.424560+0000","last_active":"2024-04-25T19:19:33.257430+0000","last_peered":"2024-04-25T19:19:33.257430+0000","last_clean":"2024-04-25T19:19:33.257430+0000","last_became_active":"2024-04-25T19:15:18.424179+0000","last_became_peered":"2024-04-25T19:15:18.424179+0000","last_unstale":"2024-04-25T19:19:33.257430+0000","last_undegraded":"2024-04-25T19:19:33.257430+0000","last_fullsized":"2024-04-25T19:19:33.257430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'131","reported_seq":150,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.296087+0000","last_change":"2024-04-25T19:15:19.415219+0000","last_active":"2024-04-25T19:19:33.296087+0000","last_peered":"2024-04-25T19:19:33.296087+0000","last_clean":"2024-04-25T19:19:33.296087+0000","last_became_active":"2024-04-25T19:15:19.414877+0000","last_became_peered":"2024-04-25T19:15:19.414877+0000","last_unstale":"2024-04-25T19:19:33.296087+0000","last_undegraded":"2024-04-25T19:19:33.296087+0000","last_fullsized":"2024-04-25T19:19:33.296087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'5","reported_seq":25,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:22.863125+0000","last_change":"2024-04-25T19:15:18.423402+0000","last_active":"2024-04-25T19:17:22.863125+0000","last_peered":"2024-04-25T19:17:22.863125+0000","last_clean":"2024-04-25T19:17:22.863125+0000","last_became_active":"2024-04-25T19:15:18.422330+0000","last_became_peered":"2024-04-25T19:15:18.422330+0000","last_unstale":"2024-04-25T19:17:22.863125+0000","last_undegraded":"2024-04-25T19:17:22.863125+0000","last_fullsized":"2024-04-25T19:17:22.863125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:39:17.512502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'145","reported_seq":163,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.304465+0000","last_change":"2024-04-25T19:15:19.421096+0000","last_active":"2024-04-25T19:19:33.304465+0000","last_peered":"2024-04-25T19:19:33.304465+0000","last_clean":"2024-04-25T19:19:33.304465+0000","last_became_active":"2024-04-25T19:15:19.420598+0000","last_became_peered":"2024-04-25T19:15:19.420598+0000","last_unstale":"2024-04-25T19:19:33.304465+0000","last_undegraded":"2024-04-25T19:19:33.304465+0000","last_fullsized":"2024-04-25T19:19:33.304465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:20.733888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"21'1","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:27.876278+0000","last_change":"2024-04-25T19:15:18.422979+0000","last_active":"2024-04-25T19:19:27.876278+0000","last_peered":"2024-04-25T19:19:27.876278+0000","last_clean":"2024-04-25T19:19:27.876278+0000","last_became_active":"2024-04-25T19:15:18.422621+0000","last_became_peered":"2024-04-25T19:15:18.422621+0000","last_unstale":"2024-04-25T19:19:27.876278+0000","last_undegraded":"2024-04-25T19:19:27.876278+0000","last_fullsized":"2024-04-25T19:19:27.876278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'143","reported_seq":161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.315917+0000","last_change":"2024-04-25T19:15:19.426379+0000","last_active":"2024-04-25T19:19:33.315917+0000","last_peered":"2024-04-25T19:19:33.315917+0000","last_clean":"2024-04-25T19:19:33.315917+0000","last_became_active":"2024-04-25T19:15:19.425997+0000","last_became_peered":"2024-04-25T19:15:19.425997+0000","last_unstale":"2024-04-25T19:19:33.315917+0000","last_undegraded":"2024-04-25T19:19:33.315917+0000","last_fullsized":"2024-04-25T19:19:33.315917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:32.972274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"21'4","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:22.871729+0000","last_change":"2024-04-25T19:15:18.427748+0000","last_active":"2024-04-25T19:17:22.871729+0000","last_peered":"2024-04-25T19:17:22.871729+0000","last_clean":"2024-04-25T19:17:22.871729+0000","last_became_active":"2024-04-25T19:15:18.427374+0000","last_became_peered":"2024-04-25T19:15:18.427374+0000","last_unstale":"2024-04-25T19:17:22.871729+0000","last_undegraded":"2024-04-25T19:17:22.871729+0000","last_fullsized":"2024-04-25T19:17:22.871729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:47:41.469558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'144","reported_seq":163,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.292938+0000","last_change":"2024-04-25T19:15:19.422196+0000","last_active":"2024-04-25T19:19:33.292938+0000","last_peered":"2024-04-25T19:19:33.292938+0000","last_clean":"2024-04-25T19:19:33.292938+0000","last_became_active":"2024-04-25T19:15:19.421835+0000","last_became_peered":"2024-04-25T19:15:19.421835+0000","last_unstale":"2024-04-25T19:19:33.292938+0000","last_undegraded":"2024-04-25T19:19:33.292938+0000","last_fullsized":"2024-04-25T19:19:33.292938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'27","reported_seq":56,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:37.164797+0000","last_change":"2024-04-25T19:19:37.164797+0000","last_active":"2024-04-25T19:19:37.164797+0000","last_peered":"2024-04-25T19:19:37.164797+0000","last_clean":"2024-04-25T19:19:37.164797+0000","last_became_active":"2024-04-25T19:15:18.418775+0000","last_became_peered":"2024-04-25T19:15:18.418775+0000","last_unstale":"2024-04-25T19:19:37.164797+0000","last_undegraded":"2024-04-25T19:19:37.164797+0000","last_fullsized":"2024-04-25T19:19:37.164797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'27","last_scrub_stamp":"2024-04-25T19:19:37.164683+0000","last_deep_scrub":"23'27","last_deep_scrub_stamp":"2024-04-25T19:19:37.164683+0000","last_clean_scrub_stamp":"2024-04-25T19:19:37.164683+0000","objects_scrubbed":4,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:16.469492+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'143","reported_seq":161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.313614+0000","last_change":"2024-04-25T19:15:19.432155+0000","last_active":"2024-04-25T19:19:33.313614+0000","last_peered":"2024-04-25T19:19:33.313614+0000","last_clean":"2024-04-25T19:19:33.313614+0000","last_became_active":"2024-04-25T19:15:19.431905+0000","last_became_peered":"2024-04-25T19:15:19.431905+0000","last_unstale":"2024-04-25T19:19:33.313614+0000","last_undegraded":"2024-04-25T19:19:33.313614+0000","last_fullsized":"2024-04-25T19:19:33.313614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'8","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:16.546354+0000","last_change":"2024-04-25T19:15:18.423144+0000","last_active":"2024-04-25T19:19:16.546354+0000","last_peered":"2024-04-25T19:19:16.546354+0000","last_clean":"2024-04-25T19:19:16.546354+0000","last_became_active":"2024-04-25T19:15:18.422838+0000","last_became_peered":"2024-04-25T19:15:18.422838+0000","last_unstale":"2024-04-25T19:19:16.546354+0000","last_undegraded":"2024-04-25T19:19:16.546354+0000","last_fullsized":"2024-04-25T19:19:16.546354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:28.026912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'163","reported_seq":181,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.305454+0000","last_change":"2024-04-25T19:15:19.420582+0000","last_active":"2024-04-25T19:19:33.305454+0000","last_peered":"2024-04-25T19:19:33.305454+0000","last_clean":"2024-04-25T19:19:33.305454+0000","last_became_active":"2024-04-25T19:15:19.420270+0000","last_became_peered":"2024-04-25T19:15:19.420270+0000","last_unstale":"2024-04-25T19:19:33.305454+0000","last_undegraded":"2024-04-25T19:19:33.305454+0000","last_fullsized":"2024-04-25T19:19:33.305454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"21'2","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:27.872211+0000","last_change":"2024-04-25T19:15:18.425469+0000","last_active":"2024-04-25T19:17:27.872211+0000","last_peered":"2024-04-25T19:17:27.872211+0000","last_clean":"2024-04-25T19:17:27.872211+0000","last_became_active":"2024-04-25T19:15:18.425084+0000","last_became_peered":"2024-04-25T19:15:18.425084+0000","last_unstale":"2024-04-25T19:17:27.872211+0000","last_undegraded":"2024-04-25T19:17:27.872211+0000","last_fullsized":"2024-04-25T19:17:27.872211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:20:14.363158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'134","reported_seq":151,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.270401+0000","last_change":"2024-04-25T19:15:19.425159+0000","last_active":"2024-04-25T19:19:33.270401+0000","last_peered":"2024-04-25T19:19:33.270401+0000","last_clean":"2024-04-25T19:19:33.270401+0000","last_became_active":"2024-04-25T19:15:19.424847+0000","last_became_peered":"2024-04-25T19:15:19.424847+0000","last_unstale":"2024-04-25T19:19:33.270401+0000","last_undegraded":"2024-04-25T19:19:33.270401+0000","last_fullsized":"2024-04-25T19:19:33.270401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:28.715286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"21'7","reported_seq":28,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:36.904879+0000","last_change":"2024-04-25T19:15:18.424645+0000","last_active":"2024-04-25T19:19:36.904879+0000","last_peered":"2024-04-25T19:19:36.904879+0000","last_clean":"2024-04-25T19:19:36.904879+0000","last_became_active":"2024-04-25T19:15:18.424241+0000","last_became_peered":"2024-04-25T19:15:18.424241+0000","last_unstale":"2024-04-25T19:19:36.904879+0000","last_undegraded":"2024-04-25T19:19:36.904879+0000","last_fullsized":"2024-04-25T19:19:36.904879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'153","reported_seq":171,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.306190+0000","last_change":"2024-04-25T19:15:19.421095+0000","last_active":"2024-04-25T19:19:33.306190+0000","last_peered":"2024-04-25T19:19:33.306190+0000","last_clean":"2024-04-25T19:19:33.306190+0000","last_became_active":"2024-04-25T19:15:19.420795+0000","last_became_peered":"2024-04-25T19:15:19.420795+0000","last_unstale":"2024-04-25T19:19:33.306190+0000","last_undegraded":"2024-04-25T19:19:33.306190+0000","last_fullsized":"2024-04-25T19:19:33.306190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'3","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:36.904894+0000","last_change":"2024-04-25T19:15:18.423841+0000","last_active":"2024-04-25T19:19:36.904894+0000","last_peered":"2024-04-25T19:19:36.904894+0000","last_clean":"2024-04-25T19:19:36.904894+0000","last_became_active":"2024-04-25T19:15:18.423500+0000","last_became_peered":"2024-04-25T19:15:18.423500+0000","last_unstale":"2024-04-25T19:19:36.904894+0000","last_undegraded":"2024-04-25T19:19:36.904894+0000","last_fullsized":"2024-04-25T19:19:36.904894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'143","reported_seq":161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.302562+0000","last_change":"2024-04-25T19:15:19.416232+0000","last_active":"2024-04-25T19:19:33.302562+0000","last_peered":"2024-04-25T19:19:33.302562+0000","last_clean":"2024-04-25T19:19:33.302562+0000","last_became_active":"2024-04-25T19:15:19.415828+0000","last_became_peered":"2024-04-25T19:15:19.415828+0000","last_unstale":"2024-04-25T19:19:33.302562+0000","last_undegraded":"2024-04-25T19:19:33.302562+0000","last_fullsized":"2024-04-25T19:19:33.302562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:32.872674+0000","last_change":"2024-04-25T19:15:18.423918+0000","last_active":"2024-04-25T19:17:32.872674+0000","last_peered":"2024-04-25T19:17:32.872674+0000","last_clean":"2024-04-25T19:17:32.872674+0000","last_became_active":"2024-04-25T19:15:18.423588+0000","last_became_peered":"2024-04-25T19:15:18.423588+0000","last_unstale":"2024-04-25T19:17:32.872674+0000","last_undegraded":"2024-04-25T19:17:32.872674+0000","last_fullsized":"2024-04-25T19:17:32.872674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:16.378285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'128","reported_seq":145,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.499523+0000","last_change":"2024-04-25T19:15:19.422984+0000","last_active":"2024-04-25T19:19:33.499523+0000","last_peered":"2024-04-25T19:19:33.499523+0000","last_clean":"2024-04-25T19:19:33.499523+0000","last_became_active":"2024-04-25T19:15:19.422651+0000","last_became_peered":"2024-04-25T19:15:19.422651+0000","last_unstale":"2024-04-25T19:19:33.499523+0000","last_undegraded":"2024-04-25T19:19:33.499523+0000","last_fullsized":"2024-04-25T19:19:33.499523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:54.984914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'183","reported_seq":200,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.317530+0000","last_change":"2024-04-25T19:15:19.423982+0000","last_active":"2024-04-25T19:19:33.317530+0000","last_peered":"2024-04-25T19:19:33.317530+0000","last_clean":"2024-04-25T19:19:33.317530+0000","last_became_active":"2024-04-25T19:15:19.423341+0000","last_became_peered":"2024-04-25T19:15:19.423341+0000","last_unstale":"2024-04-25T19:19:33.317530+0000","last_undegraded":"2024-04-25T19:19:33.317530+0000","last_fullsized":"2024-04-25T19:19:33.317530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:51.808352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'5","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:16:17.549697+0000","last_change":"2024-04-25T19:15:18.424455+0000","last_active":"2024-04-25T19:16:17.549697+0000","last_peered":"2024-04-25T19:16:17.549697+0000","last_clean":"2024-04-25T19:16:17.549697+0000","last_became_active":"2024-04-25T19:15:18.424119+0000","last_became_peered":"2024-04-25T19:15:18.424119+0000","last_unstale":"2024-04-25T19:16:17.549697+0000","last_undegraded":"2024-04-25T19:16:17.549697+0000","last_fullsized":"2024-04-25T19:16:17.549697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:43.135567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'141","reported_seq":158,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.315295+0000","last_change":"2024-04-25T19:15:19.424064+0000","last_active":"2024-04-25T19:19:33.315295+0000","last_peered":"2024-04-25T19:19:33.315295+0000","last_clean":"2024-04-25T19:19:33.315295+0000","last_became_active":"2024-04-25T19:15:19.423187+0000","last_became_peered":"2024-04-25T19:15:19.423187+0000","last_unstale":"2024-04-25T19:19:33.315295+0000","last_undegraded":"2024-04-25T19:19:33.315295+0000","last_fullsized":"2024-04-25T19:19:33.315295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:43:31.590700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"23'13","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:17:47.874568+0000","last_change":"2024-04-25T19:15:18.423151+0000","last_active":"2024-04-25T19:17:47.874568+0000","last_peered":"2024-04-25T19:17:47.874568+0000","last_clean":"2024-04-25T19:17:47.874568+0000","last_became_active":"2024-04-25T19:15:18.422831+0000","last_became_peered":"2024-04-25T19:15:18.422831+0000","last_unstale":"2024-04-25T19:17:47.874568+0000","last_undegraded":"2024-04-25T19:17:47.874568+0000","last_fullsized":"2024-04-25T19:17:47.874568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:43.244963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'135","reported_seq":154,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.296276+0000","last_change":"2024-04-25T19:15:19.416717+0000","last_active":"2024-04-25T19:19:33.296276+0000","last_peered":"2024-04-25T19:19:33.296276+0000","last_clean":"2024-04-25T19:19:33.296276+0000","last_became_active":"2024-04-25T19:15:19.416163+0000","last_became_peered":"2024-04-25T19:15:19.416163+0000","last_unstale":"2024-04-25T19:19:33.296276+0000","last_undegraded":"2024-04-25T19:19:33.296276+0000","last_fullsized":"2024-04-25T19:19:33.296276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:17.876204+0000","last_change":"2024-04-25T19:15:18.422875+0000","last_active":"2024-04-25T19:19:17.876204+0000","last_peered":"2024-04-25T19:19:17.876204+0000","last_clean":"2024-04-25T19:19:17.876204+0000","last_became_active":"2024-04-25T19:15:18.422489+0000","last_became_peered":"2024-04-25T19:15:18.422489+0000","last_unstale":"2024-04-25T19:19:17.876204+0000","last_undegraded":"2024-04-25T19:19:17.876204+0000","last_fullsized":"2024-04-25T19:19:17.876204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:17.384331+0000","last_clean_scrub_stamp":"2024-04-25T19:15:17.384331+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'152","reported_seq":171,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:33.304532+0000","last_change":"2024-04-25T19:15:19.416987+0000","last_active":"2024-04-25T19:19:33.304532+0000","last_peered":"2024-04-25T19:19:33.304532+0000","last_clean":"2024-04-25T19:19:33.304532+0000","last_became_active":"2024-04-25T19:15:19.414173+0000","last_became_peered":"2024-04-25T19:15:19.414173+0000","last_unstale":"2024-04-25T19:19:33.304532+0000","last_undegraded":"2024-04-25T19:19:33.304532+0000","last_fullsized":"2024-04-25T19:19:33.304532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T19:15:18.390824+0000","last_clean_scrub_stamp":"2024-04-25T19:15:18.390824+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"23'19","reported_seq":48,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:37.724482+0000","last_change":"2024-04-25T19:19:37.724514+0000","last_active":"2024-04-25T19:19:37.724482+0000","last_peered":"2024-04-25T19:19:37.724482+0000","last_clean":"2024-04-25T19:19:37.724482+0000","last_became_active":"2024-04-25T19:15:18.423532+0000","last_became_peered":"2024-04-25T19:15:18.423532+0000","last_unstale":"2024-04-25T19:19:37.724482+0000","last_undegraded":"2024-04-25T19:19:37.724482+0000","last_fullsized":"2024-04-25T19:19:37.724482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'19","last_scrub_stamp":"2024-04-25T19:19:37.724409+0000","last_deep_scrub":"23'19","last_deep_scrub_stamp":"2024-04-25T19:19:37.724409+0000","last_clean_scrub_stamp":"2024-04-25T19:19:37.724409+0000","objects_scrubbed":6,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:35.236962+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9717,"ondisk_log_size":9717,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":225467,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":353,"num_write_kb":342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":696320,"data_stored":450934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":330,"ondisk_log_size":330,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64838,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574904,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39832,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332008,"statfs":{"total":96636764160,"available":96595976192,"internally_reserved":0,"allocated":344064,"data_stored":680845,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35808,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336032,"statfs":{"total":96636764160,"available":96600096768,"internally_reserved":0,"allocated":483328,"data_stored":305961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574904,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39780,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332060,"statfs":{"total":96636764160,"available":96596029440,"internally_reserved":0,"allocated":290816,"data_stored":92309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39756,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332084,"statfs":{"total":96636764160,"available":96596054016,"internally_reserved":0,"allocated":266240,"data_stored":89707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574904,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39976,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331864,"statfs":{"total":96636764160,"available":96595828736,"internally_reserved":0,"allocated":491520,"data_stored":306500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35608,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336232,"statfs":{"total":96636764160,"available":96600301568,"internally_reserved":0,"allocated":278528,"data_stored":90344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35600,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336240,"statfs":{"total":96636764160,"available":96600309760,"internally_reserved":0,"allocated":270336,"data_stored":89327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39824,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332016,"statfs":{"total":96636764160,"available":96595984384,"internally_reserved":0,"allocated":335872,"data_stored":679037,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":249856,"data_stored":218705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":218166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T19:19:40.025 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.026 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.026 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.026 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.026 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.026 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.026 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.026 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.026 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.027 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.027 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.027 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.027 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.027 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.027 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.027 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.027 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.027 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.027 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.028 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.028 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.028 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.028 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.028 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.028 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.028 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.028 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.028 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.029 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.029 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.029 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.029 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.029 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.029 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.029 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.029 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.029 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.029 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.030 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.030 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.030 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.030 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.030 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.030 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.030 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.030 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.030 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.031 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.031 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.031 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.031 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.031 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.031 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.031 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.031 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.032 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.032 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.032 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.032 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.032 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.032 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.032 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.032 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.032 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T19:15:14.367890+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=14, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.032 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.033 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.033 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.033 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.033 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.033 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.033 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.033 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.033 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.033 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.034 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.034 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.034 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.034 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.034 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.034 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.034 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.034 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.034 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.035 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.035 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.035 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.035 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.035 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.035 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.035 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.035 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.035 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.036 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-04-25T19:15:17.384331+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=17, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.036 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-25T19:15:18.390824+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=15, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=19, tm_min=19, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T19:19:40.036 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T19:19:57.043 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:19:57.046 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:20:00.037 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T19:20:00.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.156+0000 7f8aa1f85640 1 Processor -- start 2024-04-25T19:20:00.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.156+0000 7f8aa1f85640 1 -- start start 2024-04-25T19:20:00.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.158+0000 7f8aa1f85640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8a9c0c7750 0x7f8a9c153590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:00.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.158+0000 7f8aa1f85640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8a9c0c7b60 0x7f8a9c0cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:00.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.158+0000 7f8aa1f85640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a9c0c8020 con 0x7f8a9c0ce2a0 2024-04-25T19:20:00.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.158+0000 7f8aa1f85640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f8a9c1598a0 con 0x7f8a9c0c7750 2024-04-25T19:20:00.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.158+0000 7f8aa1f85640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f8a9c159b40 con 0x7f8a9c0c7b60 2024-04-25T19:20:00.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.158+0000 7f8a9b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8a9c0c7750 0x7f8a9c153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:00.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.158+0000 7f8a9b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8a9c0c7750 0x7f8a9c153590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58130/0 (socket says 172.21.15.76:58130) 2024-04-25T19:20:00.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.158+0000 7f8a9b7fe640 1 -- 172.21.15.76:0/3284981492 learned_addr learned my addr 172.21.15.76:0/3284981492 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:20:00.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.158+0000 7f8a93fff640 1 --2- 172.21.15.76:0/3284981492 >> v2:172.21.15.105:3300/0 conn(0x7f8a9c0c7b60 0x7f8a9c0cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:00.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.158+0000 7f8a9b7fe640 1 -- 172.21.15.76:0/3284981492 >> v2:172.21.15.105:3300/0 conn(0x7f8a9c0c7b60 msgr2=0x7f8a9c0cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:00.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.158+0000 7f8a9b7fe640 1 --2- 172.21.15.76:0/3284981492 >> v2:172.21.15.105:3300/0 conn(0x7f8a9c0c7b60 0x7f8a9c0cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:00.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.158+0000 7f8a9b7fe640 1 -- 172.21.15.76:0/3284981492 >> v1:172.21.15.105:6789/0 conn(0x7f8a9c0ce2a0 legacy=0x7f8a9c15cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:00.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.158+0000 7f8a9b7fe640 1 -- 172.21.15.76:0/3284981492 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a9c159e20 con 0x7f8a9c0c7750 2024-04-25T19:20:00.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.159+0000 7f8a9b7fe640 1 --2- 172.21.15.76:0/3284981492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a9c0c7750 0x7f8a9c153590 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c073d10 tx=0x7f8a8c02c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337d81d99bba8d7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:00.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.159+0000 7f8a9affd640 1 -- 172.21.15.76:0/3284981492 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c036020 con 0x7f8a9c0c7750 2024-04-25T19:20:00.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.159+0000 7f8a9affd640 1 -- 172.21.15.76:0/3284981492 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a8c032640 con 0x7f8a9c0c7750 2024-04-25T19:20:00.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.159+0000 7f8a9affd640 1 -- 172.21.15.76:0/3284981492 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c032a10 con 0x7f8a9c0c7750 2024-04-25T19:20:00.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.160+0000 7f8aa1f85640 1 -- 172.21.15.76:0/3284981492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a9c0c7750 msgr2=0x7f8a9c153590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:00.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.160+0000 7f8aa1f85640 1 --2- 172.21.15.76:0/3284981492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a9c0c7750 0x7f8a9c153590 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c073d10 tx=0x7f8a8c02c940 comp rx=0 tx=0).stop 2024-04-25T19:20:00.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.160+0000 7f8aa1f85640 1 -- 172.21.15.76:0/3284981492 shutdown_connections 2024-04-25T19:20:00.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.160+0000 7f8aa1f85640 1 --2- 172.21.15.76:0/3284981492 >> v2:172.21.15.105:3300/0 conn(0x7f8a9c0c7b60 0x7f8a9c0cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:00.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.160+0000 7f8aa1f85640 1 --2- 172.21.15.76:0/3284981492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a9c0c7750 0x7f8a9c153590 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:00.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.160+0000 7f8aa1f85640 1 -- 172.21.15.76:0/3284981492 >> 172.21.15.76:0/3284981492 conn(0x7f8a9c070800 msgr2=0x7f8a9c070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:00.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.160+0000 7f8aa1f85640 1 -- 172.21.15.76:0/3284981492 shutdown_connections 2024-04-25T19:20:00.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.160+0000 7f8aa1f85640 1 -- 172.21.15.76:0/3284981492 wait complete. 2024-04-25T19:20:00.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.162+0000 7f8aa1f85640 1 Processor -- start 2024-04-25T19:20:00.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.162+0000 7f8aa1f85640 1 -- start start 2024-04-25T19:20:00.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.163+0000 7f8aa1f85640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a9c0c7750 0x7f8a9c1f6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:00.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.163+0000 7f8aa1f85640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a9c0ce2a0 0x7f8a9c0bf290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:00.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.163+0000 7f8aa1f85640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8a9c0be7a0 0x7f8a9c0bf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:00.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.163+0000 7f8aa1f85640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9c153ad0 con 0x7f8a9c0ce2a0 2024-04-25T19:20:00.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.163+0000 7f8aa1f85640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9c15a3e0 con 0x7f8a9c0c7750 2024-04-25T19:20:00.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.163+0000 7f8aa1f85640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f8a9c0c8520 con 0x7f8a9c0be7a0 2024-04-25T19:20:00.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.163+0000 7f8a93fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a9c0ce2a0 0x7f8a9c0bf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:00.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.163+0000 7f8a93fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a9c0ce2a0 0x7f8a9c0bf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58140/0 (socket says 172.21.15.76:58140) 2024-04-25T19:20:00.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.163+0000 7f8a93fff640 1 -- 172.21.15.76:0/3398406307 learned_addr learned my addr 172.21.15.76:0/3398406307 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:20:00.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.163+0000 7f8a9b7fe640 1 --2- 172.21.15.76:0/3398406307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a9c0c7750 0x7f8a9c1f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:00.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.163+0000 7f8a9bfff640 1 --2- 172.21.15.76:0/3398406307 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8a9c0be7a0 0x7f8a9c0bf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:00.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.163+0000 7f8a93fff640 1 -- 172.21.15.76:0/3398406307 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8a9c0be7a0 msgr2=0x7f8a9c0bf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:00.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.163+0000 7f8a93fff640 1 --2- 172.21.15.76:0/3398406307 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8a9c0be7a0 0x7f8a9c0bf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:00.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.164+0000 7f8a93fff640 1 -- 172.21.15.76:0/3398406307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a9c0c7750 msgr2=0x7f8a9c1f6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:00.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.164+0000 7f8a93fff640 1 --2- 172.21.15.76:0/3398406307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a9c0c7750 0x7f8a9c1f6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:00.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.164+0000 7f8a93fff640 1 -- 172.21.15.76:0/3398406307 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a9c159630 con 0x7f8a9c0ce2a0 2024-04-25T19:20:00.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.164+0000 7f8a9bfff640 1 --2- 172.21.15.76:0/3398406307 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8a9c0be7a0 0x7f8a9c0bf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:20:00.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.164+0000 7f8a93fff640 1 --2- 172.21.15.76:0/3398406307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a9c0ce2a0 0x7f8a9c0bf290 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400e870 tx=0x7f8a8400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:00.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.164+0000 7f8a98ff9640 1 -- 172.21.15.76:0/3398406307 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a84017070 con 0x7f8a9c0ce2a0 2024-04-25T19:20:00.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.164+0000 7f8a98ff9640 1 -- 172.21.15.76:0/3398406307 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a8400c8e0 con 0x7f8a9c0ce2a0 2024-04-25T19:20:00.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.164+0000 7f8a98ff9640 1 -- 172.21.15.76:0/3398406307 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8400ccb0 con 0x7f8a9c0ce2a0 2024-04-25T19:20:00.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.164+0000 7f8aa1f85640 1 -- 172.21.15.76:0/3398406307 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a9c1f6b70 con 0x7f8a9c0ce2a0 2024-04-25T19:20:00.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.164+0000 7f8aa1f85640 1 -- 172.21.15.76:0/3398406307 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a9c159350 con 0x7f8a9c0ce2a0 2024-04-25T19:20:00.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.165+0000 7f8aa1f85640 1 -- 172.21.15.76:0/3398406307 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a9c06a390 con 0x7f8a9c0ce2a0 2024-04-25T19:20:00.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.166+0000 7f8a98ff9640 1 -- 172.21.15.76:0/3398406307 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8a8401a070 con 0x7f8a9c0ce2a0 2024-04-25T19:20:00.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.166+0000 7f8a98ff9640 1 --2- 172.21.15.76:0/3398406307 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8a7c0744b0 0x7f8a7c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:00.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.166+0000 7f8a98ff9640 1 -- 172.21.15.76:0/3398406307 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f8a84095460 con 0x7f8a9c0ce2a0 2024-04-25T19:20:00.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.166+0000 7f8a9b7fe640 1 --2- 172.21.15.76:0/3398406307 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8a7c0744b0 0x7f8a7c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:00.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.167+0000 7f8a9b7fe640 1 --2- 172.21.15.76:0/3398406307 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8a7c0744b0 0x7f8a7c0769c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c037000 tx=0x7f8a8c032000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:00.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.171+0000 7f8a98ff9640 1 -- 172.21.15.76:0/3398406307 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8a8405f520 con 0x7f8a9c0ce2a0 2024-04-25T19:20:00.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.344+0000 7f8aa1f85640 1 -- 172.21.15.76:0/3398406307 --> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8a9c158e20 con 0x7f8a7c0744b0 2024-04-25T19:20:00.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.350+0000 7f8a98ff9640 1 -- 172.21.15.76:0/3398406307 <== mgr.4109 v2:172.21.15.105:6832/2585775531 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+253535 (secure 0 0 0) 0x7f8a9c158e20 con 0x7f8a7c0744b0 2024-04-25T19:20:00.354 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:20:00.356 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-04-25T19:20:00.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.353+0000 7f8aa1f85640 1 -- 172.21.15.76:0/3398406307 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8a7c0744b0 msgr2=0x7f8a7c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:00.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.353+0000 7f8aa1f85640 1 --2- 172.21.15.76:0/3398406307 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8a7c0744b0 0x7f8a7c0769c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c037000 tx=0x7f8a8c032000 comp rx=0 tx=0).stop 2024-04-25T19:20:00.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.353+0000 7f8aa1f85640 1 -- 172.21.15.76:0/3398406307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a9c0ce2a0 msgr2=0x7f8a9c0bf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:00.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.353+0000 7f8aa1f85640 1 --2- 172.21.15.76:0/3398406307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a9c0ce2a0 0x7f8a9c0bf290 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400e870 tx=0x7f8a8400ed30 comp rx=0 tx=0).stop 2024-04-25T19:20:00.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.353+0000 7f8aa1f85640 1 -- 172.21.15.76:0/3398406307 shutdown_connections 2024-04-25T19:20:00.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.353+0000 7f8aa1f85640 1 --2- 172.21.15.76:0/3398406307 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f8a7c0744b0 0x7f8a7c0769c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:00.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.353+0000 7f8aa1f85640 1 --2- 172.21.15.76:0/3398406307 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f8a9c0be7a0 0x7f8a9c0bf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:00.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.353+0000 7f8aa1f85640 1 --2- 172.21.15.76:0/3398406307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a9c0ce2a0 0x7f8a9c0bf290 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:00.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.353+0000 7f8aa1f85640 1 --2- 172.21.15.76:0/3398406307 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8a9c0c7750 0x7f8a9c1f6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:00.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.353+0000 7f8aa1f85640 1 -- 172.21.15.76:0/3398406307 >> 172.21.15.76:0/3398406307 conn(0x7f8a9c070800 msgr2=0x7f8a9c0729f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:00.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.353+0000 7f8aa1f85640 1 -- 172.21.15.76:0/3398406307 shutdown_connections 2024-04-25T19:20:00.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.353+0000 7f8aa1f85640 1 -- 172.21.15.76:0/3398406307 wait complete. 2024-04-25T19:20:00.366 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-04-25T19:19:59.904284+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":815835,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":70,"num_read_kb":59,"num_write":9670,"num_write_kb":25167794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25000,"num_omap_keys":50,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10194,"ondisk_log_size":10194,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":306184,"kb_used_data":2696,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754668536,"statfs":{"total":773094113280,"available":772780580864,"internally_reserved":0,"allocated":2760704,"data_stored":2334030,"data_compressed":64838,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":310693424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003628"},"pg_stats":[{"pgid":"3.3f","version":"23'146","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:54.911349+0000","last_change":"2024-04-25T19:19:54.911399+0000","last_active":"2024-04-25T19:19:54.911349+0000","last_peered":"2024-04-25T19:19:54.911349+0000","last_clean":"2024-04-25T19:19:54.911349+0000","last_became_active":"2024-04-25T19:15:19.431332+0000","last_became_peered":"2024-04-25T19:15:19.431332+0000","last_unstale":"2024-04-25T19:19:54.911349+0000","last_undegraded":"2024-04-25T19:19:54.911349+0000","last_fullsized":"2024-04-25T19:19:54.911349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'146","last_scrub_stamp":"2024-04-25T19:19:54.911241+0000","last_deep_scrub":"23'146","last_deep_scrub_stamp":"2024-04-25T19:19:54.911241+0000","last_clean_scrub_stamp":"2024-04-25T19:19:54.911241+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:28:09.001293+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"23'130","reported_seq":157,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:44.197619+0000","last_change":"2024-04-25T19:19:44.197650+0000","last_active":"2024-04-25T19:19:44.197619+0000","last_peered":"2024-04-25T19:19:44.197619+0000","last_clean":"2024-04-25T19:19:44.197619+0000","last_became_active":"2024-04-25T19:15:19.418235+0000","last_became_peered":"2024-04-25T19:15:19.418235+0000","last_unstale":"2024-04-25T19:19:44.197619+0000","last_undegraded":"2024-04-25T19:19:44.197619+0000","last_fullsized":"2024-04-25T19:19:44.197619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'130","last_scrub_stamp":"2024-04-25T19:19:44.197520+0000","last_deep_scrub":"23'130","last_deep_scrub_stamp":"2024-04-25T19:19:44.197520+0000","last_clean_scrub_stamp":"2024-04-25T19:19:44.197520+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:31.152406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"23'151","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:41.741166+0000","last_change":"2024-04-25T19:19:41.741196+0000","last_active":"2024-04-25T19:19:41.741166+0000","last_peered":"2024-04-25T19:19:41.741166+0000","last_clean":"2024-04-25T19:19:41.741166+0000","last_became_active":"2024-04-25T19:15:19.416443+0000","last_became_peered":"2024-04-25T19:15:19.416443+0000","last_unstale":"2024-04-25T19:19:41.741166+0000","last_undegraded":"2024-04-25T19:19:41.741166+0000","last_fullsized":"2024-04-25T19:19:41.741166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'151","last_scrub_stamp":"2024-04-25T19:19:41.741059+0000","last_deep_scrub":"23'151","last_deep_scrub_stamp":"2024-04-25T19:19:41.741059+0000","last_clean_scrub_stamp":"2024-04-25T19:19:41.741059+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:41.621614+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"23'173","reported_seq":200,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:51.842410+0000","last_change":"2024-04-25T19:19:51.842463+0000","last_active":"2024-04-25T19:19:51.842410+0000","last_peered":"2024-04-25T19:19:51.842410+0000","last_clean":"2024-04-25T19:19:51.842410+0000","last_became_active":"2024-04-25T19:15:19.428419+0000","last_became_peered":"2024-04-25T19:15:19.428419+0000","last_unstale":"2024-04-25T19:19:51.842410+0000","last_undegraded":"2024-04-25T19:19:51.842410+0000","last_fullsized":"2024-04-25T19:19:51.842410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'173","last_scrub_stamp":"2024-04-25T19:19:51.842223+0000","last_deep_scrub":"23'173","last_deep_scrub_stamp":"2024-04-25T19:19:51.842223+0000","last_clean_scrub_stamp":"2024-04-25T19:19:51.842223+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:33:36.448907+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"23'148","reported_seq":174,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:48.000696+0000","last_change":"2024-04-25T19:19:48.000759+0000","last_active":"2024-04-25T19:19:48.000696+0000","last_peered":"2024-04-25T19:19:48.000696+0000","last_clean":"2024-04-25T19:19:48.000696+0000","last_became_active":"2024-04-25T19:15:19.426968+0000","last_became_peered":"2024-04-25T19:15:19.426968+0000","last_unstale":"2024-04-25T19:19:48.000696+0000","last_undegraded":"2024-04-25T19:19:48.000696+0000","last_fullsized":"2024-04-25T19:19:48.000696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'148","last_scrub_stamp":"2024-04-25T19:19:48.000559+0000","last_deep_scrub":"23'148","last_deep_scrub_stamp":"2024-04-25T19:19:48.000559+0000","last_clean_scrub_stamp":"2024-04-25T19:19:48.000559+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:10.132104+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"23'135","reported_seq":161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:44.713199+0000","last_change":"2024-04-25T19:19:44.713231+0000","last_active":"2024-04-25T19:19:44.713199+0000","last_peered":"2024-04-25T19:19:44.713199+0000","last_clean":"2024-04-25T19:19:44.713199+0000","last_became_active":"2024-04-25T19:15:19.417292+0000","last_became_peered":"2024-04-25T19:15:19.417292+0000","last_unstale":"2024-04-25T19:19:44.713199+0000","last_undegraded":"2024-04-25T19:19:44.713199+0000","last_fullsized":"2024-04-25T19:19:44.713199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'135","last_scrub_stamp":"2024-04-25T19:19:44.713099+0000","last_deep_scrub":"23'135","last_deep_scrub_stamp":"2024-04-25T19:19:44.713099+0000","last_clean_scrub_stamp":"2024-04-25T19:19:44.713099+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:23.088576+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"23'164","reported_seq":190,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:39.977701+0000","last_change":"2024-04-25T19:19:39.977733+0000","last_active":"2024-04-25T19:19:39.977701+0000","last_peered":"2024-04-25T19:19:39.977701+0000","last_clean":"2024-04-25T19:19:39.977701+0000","last_became_active":"2024-04-25T19:15:19.427852+0000","last_became_peered":"2024-04-25T19:15:19.427852+0000","last_unstale":"2024-04-25T19:19:39.977701+0000","last_undegraded":"2024-04-25T19:19:39.977701+0000","last_fullsized":"2024-04-25T19:19:39.977701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'164","last_scrub_stamp":"2024-04-25T19:19:39.977597+0000","last_deep_scrub":"23'164","last_deep_scrub_stamp":"2024-04-25T19:19:39.977597+0000","last_clean_scrub_stamp":"2024-04-25T19:19:39.977597+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:24.444747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"23'186","reported_seq":212,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:44.985155+0000","last_change":"2024-04-25T19:19:44.985182+0000","last_active":"2024-04-25T19:19:44.985155+0000","last_peered":"2024-04-25T19:19:44.985155+0000","last_clean":"2024-04-25T19:19:44.985155+0000","last_became_active":"2024-04-25T19:15:19.417183+0000","last_became_peered":"2024-04-25T19:15:19.417183+0000","last_unstale":"2024-04-25T19:19:44.985155+0000","last_undegraded":"2024-04-25T19:19:44.985155+0000","last_fullsized":"2024-04-25T19:19:44.985155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'186","last_scrub_stamp":"2024-04-25T19:19:44.985063+0000","last_deep_scrub":"23'186","last_deep_scrub_stamp":"2024-04-25T19:19:44.985063+0000","last_clean_scrub_stamp":"2024-04-25T19:19:44.985063+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:10.807727+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"23'152","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:52.930156+0000","last_change":"2024-04-25T19:19:52.930185+0000","last_active":"2024-04-25T19:19:52.930156+0000","last_peered":"2024-04-25T19:19:52.930156+0000","last_clean":"2024-04-25T19:19:52.930156+0000","last_became_active":"2024-04-25T19:15:19.431240+0000","last_became_peered":"2024-04-25T19:15:19.431240+0000","last_unstale":"2024-04-25T19:19:52.930156+0000","last_undegraded":"2024-04-25T19:19:52.930156+0000","last_fullsized":"2024-04-25T19:19:52.930156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'152","last_scrub_stamp":"2024-04-25T19:19:52.930060+0000","last_deep_scrub":"23'152","last_deep_scrub_stamp":"2024-04-25T19:19:52.930060+0000","last_clean_scrub_stamp":"2024-04-25T19:19:52.930060+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:45.376619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"23'156","reported_seq":187,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:50.422171+0000","last_change":"2024-04-25T19:19:50.422222+0000","last_active":"2024-04-25T19:19:50.422171+0000","last_peered":"2024-04-25T19:19:50.422171+0000","last_clean":"2024-04-25T19:19:50.422171+0000","last_became_active":"2024-04-25T19:16:17.554771+0000","last_became_peered":"2024-04-25T19:16:17.554771+0000","last_unstale":"2024-04-25T19:19:50.422171+0000","last_undegraded":"2024-04-25T19:19:50.422171+0000","last_fullsized":"2024-04-25T19:19:50.422171+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'156","last_scrub_stamp":"2024-04-25T19:19:50.422038+0000","last_deep_scrub":"23'156","last_deep_scrub_stamp":"2024-04-25T19:19:50.422038+0000","last_clean_scrub_stamp":"2024-04-25T19:19:50.422038+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:16.087699+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"23'189","reported_seq":215,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:46.407831+0000","last_change":"2024-04-25T19:19:46.407860+0000","last_active":"2024-04-25T19:19:46.407831+0000","last_peered":"2024-04-25T19:19:46.407831+0000","last_clean":"2024-04-25T19:19:46.407831+0000","last_became_active":"2024-04-25T19:15:19.417230+0000","last_became_peered":"2024-04-25T19:15:19.417230+0000","last_unstale":"2024-04-25T19:19:46.407831+0000","last_undegraded":"2024-04-25T19:19:46.407831+0000","last_fullsized":"2024-04-25T19:19:46.407831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'189","last_scrub_stamp":"2024-04-25T19:19:46.407737+0000","last_deep_scrub":"23'189","last_deep_scrub_stamp":"2024-04-25T19:19:46.407737+0000","last_clean_scrub_stamp":"2024-04-25T19:19:46.407737+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:21:58.539078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"23'142","reported_seq":168,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:39.422011+0000","last_change":"2024-04-25T19:19:39.422070+0000","last_active":"2024-04-25T19:19:39.422011+0000","last_peered":"2024-04-25T19:19:39.422011+0000","last_clean":"2024-04-25T19:19:39.422011+0000","last_became_active":"2024-04-25T19:15:19.420044+0000","last_became_peered":"2024-04-25T19:15:19.420044+0000","last_unstale":"2024-04-25T19:19:39.422011+0000","last_undegraded":"2024-04-25T19:19:39.422011+0000","last_fullsized":"2024-04-25T19:19:39.422011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'142","last_scrub_stamp":"2024-04-25T19:19:39.421896+0000","last_deep_scrub":"23'142","last_deep_scrub_stamp":"2024-04-25T19:19:39.421896+0000","last_clean_scrub_stamp":"2024-04-25T19:19:39.421896+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:21.310067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"23'151","reported_seq":177,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:40.951267+0000","last_change":"2024-04-25T19:19:40.951339+0000","last_active":"2024-04-25T19:19:40.951267+0000","last_peered":"2024-04-25T19:19:40.951267+0000","last_clean":"2024-04-25T19:19:40.951267+0000","last_became_active":"2024-04-25T19:15:19.414007+0000","last_became_peered":"2024-04-25T19:15:19.414007+0000","last_unstale":"2024-04-25T19:19:40.951267+0000","last_undegraded":"2024-04-25T19:19:40.951267+0000","last_fullsized":"2024-04-25T19:19:40.951267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'151","last_scrub_stamp":"2024-04-25T19:19:40.951137+0000","last_deep_scrub":"23'151","last_deep_scrub_stamp":"2024-04-25T19:19:40.951137+0000","last_clean_scrub_stamp":"2024-04-25T19:19:40.951137+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:03.945338+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"23'149","reported_seq":175,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:44.741360+0000","last_change":"2024-04-25T19:19:44.741390+0000","last_active":"2024-04-25T19:19:44.741360+0000","last_peered":"2024-04-25T19:19:44.741360+0000","last_clean":"2024-04-25T19:19:44.741360+0000","last_became_active":"2024-04-25T19:15:19.422575+0000","last_became_peered":"2024-04-25T19:15:19.422575+0000","last_unstale":"2024-04-25T19:19:44.741360+0000","last_undegraded":"2024-04-25T19:19:44.741360+0000","last_fullsized":"2024-04-25T19:19:44.741360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'149","last_scrub_stamp":"2024-04-25T19:19:44.741267+0000","last_deep_scrub":"23'149","last_deep_scrub_stamp":"2024-04-25T19:19:44.741267+0000","last_clean_scrub_stamp":"2024-04-25T19:19:44.741267+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:01.588199+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"23'151","reported_seq":177,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:47.241189+0000","last_change":"2024-04-25T19:19:47.241215+0000","last_active":"2024-04-25T19:19:47.241189+0000","last_peered":"2024-04-25T19:19:47.241189+0000","last_clean":"2024-04-25T19:19:47.241189+0000","last_became_active":"2024-04-25T19:15:19.422953+0000","last_became_peered":"2024-04-25T19:15:19.422953+0000","last_unstale":"2024-04-25T19:19:47.241189+0000","last_undegraded":"2024-04-25T19:19:47.241189+0000","last_fullsized":"2024-04-25T19:19:47.241189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'151","last_scrub_stamp":"2024-04-25T19:19:47.241106+0000","last_deep_scrub":"23'151","last_deep_scrub_stamp":"2024-04-25T19:19:47.241106+0000","last_clean_scrub_stamp":"2024-04-25T19:19:47.241106+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:45.948044+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"23'147","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:45.236748+0000","last_change":"2024-04-25T19:19:45.236792+0000","last_active":"2024-04-25T19:19:45.236748+0000","last_peered":"2024-04-25T19:19:45.236748+0000","last_clean":"2024-04-25T19:19:45.236748+0000","last_became_active":"2024-04-25T19:16:17.801500+0000","last_became_peered":"2024-04-25T19:16:17.801500+0000","last_unstale":"2024-04-25T19:19:45.236748+0000","last_undegraded":"2024-04-25T19:19:45.236748+0000","last_fullsized":"2024-04-25T19:19:45.236748+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'147","last_scrub_stamp":"2024-04-25T19:19:45.236653+0000","last_deep_scrub":"23'147","last_deep_scrub_stamp":"2024-04-25T19:19:45.236653+0000","last_clean_scrub_stamp":"2024-04-25T19:19:45.236653+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:22:24.927160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"23'146","reported_seq":172,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:54.879198+0000","last_change":"2024-04-25T19:19:54.879227+0000","last_active":"2024-04-25T19:19:54.879198+0000","last_peered":"2024-04-25T19:19:54.879198+0000","last_clean":"2024-04-25T19:19:54.879198+0000","last_became_active":"2024-04-25T19:15:19.433836+0000","last_became_peered":"2024-04-25T19:15:19.433836+0000","last_unstale":"2024-04-25T19:19:54.879198+0000","last_undegraded":"2024-04-25T19:19:54.879198+0000","last_fullsized":"2024-04-25T19:19:54.879198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'146","last_scrub_stamp":"2024-04-25T19:19:54.879089+0000","last_deep_scrub":"23'146","last_deep_scrub_stamp":"2024-04-25T19:19:54.879089+0000","last_clean_scrub_stamp":"2024-04-25T19:19:54.879089+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:52.320442+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"23'157","reported_seq":183,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:42.197610+0000","last_change":"2024-04-25T19:19:42.197658+0000","last_active":"2024-04-25T19:19:42.197610+0000","last_peered":"2024-04-25T19:19:42.197610+0000","last_clean":"2024-04-25T19:19:42.197610+0000","last_became_active":"2024-04-25T19:15:19.415177+0000","last_became_peered":"2024-04-25T19:15:19.415177+0000","last_unstale":"2024-04-25T19:19:42.197610+0000","last_undegraded":"2024-04-25T19:19:42.197610+0000","last_fullsized":"2024-04-25T19:19:42.197610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'157","last_scrub_stamp":"2024-04-25T19:19:42.197500+0000","last_deep_scrub":"23'157","last_deep_scrub_stamp":"2024-04-25T19:19:42.197500+0000","last_clean_scrub_stamp":"2024-04-25T19:19:42.197500+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:50:27.585543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"23'172","reported_seq":199,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:51.916860+0000","last_change":"2024-04-25T19:19:51.916860+0000","last_active":"2024-04-25T19:19:51.916860+0000","last_peered":"2024-04-25T19:19:51.916860+0000","last_clean":"2024-04-25T19:19:51.916860+0000","last_became_active":"2024-04-25T19:15:19.427226+0000","last_became_peered":"2024-04-25T19:15:19.427226+0000","last_unstale":"2024-04-25T19:19:51.916860+0000","last_undegraded":"2024-04-25T19:19:51.916860+0000","last_fullsized":"2024-04-25T19:19:51.916860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'172","last_scrub_stamp":"2024-04-25T19:19:51.916722+0000","last_deep_scrub":"23'172","last_deep_scrub_stamp":"2024-04-25T19:19:51.916722+0000","last_clean_scrub_stamp":"2024-04-25T19:19:51.916722+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:36.592648+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"23'161","reported_seq":188,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:47.415060+0000","last_change":"2024-04-25T19:19:47.415060+0000","last_active":"2024-04-25T19:19:47.415060+0000","last_peered":"2024-04-25T19:19:47.415060+0000","last_clean":"2024-04-25T19:19:47.415060+0000","last_became_active":"2024-04-25T19:15:19.421300+0000","last_became_peered":"2024-04-25T19:15:19.421300+0000","last_unstale":"2024-04-25T19:19:47.415060+0000","last_undegraded":"2024-04-25T19:19:47.415060+0000","last_fullsized":"2024-04-25T19:19:47.415060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'161","last_scrub_stamp":"2024-04-25T19:19:47.414923+0000","last_deep_scrub":"23'161","last_deep_scrub_stamp":"2024-04-25T19:19:47.414923+0000","last_clean_scrub_stamp":"2024-04-25T19:19:47.414923+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:17:49.165224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"23'137","reported_seq":164,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:47.760691+0000","last_change":"2024-04-25T19:19:47.760691+0000","last_active":"2024-04-25T19:19:47.760691+0000","last_peered":"2024-04-25T19:19:47.760691+0000","last_clean":"2024-04-25T19:19:47.760691+0000","last_became_active":"2024-04-25T19:15:19.427579+0000","last_became_peered":"2024-04-25T19:15:19.427579+0000","last_unstale":"2024-04-25T19:19:47.760691+0000","last_undegraded":"2024-04-25T19:19:47.760691+0000","last_fullsized":"2024-04-25T19:19:47.760691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'137","last_scrub_stamp":"2024-04-25T19:19:47.760495+0000","last_deep_scrub":"23'137","last_deep_scrub_stamp":"2024-04-25T19:19:47.760495+0000","last_clean_scrub_stamp":"2024-04-25T19:19:47.760495+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:10:48.482569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"23'166","reported_seq":193,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:53.861974+0000","last_change":"2024-04-25T19:19:53.862005+0000","last_active":"2024-04-25T19:19:53.861974+0000","last_peered":"2024-04-25T19:19:53.861974+0000","last_clean":"2024-04-25T19:19:53.861974+0000","last_became_active":"2024-04-25T19:15:19.428305+0000","last_became_peered":"2024-04-25T19:15:19.428305+0000","last_unstale":"2024-04-25T19:19:53.861974+0000","last_undegraded":"2024-04-25T19:19:53.861974+0000","last_fullsized":"2024-04-25T19:19:53.861974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'166","last_scrub_stamp":"2024-04-25T19:19:53.861865+0000","last_deep_scrub":"23'166","last_deep_scrub_stamp":"2024-04-25T19:19:53.861865+0000","last_clean_scrub_stamp":"2024-04-25T19:19:53.861865+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:07.441736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"23'134","reported_seq":161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:49.089850+0000","last_change":"2024-04-25T19:19:49.089850+0000","last_active":"2024-04-25T19:19:49.089850+0000","last_peered":"2024-04-25T19:19:49.089850+0000","last_clean":"2024-04-25T19:19:49.089850+0000","last_became_active":"2024-04-25T19:15:19.422763+0000","last_became_peered":"2024-04-25T19:15:19.422763+0000","last_unstale":"2024-04-25T19:19:49.089850+0000","last_undegraded":"2024-04-25T19:19:49.089850+0000","last_fullsized":"2024-04-25T19:19:49.089850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'134","last_scrub_stamp":"2024-04-25T19:19:49.089641+0000","last_deep_scrub":"23'134","last_deep_scrub_stamp":"2024-04-25T19:19:49.089641+0000","last_clean_scrub_stamp":"2024-04-25T19:19:49.089641+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:22.612701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"23'180","reported_seq":207,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:45.744187+0000","last_change":"2024-04-25T19:19:45.744216+0000","last_active":"2024-04-25T19:19:45.744187+0000","last_peered":"2024-04-25T19:19:45.744187+0000","last_clean":"2024-04-25T19:19:45.744187+0000","last_became_active":"2024-04-25T19:15:19.419936+0000","last_became_peered":"2024-04-25T19:15:19.419936+0000","last_unstale":"2024-04-25T19:19:45.744187+0000","last_undegraded":"2024-04-25T19:19:45.744187+0000","last_fullsized":"2024-04-25T19:19:45.744187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'180","last_scrub_stamp":"2024-04-25T19:19:45.744086+0000","last_deep_scrub":"23'180","last_deep_scrub_stamp":"2024-04-25T19:19:45.744086+0000","last_clean_scrub_stamp":"2024-04-25T19:19:45.744086+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:03:12.593704+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"23'134","reported_seq":161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:48.055034+0000","last_change":"2024-04-25T19:19:48.055066+0000","last_active":"2024-04-25T19:19:48.055034+0000","last_peered":"2024-04-25T19:19:48.055034+0000","last_clean":"2024-04-25T19:19:48.055034+0000","last_became_active":"2024-04-25T19:15:19.423116+0000","last_became_peered":"2024-04-25T19:15:19.423116+0000","last_unstale":"2024-04-25T19:19:48.055034+0000","last_undegraded":"2024-04-25T19:19:48.055034+0000","last_fullsized":"2024-04-25T19:19:48.055034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'134","last_scrub_stamp":"2024-04-25T19:19:48.054940+0000","last_deep_scrub":"23'134","last_deep_scrub_stamp":"2024-04-25T19:19:48.054940+0000","last_clean_scrub_stamp":"2024-04-25T19:19:48.054940+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:26.301056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"23'198","reported_seq":225,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:49.419561+0000","last_change":"2024-04-25T19:19:49.419610+0000","last_active":"2024-04-25T19:19:49.419561+0000","last_peered":"2024-04-25T19:19:49.419561+0000","last_clean":"2024-04-25T19:19:49.419561+0000","last_became_active":"2024-04-25T19:15:19.428474+0000","last_became_peered":"2024-04-25T19:15:19.428474+0000","last_unstale":"2024-04-25T19:19:49.419561+0000","last_undegraded":"2024-04-25T19:19:49.419561+0000","last_fullsized":"2024-04-25T19:19:49.419561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'198","last_scrub_stamp":"2024-04-25T19:19:49.419396+0000","last_deep_scrub":"23'198","last_deep_scrub_stamp":"2024-04-25T19:19:49.419396+0000","last_clean_scrub_stamp":"2024-04-25T19:19:49.419396+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:15:48.199620+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"23'144","reported_seq":171,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:46.195084+0000","last_change":"2024-04-25T19:19:46.195134+0000","last_active":"2024-04-25T19:19:46.195084+0000","last_peered":"2024-04-25T19:19:46.195084+0000","last_clean":"2024-04-25T19:19:46.195084+0000","last_became_active":"2024-04-25T19:15:19.422795+0000","last_became_peered":"2024-04-25T19:15:19.422795+0000","last_unstale":"2024-04-25T19:19:46.195084+0000","last_undegraded":"2024-04-25T19:19:46.195084+0000","last_fullsized":"2024-04-25T19:19:46.195084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'144","last_scrub_stamp":"2024-04-25T19:19:46.194989+0000","last_deep_scrub":"23'144","last_deep_scrub_stamp":"2024-04-25T19:19:46.194989+0000","last_clean_scrub_stamp":"2024-04-25T19:19:46.194989+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:24.044058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"23'133","reported_seq":160,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:47.012155+0000","last_change":"2024-04-25T19:19:47.012184+0000","last_active":"2024-04-25T19:19:47.012155+0000","last_peered":"2024-04-25T19:19:47.012155+0000","last_clean":"2024-04-25T19:19:47.012155+0000","last_became_active":"2024-04-25T19:15:19.421515+0000","last_became_peered":"2024-04-25T19:15:19.421515+0000","last_unstale":"2024-04-25T19:19:47.012155+0000","last_undegraded":"2024-04-25T19:19:47.012155+0000","last_fullsized":"2024-04-25T19:19:47.012155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'133","last_scrub_stamp":"2024-04-25T19:19:47.012061+0000","last_deep_scrub":"23'133","last_deep_scrub_stamp":"2024-04-25T19:19:47.012061+0000","last_clean_scrub_stamp":"2024-04-25T19:19:47.012061+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:57.971833+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"23'118","reported_seq":145,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:38.730054+0000","last_change":"2024-04-25T19:19:38.730078+0000","last_active":"2024-04-25T19:19:38.730054+0000","last_peered":"2024-04-25T19:19:38.730054+0000","last_clean":"2024-04-25T19:19:38.730054+0000","last_became_active":"2024-04-25T19:15:19.409857+0000","last_became_peered":"2024-04-25T19:15:19.409857+0000","last_unstale":"2024-04-25T19:19:38.730054+0000","last_undegraded":"2024-04-25T19:19:38.730054+0000","last_fullsized":"2024-04-25T19:19:38.730054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'118","last_scrub_stamp":"2024-04-25T19:19:38.729976+0000","last_deep_scrub":"23'118","last_deep_scrub_stamp":"2024-04-25T19:19:38.729976+0000","last_clean_scrub_stamp":"2024-04-25T19:19:38.729976+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:36:08.727346+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"23'131","reported_seq":158,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:48.804536+0000","last_change":"2024-04-25T19:19:48.804567+0000","last_active":"2024-04-25T19:19:48.804536+0000","last_peered":"2024-04-25T19:19:48.804536+0000","last_clean":"2024-04-25T19:19:48.804536+0000","last_became_active":"2024-04-25T19:15:19.424578+0000","last_became_peered":"2024-04-25T19:15:19.424578+0000","last_unstale":"2024-04-25T19:19:48.804536+0000","last_undegraded":"2024-04-25T19:19:48.804536+0000","last_fullsized":"2024-04-25T19:19:48.804536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'131","last_scrub_stamp":"2024-04-25T19:19:48.804420+0000","last_deep_scrub":"23'131","last_deep_scrub_stamp":"2024-04-25T19:19:48.804420+0000","last_clean_scrub_stamp":"2024-04-25T19:19:48.804420+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:41:19.319289+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"23'135","reported_seq":162,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:43.209123+0000","last_change":"2024-04-25T19:19:43.209185+0000","last_active":"2024-04-25T19:19:43.209123+0000","last_peered":"2024-04-25T19:19:43.209123+0000","last_clean":"2024-04-25T19:19:43.209123+0000","last_became_active":"2024-04-25T19:15:19.426139+0000","last_became_peered":"2024-04-25T19:15:19.426139+0000","last_unstale":"2024-04-25T19:19:43.209123+0000","last_undegraded":"2024-04-25T19:19:43.209123+0000","last_fullsized":"2024-04-25T19:19:43.209123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'135","last_scrub_stamp":"2024-04-25T19:19:43.209030+0000","last_deep_scrub":"23'135","last_deep_scrub_stamp":"2024-04-25T19:19:43.209030+0000","last_clean_scrub_stamp":"2024-04-25T19:19:43.209030+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:26.515277+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"23'150","reported_seq":177,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:45.375155+0000","last_change":"2024-04-25T19:19:45.375186+0000","last_active":"2024-04-25T19:19:45.375155+0000","last_peered":"2024-04-25T19:19:45.375155+0000","last_clean":"2024-04-25T19:19:45.375155+0000","last_became_active":"2024-04-25T19:15:19.413455+0000","last_became_peered":"2024-04-25T19:15:19.413455+0000","last_unstale":"2024-04-25T19:19:45.375155+0000","last_undegraded":"2024-04-25T19:19:45.375155+0000","last_fullsized":"2024-04-25T19:19:45.375155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'150","last_scrub_stamp":"2024-04-25T19:19:45.375061+0000","last_deep_scrub":"23'150","last_deep_scrub_stamp":"2024-04-25T19:19:45.375061+0000","last_clean_scrub_stamp":"2024-04-25T19:19:45.375061+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:08.887517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"23'130","reported_seq":156,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:41.362625+0000","last_change":"2024-04-25T19:19:41.362655+0000","last_active":"2024-04-25T19:19:41.362625+0000","last_peered":"2024-04-25T19:19:41.362625+0000","last_clean":"2024-04-25T19:19:41.362625+0000","last_became_active":"2024-04-25T19:15:19.423406+0000","last_became_peered":"2024-04-25T19:15:19.423406+0000","last_unstale":"2024-04-25T19:19:41.362625+0000","last_undegraded":"2024-04-25T19:19:41.362625+0000","last_fullsized":"2024-04-25T19:19:41.362625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'130","last_scrub_stamp":"2024-04-25T19:19:41.362535+0000","last_deep_scrub":"23'130","last_deep_scrub_stamp":"2024-04-25T19:19:41.362535+0000","last_clean_scrub_stamp":"2024-04-25T19:19:41.362535+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:26.824888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"23'8","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:38.933005+0000","last_change":"2024-04-25T19:19:38.933034+0000","last_active":"2024-04-25T19:19:38.933005+0000","last_peered":"2024-04-25T19:19:38.933005+0000","last_clean":"2024-04-25T19:19:38.933005+0000","last_became_active":"2024-04-25T19:15:18.421658+0000","last_became_peered":"2024-04-25T19:15:18.421658+0000","last_unstale":"2024-04-25T19:19:38.933005+0000","last_undegraded":"2024-04-25T19:19:38.933005+0000","last_fullsized":"2024-04-25T19:19:38.933005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8","last_scrub_stamp":"2024-04-25T19:19:38.932930+0000","last_deep_scrub":"23'8","last_deep_scrub_stamp":"2024-04-25T19:19:38.932930+0000","last_clean_scrub_stamp":"2024-04-25T19:19:38.932930+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:46.276430+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"23'137","reported_seq":163,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:43.773342+0000","last_change":"2024-04-25T19:19:43.773371+0000","last_active":"2024-04-25T19:19:43.773342+0000","last_peered":"2024-04-25T19:19:43.773342+0000","last_clean":"2024-04-25T19:19:43.773342+0000","last_became_active":"2024-04-25T19:15:19.421497+0000","last_became_peered":"2024-04-25T19:15:19.421497+0000","last_unstale":"2024-04-25T19:19:43.773342+0000","last_undegraded":"2024-04-25T19:19:43.773342+0000","last_fullsized":"2024-04-25T19:19:43.773342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'137","last_scrub_stamp":"2024-04-25T19:19:43.773249+0000","last_deep_scrub":"23'137","last_deep_scrub_stamp":"2024-04-25T19:19:43.773249+0000","last_clean_scrub_stamp":"2024-04-25T19:19:43.773249+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:27:49.971665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"23'10","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:39.413964+0000","last_change":"2024-04-25T19:19:39.414028+0000","last_active":"2024-04-25T19:19:39.413964+0000","last_peered":"2024-04-25T19:19:39.413964+0000","last_clean":"2024-04-25T19:19:39.413964+0000","last_became_active":"2024-04-25T19:15:18.416539+0000","last_became_peered":"2024-04-25T19:15:18.416539+0000","last_unstale":"2024-04-25T19:19:39.413964+0000","last_undegraded":"2024-04-25T19:19:39.413964+0000","last_fullsized":"2024-04-25T19:19:39.413964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10","last_scrub_stamp":"2024-04-25T19:19:39.413851+0000","last_deep_scrub":"23'10","last_deep_scrub_stamp":"2024-04-25T19:19:39.413851+0000","last_clean_scrub_stamp":"2024-04-25T19:19:39.413851+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:51:58.810891+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"23'5","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:37.907836+0000","last_change":"2024-04-25T19:19:37.907868+0000","last_active":"2024-04-25T19:19:37.907836+0000","last_peered":"2024-04-25T19:19:37.907836+0000","last_clean":"2024-04-25T19:19:37.907836+0000","last_became_active":"2024-04-25T19:15:18.419732+0000","last_became_peered":"2024-04-25T19:15:18.419732+0000","last_unstale":"2024-04-25T19:19:37.907836+0000","last_undegraded":"2024-04-25T19:19:37.907836+0000","last_fullsized":"2024-04-25T19:19:37.907836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5","last_scrub_stamp":"2024-04-25T19:19:37.907769+0000","last_deep_scrub":"23'5","last_deep_scrub_stamp":"2024-04-25T19:19:37.907769+0000","last_clean_scrub_stamp":"2024-04-25T19:19:37.907769+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:06:57.534830+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"23'170","reported_seq":190,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:49.258765+0000","last_change":"2024-04-25T19:19:49.258800+0000","last_active":"2024-04-25T19:19:49.258765+0000","last_peered":"2024-04-25T19:19:49.258765+0000","last_clean":"2024-04-25T19:19:49.258765+0000","last_became_active":"2024-04-25T19:16:17.801451+0000","last_became_peered":"2024-04-25T19:16:17.801451+0000","last_unstale":"2024-04-25T19:19:49.258765+0000","last_undegraded":"2024-04-25T19:19:49.258765+0000","last_fullsized":"2024-04-25T19:19:49.258765+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'170","last_scrub_stamp":"2024-04-25T19:19:49.258660+0000","last_deep_scrub":"23'170","last_deep_scrub_stamp":"2024-04-25T19:19:49.258660+0000","last_clean_scrub_stamp":"2024-04-25T19:19:49.258660+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:38.671147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"23'4","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:39.692863+0000","last_change":"2024-04-25T19:19:39.692938+0000","last_active":"2024-04-25T19:19:39.692863+0000","last_peered":"2024-04-25T19:19:39.692863+0000","last_clean":"2024-04-25T19:19:39.692863+0000","last_became_active":"2024-04-25T19:15:18.420362+0000","last_became_peered":"2024-04-25T19:15:18.420362+0000","last_unstale":"2024-04-25T19:19:39.692863+0000","last_undegraded":"2024-04-25T19:19:39.692863+0000","last_fullsized":"2024-04-25T19:19:39.692863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4","last_scrub_stamp":"2024-04-25T19:19:39.692754+0000","last_deep_scrub":"23'4","last_deep_scrub_stamp":"2024-04-25T19:19:39.692754+0000","last_clean_scrub_stamp":"2024-04-25T19:19:39.692754+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:52.879002+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"23'157","reported_seq":183,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:48.246375+0000","last_change":"2024-04-25T19:19:48.246426+0000","last_active":"2024-04-25T19:19:48.246375+0000","last_peered":"2024-04-25T19:19:48.246375+0000","last_clean":"2024-04-25T19:19:48.246375+0000","last_became_active":"2024-04-25T19:15:19.423556+0000","last_became_peered":"2024-04-25T19:15:19.423556+0000","last_unstale":"2024-04-25T19:19:48.246375+0000","last_undegraded":"2024-04-25T19:19:48.246375+0000","last_fullsized":"2024-04-25T19:19:48.246375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'157","last_scrub_stamp":"2024-04-25T19:19:48.246265+0000","last_deep_scrub":"23'157","last_deep_scrub_stamp":"2024-04-25T19:19:48.246265+0000","last_clean_scrub_stamp":"2024-04-25T19:19:48.246265+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:41:12.362003+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'10","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:41.407604+0000","last_change":"2024-04-25T19:19:41.407640+0000","last_active":"2024-04-25T19:19:41.407604+0000","last_peered":"2024-04-25T19:19:41.407604+0000","last_clean":"2024-04-25T19:19:41.407604+0000","last_became_active":"2024-04-25T19:15:18.418927+0000","last_became_peered":"2024-04-25T19:15:18.418927+0000","last_unstale":"2024-04-25T19:19:41.407604+0000","last_undegraded":"2024-04-25T19:19:41.407604+0000","last_fullsized":"2024-04-25T19:19:41.407604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'10","last_scrub_stamp":"2024-04-25T19:19:41.407518+0000","last_deep_scrub":"23'10","last_deep_scrub_stamp":"2024-04-25T19:19:41.407518+0000","last_clean_scrub_stamp":"2024-04-25T19:19:41.407518+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:30:23.036559+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"23'229","reported_seq":255,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:52.859246+0000","last_change":"2024-04-25T19:19:52.859285+0000","last_active":"2024-04-25T19:19:52.859246+0000","last_peered":"2024-04-25T19:19:52.859246+0000","last_clean":"2024-04-25T19:19:52.859246+0000","last_became_active":"2024-04-25T19:15:19.428985+0000","last_became_peered":"2024-04-25T19:15:19.428985+0000","last_unstale":"2024-04-25T19:19:52.859246+0000","last_undegraded":"2024-04-25T19:19:52.859246+0000","last_fullsized":"2024-04-25T19:19:52.859246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'229","last_scrub_stamp":"2024-04-25T19:19:52.859142+0000","last_deep_scrub":"23'229","last_deep_scrub_stamp":"2024-04-25T19:19:52.859142+0000","last_clean_scrub_stamp":"2024-04-25T19:19:52.859142+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:48:19.277198+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"23'6","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:40.726752+0000","last_change":"2024-04-25T19:19:40.726816+0000","last_active":"2024-04-25T19:19:40.726752+0000","last_peered":"2024-04-25T19:19:40.726752+0000","last_clean":"2024-04-25T19:19:40.726752+0000","last_became_active":"2024-04-25T19:15:18.422610+0000","last_became_peered":"2024-04-25T19:15:18.422610+0000","last_unstale":"2024-04-25T19:19:40.726752+0000","last_undegraded":"2024-04-25T19:19:40.726752+0000","last_fullsized":"2024-04-25T19:19:40.726752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6","last_scrub_stamp":"2024-04-25T19:19:40.726640+0000","last_deep_scrub":"23'6","last_deep_scrub_stamp":"2024-04-25T19:19:40.726640+0000","last_clean_scrub_stamp":"2024-04-25T19:19:40.726640+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:45:40.171225+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"23'141","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:39.744342+0000","last_change":"2024-04-25T19:19:39.744367+0000","last_active":"2024-04-25T19:19:39.744342+0000","last_peered":"2024-04-25T19:19:39.744342+0000","last_clean":"2024-04-25T19:19:39.744342+0000","last_became_active":"2024-04-25T19:15:19.415734+0000","last_became_peered":"2024-04-25T19:15:19.415734+0000","last_unstale":"2024-04-25T19:19:39.744342+0000","last_undegraded":"2024-04-25T19:19:39.744342+0000","last_fullsized":"2024-04-25T19:19:39.744342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'141","last_scrub_stamp":"2024-04-25T19:19:39.744263+0000","last_deep_scrub":"23'141","last_deep_scrub_stamp":"2024-04-25T19:19:39.744263+0000","last_clean_scrub_stamp":"2024-04-25T19:19:39.744263+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:13:43.786923+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'13","reported_seq":41,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:42.210007+0000","last_change":"2024-04-25T19:19:42.210052+0000","last_active":"2024-04-25T19:19:42.210007+0000","last_peered":"2024-04-25T19:19:42.210007+0000","last_clean":"2024-04-25T19:19:42.210007+0000","last_became_active":"2024-04-25T19:15:18.428432+0000","last_became_peered":"2024-04-25T19:15:18.428432+0000","last_unstale":"2024-04-25T19:19:42.210007+0000","last_undegraded":"2024-04-25T19:19:42.210007+0000","last_fullsized":"2024-04-25T19:19:42.210007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'13","last_scrub_stamp":"2024-04-25T19:19:42.209918+0000","last_deep_scrub":"23'13","last_deep_scrub_stamp":"2024-04-25T19:19:42.209918+0000","last_clean_scrub_stamp":"2024-04-25T19:19:42.209918+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:29.688162+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'149","reported_seq":175,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:43.291440+0000","last_change":"2024-04-25T19:19:43.291469+0000","last_active":"2024-04-25T19:19:43.291440+0000","last_peered":"2024-04-25T19:19:43.291440+0000","last_clean":"2024-04-25T19:19:43.291440+0000","last_became_active":"2024-04-25T19:15:19.427941+0000","last_became_peered":"2024-04-25T19:15:19.427941+0000","last_unstale":"2024-04-25T19:19:43.291440+0000","last_undegraded":"2024-04-25T19:19:43.291440+0000","last_fullsized":"2024-04-25T19:19:43.291440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'149","last_scrub_stamp":"2024-04-25T19:19:43.291329+0000","last_deep_scrub":"23'149","last_deep_scrub_stamp":"2024-04-25T19:19:43.291329+0000","last_clean_scrub_stamp":"2024-04-25T19:19:43.291329+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:42.828832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'106","reported_seq":135,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:42.375737+0000","last_change":"2024-04-25T19:19:42.375737+0000","last_active":"2024-04-25T19:19:42.375737+0000","last_peered":"2024-04-25T19:19:42.375737+0000","last_clean":"2024-04-25T19:19:42.375737+0000","last_became_active":"2024-04-25T19:15:18.420303+0000","last_became_peered":"2024-04-25T19:15:18.420303+0000","last_unstale":"2024-04-25T19:19:42.375737+0000","last_undegraded":"2024-04-25T19:19:42.375737+0000","last_fullsized":"2024-04-25T19:19:42.375737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'106","last_scrub_stamp":"2024-04-25T19:19:42.375605+0000","last_deep_scrub":"23'106","last_deep_scrub_stamp":"2024-04-25T19:19:42.375605+0000","last_clean_scrub_stamp":"2024-04-25T19:19:42.375605+0000","objects_scrubbed":6,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:56.235780+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":216794,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103,"num_write_kb":251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"23'150","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:44.239472+0000","last_change":"2024-04-25T19:19:44.239508+0000","last_active":"2024-04-25T19:19:44.239472+0000","last_peered":"2024-04-25T19:19:44.239472+0000","last_clean":"2024-04-25T19:19:44.239472+0000","last_became_active":"2024-04-25T19:16:17.661023+0000","last_became_peered":"2024-04-25T19:16:17.661023+0000","last_unstale":"2024-04-25T19:19:44.239472+0000","last_undegraded":"2024-04-25T19:19:44.239472+0000","last_fullsized":"2024-04-25T19:19:44.239472+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'150","last_scrub_stamp":"2024-04-25T19:19:44.239367+0000","last_deep_scrub":"23'150","last_deep_scrub_stamp":"2024-04-25T19:19:44.239367+0000","last_clean_scrub_stamp":"2024-04-25T19:19:44.239367+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:27:31.255151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.13","version":"23'15","reported_seq":44,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:36.690953+0000","last_change":"2024-04-25T19:19:36.690953+0000","last_active":"2024-04-25T19:19:36.690953+0000","last_peered":"2024-04-25T19:19:36.690953+0000","last_clean":"2024-04-25T19:19:36.690953+0000","last_became_active":"2024-04-25T19:15:18.420402+0000","last_became_peered":"2024-04-25T19:15:18.420402+0000","last_unstale":"2024-04-25T19:19:36.690953+0000","last_undegraded":"2024-04-25T19:19:36.690953+0000","last_fullsized":"2024-04-25T19:19:36.690953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'15","last_scrub_stamp":"2024-04-25T19:19:36.690811+0000","last_deep_scrub":"23'15","last_deep_scrub_stamp":"2024-04-25T19:19:36.690811+0000","last_clean_scrub_stamp":"2024-04-25T19:19:36.690811+0000","objects_scrubbed":6,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:58:33.775049+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"23'157","reported_seq":183,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:45.716565+0000","last_change":"2024-04-25T19:19:45.716611+0000","last_active":"2024-04-25T19:19:45.716565+0000","last_peered":"2024-04-25T19:19:45.716565+0000","last_clean":"2024-04-25T19:19:45.716565+0000","last_became_active":"2024-04-25T19:15:19.424512+0000","last_became_peered":"2024-04-25T19:15:19.424512+0000","last_unstale":"2024-04-25T19:19:45.716565+0000","last_undegraded":"2024-04-25T19:19:45.716565+0000","last_fullsized":"2024-04-25T19:19:45.716565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'157","last_scrub_stamp":"2024-04-25T19:19:45.716445+0000","last_deep_scrub":"23'157","last_deep_scrub_stamp":"2024-04-25T19:19:45.716445+0000","last_clean_scrub_stamp":"2024-04-25T19:19:45.716445+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:14:27.600184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"23'8","reported_seq":37,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:43.763243+0000","last_change":"2024-04-25T19:19:43.763243+0000","last_active":"2024-04-25T19:19:43.763243+0000","last_peered":"2024-04-25T19:19:43.763243+0000","last_clean":"2024-04-25T19:19:43.763243+0000","last_became_active":"2024-04-25T19:15:18.426472+0000","last_became_peered":"2024-04-25T19:15:18.426472+0000","last_unstale":"2024-04-25T19:19:43.763243+0000","last_undegraded":"2024-04-25T19:19:43.763243+0000","last_fullsized":"2024-04-25T19:19:43.763243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8","last_scrub_stamp":"2024-04-25T19:19:43.763104+0000","last_deep_scrub":"23'8","last_deep_scrub_stamp":"2024-04-25T19:19:43.763104+0000","last_clean_scrub_stamp":"2024-04-25T19:19:43.763104+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:10:31.634869+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'171","reported_seq":197,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:48.955114+0000","last_change":"2024-04-25T19:19:48.955145+0000","last_active":"2024-04-25T19:19:48.955114+0000","last_peered":"2024-04-25T19:19:48.955114+0000","last_clean":"2024-04-25T19:19:48.955114+0000","last_became_active":"2024-04-25T19:15:19.428450+0000","last_became_peered":"2024-04-25T19:15:19.428450+0000","last_unstale":"2024-04-25T19:19:48.955114+0000","last_undegraded":"2024-04-25T19:19:48.955114+0000","last_fullsized":"2024-04-25T19:19:48.955114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'171","last_scrub_stamp":"2024-04-25T19:19:48.954997+0000","last_deep_scrub":"23'171","last_deep_scrub_stamp":"2024-04-25T19:19:48.954997+0000","last_clean_scrub_stamp":"2024-04-25T19:19:48.954997+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:44.880920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"23'9","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:40.421171+0000","last_change":"2024-04-25T19:19:40.421171+0000","last_active":"2024-04-25T19:19:40.421171+0000","last_peered":"2024-04-25T19:19:40.421171+0000","last_clean":"2024-04-25T19:19:40.421171+0000","last_became_active":"2024-04-25T19:15:18.417404+0000","last_became_peered":"2024-04-25T19:15:18.417404+0000","last_unstale":"2024-04-25T19:19:40.421171+0000","last_undegraded":"2024-04-25T19:19:40.421171+0000","last_fullsized":"2024-04-25T19:19:40.421171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9","last_scrub_stamp":"2024-04-25T19:19:40.421014+0000","last_deep_scrub":"23'9","last_deep_scrub_stamp":"2024-04-25T19:19:40.421014+0000","last_clean_scrub_stamp":"2024-04-25T19:19:40.421014+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:00.381909+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"23'165","reported_seq":191,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:44.407572+0000","last_change":"2024-04-25T19:19:44.407603+0000","last_active":"2024-04-25T19:19:44.407572+0000","last_peered":"2024-04-25T19:19:44.407572+0000","last_clean":"2024-04-25T19:19:44.407572+0000","last_became_active":"2024-04-25T19:15:19.410814+0000","last_became_peered":"2024-04-25T19:15:19.410814+0000","last_unstale":"2024-04-25T19:19:44.407572+0000","last_undegraded":"2024-04-25T19:19:44.407572+0000","last_fullsized":"2024-04-25T19:19:44.407572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'165","last_scrub_stamp":"2024-04-25T19:19:44.407441+0000","last_deep_scrub":"23'165","last_deep_scrub_stamp":"2024-04-25T19:19:44.407441+0000","last_clean_scrub_stamp":"2024-04-25T19:19:44.407441+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:53.313256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'26","reported_seq":55,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:40.220532+0000","last_change":"2024-04-25T19:19:40.220532+0000","last_active":"2024-04-25T19:19:40.220532+0000","last_peered":"2024-04-25T19:19:40.220532+0000","last_clean":"2024-04-25T19:19:40.220532+0000","last_became_active":"2024-04-25T19:15:18.421900+0000","last_became_peered":"2024-04-25T19:15:18.421900+0000","last_unstale":"2024-04-25T19:19:40.220532+0000","last_undegraded":"2024-04-25T19:19:40.220532+0000","last_fullsized":"2024-04-25T19:19:40.220532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'26","last_scrub_stamp":"2024-04-25T19:19:40.220418+0000","last_deep_scrub":"23'26","last_deep_scrub_stamp":"2024-04-25T19:19:40.220418+0000","last_clean_scrub_stamp":"2024-04-25T19:19:40.220418+0000","objects_scrubbed":8,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:17.932016+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"23'189","reported_seq":215,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:49.986336+0000","last_change":"2024-04-25T19:19:49.986369+0000","last_active":"2024-04-25T19:19:49.986336+0000","last_peered":"2024-04-25T19:19:49.986336+0000","last_clean":"2024-04-25T19:19:49.986336+0000","last_became_active":"2024-04-25T19:15:19.428563+0000","last_became_peered":"2024-04-25T19:15:19.428563+0000","last_unstale":"2024-04-25T19:19:49.986336+0000","last_undegraded":"2024-04-25T19:19:49.986336+0000","last_fullsized":"2024-04-25T19:19:49.986336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'189","last_scrub_stamp":"2024-04-25T19:19:49.986238+0000","last_deep_scrub":"23'189","last_deep_scrub_stamp":"2024-04-25T19:19:49.986238+0000","last_clean_scrub_stamp":"2024-04-25T19:19:49.986238+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:36.578954+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'4","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:38.383518+0000","last_change":"2024-04-25T19:19:38.383518+0000","last_active":"2024-04-25T19:19:38.383518+0000","last_peered":"2024-04-25T19:19:38.383518+0000","last_clean":"2024-04-25T19:19:38.383518+0000","last_became_active":"2024-04-25T19:15:18.421777+0000","last_became_peered":"2024-04-25T19:15:18.421777+0000","last_unstale":"2024-04-25T19:19:38.383518+0000","last_undegraded":"2024-04-25T19:19:38.383518+0000","last_fullsized":"2024-04-25T19:19:38.383518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4","last_scrub_stamp":"2024-04-25T19:19:38.383339+0000","last_deep_scrub":"23'4","last_deep_scrub_stamp":"2024-04-25T19:19:38.383339+0000","last_clean_scrub_stamp":"2024-04-25T19:19:38.383339+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:12.638184+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"23'146","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:50.941758+0000","last_change":"2024-04-25T19:19:50.941758+0000","last_active":"2024-04-25T19:19:50.941758+0000","last_peered":"2024-04-25T19:19:50.941758+0000","last_clean":"2024-04-25T19:19:50.941758+0000","last_became_active":"2024-04-25T19:15:19.429316+0000","last_became_peered":"2024-04-25T19:15:19.429316+0000","last_unstale":"2024-04-25T19:19:50.941758+0000","last_undegraded":"2024-04-25T19:19:50.941758+0000","last_fullsized":"2024-04-25T19:19:50.941758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'146","last_scrub_stamp":"2024-04-25T19:19:50.941587+0000","last_deep_scrub":"23'146","last_deep_scrub_stamp":"2024-04-25T19:19:50.941587+0000","last_clean_scrub_stamp":"2024-04-25T19:19:50.941587+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:48:51.401561+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'7","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:38.245941+0000","last_change":"2024-04-25T19:19:38.245941+0000","last_active":"2024-04-25T19:19:38.245941+0000","last_peered":"2024-04-25T19:19:38.245941+0000","last_clean":"2024-04-25T19:19:38.245941+0000","last_became_active":"2024-04-25T19:15:18.418359+0000","last_became_peered":"2024-04-25T19:15:18.418359+0000","last_unstale":"2024-04-25T19:19:38.245941+0000","last_undegraded":"2024-04-25T19:19:38.245941+0000","last_fullsized":"2024-04-25T19:19:38.245941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7","last_scrub_stamp":"2024-04-25T19:19:38.245832+0000","last_deep_scrub":"23'7","last_deep_scrub_stamp":"2024-04-25T19:19:38.245832+0000","last_clean_scrub_stamp":"2024-04-25T19:19:38.245832+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:23.323131+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"23'138","reported_seq":165,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:55.908325+0000","last_change":"2024-04-25T19:19:55.908357+0000","last_active":"2024-04-25T19:19:55.908325+0000","last_peered":"2024-04-25T19:19:55.908325+0000","last_clean":"2024-04-25T19:19:55.908325+0000","last_became_active":"2024-04-25T19:15:19.434508+0000","last_became_peered":"2024-04-25T19:15:19.434508+0000","last_unstale":"2024-04-25T19:19:55.908325+0000","last_undegraded":"2024-04-25T19:19:55.908325+0000","last_fullsized":"2024-04-25T19:19:55.908325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'138","last_scrub_stamp":"2024-04-25T19:19:55.908212+0000","last_deep_scrub":"23'138","last_deep_scrub_stamp":"2024-04-25T19:19:55.908212+0000","last_clean_scrub_stamp":"2024-04-25T19:19:55.908212+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:24.882942+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"23'4","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:41.161632+0000","last_change":"2024-04-25T19:19:41.161632+0000","last_active":"2024-04-25T19:19:41.161632+0000","last_peered":"2024-04-25T19:19:41.161632+0000","last_clean":"2024-04-25T19:19:41.161632+0000","last_became_active":"2024-04-25T19:15:18.428332+0000","last_became_peered":"2024-04-25T19:15:18.428332+0000","last_unstale":"2024-04-25T19:19:41.161632+0000","last_undegraded":"2024-04-25T19:19:41.161632+0000","last_fullsized":"2024-04-25T19:19:41.161632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4","last_scrub_stamp":"2024-04-25T19:19:41.161512+0000","last_deep_scrub":"23'4","last_deep_scrub_stamp":"2024-04-25T19:19:41.161512+0000","last_clean_scrub_stamp":"2024-04-25T19:19:41.161512+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:47.179595+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"23'144","reported_seq":171,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:50.818752+0000","last_change":"2024-04-25T19:19:50.818752+0000","last_active":"2024-04-25T19:19:50.818752+0000","last_peered":"2024-04-25T19:19:50.818752+0000","last_clean":"2024-04-25T19:19:50.818752+0000","last_became_active":"2024-04-25T19:15:19.427992+0000","last_became_peered":"2024-04-25T19:15:19.427992+0000","last_unstale":"2024-04-25T19:19:50.818752+0000","last_undegraded":"2024-04-25T19:19:50.818752+0000","last_fullsized":"2024-04-25T19:19:50.818752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'144","last_scrub_stamp":"2024-04-25T19:19:50.818584+0000","last_deep_scrub":"23'144","last_deep_scrub_stamp":"2024-04-25T19:19:50.818584+0000","last_clean_scrub_stamp":"2024-04-25T19:19:50.818584+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:28:17.521294+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"23'9","reported_seq":38,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:39.222401+0000","last_change":"2024-04-25T19:19:39.222401+0000","last_active":"2024-04-25T19:19:39.222401+0000","last_peered":"2024-04-25T19:19:39.222401+0000","last_clean":"2024-04-25T19:19:39.222401+0000","last_became_active":"2024-04-25T19:15:18.419286+0000","last_became_peered":"2024-04-25T19:15:18.419286+0000","last_unstale":"2024-04-25T19:19:39.222401+0000","last_undegraded":"2024-04-25T19:19:39.222401+0000","last_fullsized":"2024-04-25T19:19:39.222401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9","last_scrub_stamp":"2024-04-25T19:19:39.222242+0000","last_deep_scrub":"23'9","last_deep_scrub_stamp":"2024-04-25T19:19:39.222242+0000","last_clean_scrub_stamp":"2024-04-25T19:19:39.222242+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:07:05.690722+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1752,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'151","reported_seq":178,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:53.909199+0000","last_change":"2024-04-25T19:19:53.909232+0000","last_active":"2024-04-25T19:19:53.909199+0000","last_peered":"2024-04-25T19:19:53.909199+0000","last_clean":"2024-04-25T19:19:53.909199+0000","last_became_active":"2024-04-25T19:15:19.431739+0000","last_became_peered":"2024-04-25T19:15:19.431739+0000","last_unstale":"2024-04-25T19:19:53.909199+0000","last_undegraded":"2024-04-25T19:19:53.909199+0000","last_fullsized":"2024-04-25T19:19:53.909199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'151","last_scrub_stamp":"2024-04-25T19:19:53.909071+0000","last_deep_scrub":"23'151","last_deep_scrub_stamp":"2024-04-25T19:19:53.909071+0000","last_clean_scrub_stamp":"2024-04-25T19:19:53.909071+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:33:36.174250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"23'74","reported_seq":127,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:40.942732+0000","last_change":"2024-04-25T19:19:40.942787+0000","last_active":"2024-04-25T19:19:40.942732+0000","last_peered":"2024-04-25T19:19:40.942732+0000","last_clean":"2024-04-25T19:19:40.942732+0000","last_became_active":"2024-04-25T19:15:15.603730+0000","last_became_peered":"2024-04-25T19:15:15.603730+0000","last_unstale":"2024-04-25T19:19:40.942732+0000","last_undegraded":"2024-04-25T19:19:40.942732+0000","last_fullsized":"2024-04-25T19:19:40.942732+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'74","last_scrub_stamp":"2024-04-25T19:19:40.942627+0000","last_deep_scrub":"23'74","last_deep_scrub_stamp":"2024-04-25T19:19:40.942627+0000","last_clean_scrub_stamp":"2024-04-25T19:19:40.942627+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:53.267031+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"23'11","reported_seq":40,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:42.794542+0000","last_change":"2024-04-25T19:19:42.794608+0000","last_active":"2024-04-25T19:19:42.794542+0000","last_peered":"2024-04-25T19:19:42.794542+0000","last_clean":"2024-04-25T19:19:42.794542+0000","last_became_active":"2024-04-25T19:15:18.425993+0000","last_became_peered":"2024-04-25T19:15:18.425993+0000","last_unstale":"2024-04-25T19:19:42.794542+0000","last_undegraded":"2024-04-25T19:19:42.794542+0000","last_fullsized":"2024-04-25T19:19:42.794542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11","last_scrub_stamp":"2024-04-25T19:19:42.794429+0000","last_deep_scrub":"23'11","last_deep_scrub_stamp":"2024-04-25T19:19:42.794429+0000","last_clean_scrub_stamp":"2024-04-25T19:19:42.794429+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:43.698457+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"23'158","reported_seq":185,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:43.976355+0000","last_change":"2024-04-25T19:19:43.976385+0000","last_active":"2024-04-25T19:19:43.976355+0000","last_peered":"2024-04-25T19:19:43.976355+0000","last_clean":"2024-04-25T19:19:43.976355+0000","last_became_active":"2024-04-25T19:15:19.416808+0000","last_became_peered":"2024-04-25T19:15:19.416808+0000","last_unstale":"2024-04-25T19:19:43.976355+0000","last_undegraded":"2024-04-25T19:19:43.976355+0000","last_fullsized":"2024-04-25T19:19:43.976355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'158","last_scrub_stamp":"2024-04-25T19:19:43.976245+0000","last_deep_scrub":"23'158","last_deep_scrub_stamp":"2024-04-25T19:19:43.976245+0000","last_clean_scrub_stamp":"2024-04-25T19:19:43.976245+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:52.242475+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"23'5","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:46.004065+0000","last_change":"2024-04-25T19:19:46.004101+0000","last_active":"2024-04-25T19:19:46.004065+0000","last_peered":"2024-04-25T19:19:46.004065+0000","last_clean":"2024-04-25T19:19:46.004065+0000","last_became_active":"2024-04-25T19:15:18.426037+0000","last_became_peered":"2024-04-25T19:15:18.426037+0000","last_unstale":"2024-04-25T19:19:46.004065+0000","last_undegraded":"2024-04-25T19:19:46.004065+0000","last_fullsized":"2024-04-25T19:19:46.004065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5","last_scrub_stamp":"2024-04-25T19:19:46.003967+0000","last_deep_scrub":"23'5","last_deep_scrub_stamp":"2024-04-25T19:19:46.003967+0000","last_clean_scrub_stamp":"2024-04-25T19:19:46.003967+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:25.409329+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"23'131","reported_seq":158,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:46.729523+0000","last_change":"2024-04-25T19:19:46.729555+0000","last_active":"2024-04-25T19:19:46.729523+0000","last_peered":"2024-04-25T19:19:46.729523+0000","last_clean":"2024-04-25T19:19:46.729523+0000","last_became_active":"2024-04-25T19:15:19.425626+0000","last_became_peered":"2024-04-25T19:15:19.425626+0000","last_unstale":"2024-04-25T19:19:46.729523+0000","last_undegraded":"2024-04-25T19:19:46.729523+0000","last_fullsized":"2024-04-25T19:19:46.729523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'131","last_scrub_stamp":"2024-04-25T19:19:46.729418+0000","last_deep_scrub":"23'131","last_deep_scrub_stamp":"2024-04-25T19:19:46.729418+0000","last_clean_scrub_stamp":"2024-04-25T19:19:46.729418+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:34.020405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'14","reported_seq":47,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:41.216317+0000","last_change":"2024-04-25T19:19:41.216352+0000","last_active":"2024-04-25T19:19:41.216317+0000","last_peered":"2024-04-25T19:19:41.216317+0000","last_clean":"2024-04-25T19:19:41.216317+0000","last_became_active":"2024-04-25T19:15:18.424179+0000","last_became_peered":"2024-04-25T19:15:18.424179+0000","last_unstale":"2024-04-25T19:19:41.216317+0000","last_undegraded":"2024-04-25T19:19:41.216317+0000","last_fullsized":"2024-04-25T19:19:41.216317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'14","last_scrub_stamp":"2024-04-25T19:19:41.216225+0000","last_deep_scrub":"23'14","last_deep_scrub_stamp":"2024-04-25T19:19:41.216225+0000","last_clean_scrub_stamp":"2024-04-25T19:19:41.216225+0000","objects_scrubbed":4,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:58.602753+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"23'131","reported_seq":158,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:43.247182+0000","last_change":"2024-04-25T19:19:43.247231+0000","last_active":"2024-04-25T19:19:43.247182+0000","last_peered":"2024-04-25T19:19:43.247182+0000","last_clean":"2024-04-25T19:19:43.247182+0000","last_became_active":"2024-04-25T19:15:19.414877+0000","last_became_peered":"2024-04-25T19:15:19.414877+0000","last_unstale":"2024-04-25T19:19:43.247182+0000","last_undegraded":"2024-04-25T19:19:43.247182+0000","last_fullsized":"2024-04-25T19:19:43.247182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'131","last_scrub_stamp":"2024-04-25T19:19:43.247055+0000","last_deep_scrub":"23'131","last_deep_scrub_stamp":"2024-04-25T19:19:43.247055+0000","last_clean_scrub_stamp":"2024-04-25T19:19:43.247055+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:30.693751+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'7","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:41.764285+0000","last_change":"2024-04-25T19:19:41.764318+0000","last_active":"2024-04-25T19:19:41.764285+0000","last_peered":"2024-04-25T19:19:41.764285+0000","last_clean":"2024-04-25T19:19:41.764285+0000","last_became_active":"2024-04-25T19:15:18.422330+0000","last_became_peered":"2024-04-25T19:15:18.422330+0000","last_unstale":"2024-04-25T19:19:41.764285+0000","last_undegraded":"2024-04-25T19:19:41.764285+0000","last_fullsized":"2024-04-25T19:19:41.764285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7","last_scrub_stamp":"2024-04-25T19:19:41.764178+0000","last_deep_scrub":"23'7","last_deep_scrub_stamp":"2024-04-25T19:19:41.764178+0000","last_clean_scrub_stamp":"2024-04-25T19:19:41.764178+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:00.884432+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'145","reported_seq":172,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:40.403247+0000","last_change":"2024-04-25T19:19:40.403296+0000","last_active":"2024-04-25T19:19:40.403247+0000","last_peered":"2024-04-25T19:19:40.403247+0000","last_clean":"2024-04-25T19:19:40.403247+0000","last_became_active":"2024-04-25T19:15:19.420598+0000","last_became_peered":"2024-04-25T19:15:19.420598+0000","last_unstale":"2024-04-25T19:19:40.403247+0000","last_undegraded":"2024-04-25T19:19:40.403247+0000","last_fullsized":"2024-04-25T19:19:40.403247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'145","last_scrub_stamp":"2024-04-25T19:19:40.403106+0000","last_deep_scrub":"23'145","last_deep_scrub_stamp":"2024-04-25T19:19:40.403106+0000","last_clean_scrub_stamp":"2024-04-25T19:19:40.403106+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:48.806193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"21'1","reported_seq":30,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:38.138886+0000","last_change":"2024-04-25T19:19:38.138912+0000","last_active":"2024-04-25T19:19:38.138886+0000","last_peered":"2024-04-25T19:19:38.138886+0000","last_clean":"2024-04-25T19:19:38.138886+0000","last_became_active":"2024-04-25T19:15:18.422621+0000","last_became_peered":"2024-04-25T19:15:18.422621+0000","last_unstale":"2024-04-25T19:19:38.138886+0000","last_undegraded":"2024-04-25T19:19:38.138886+0000","last_fullsized":"2024-04-25T19:19:38.138886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'1","last_scrub_stamp":"2024-04-25T19:19:38.138804+0000","last_deep_scrub":"21'1","last_deep_scrub_stamp":"2024-04-25T19:19:38.138804+0000","last_clean_scrub_stamp":"2024-04-25T19:19:38.138804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:06:56.911666+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'143","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:49.833722+0000","last_change":"2024-04-25T19:19:49.833753+0000","last_active":"2024-04-25T19:19:49.833722+0000","last_peered":"2024-04-25T19:19:49.833722+0000","last_clean":"2024-04-25T19:19:49.833722+0000","last_became_active":"2024-04-25T19:15:19.425997+0000","last_became_peered":"2024-04-25T19:15:19.425997+0000","last_unstale":"2024-04-25T19:19:49.833722+0000","last_undegraded":"2024-04-25T19:19:49.833722+0000","last_fullsized":"2024-04-25T19:19:49.833722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'143","last_scrub_stamp":"2024-04-25T19:19:49.833617+0000","last_deep_scrub":"23'143","last_deep_scrub_stamp":"2024-04-25T19:19:49.833617+0000","last_clean_scrub_stamp":"2024-04-25T19:19:49.833617+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:23:29.700916+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"23'7","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:46.955652+0000","last_change":"2024-04-25T19:19:46.955687+0000","last_active":"2024-04-25T19:19:46.955652+0000","last_peered":"2024-04-25T19:19:46.955652+0000","last_clean":"2024-04-25T19:19:46.955652+0000","last_became_active":"2024-04-25T19:15:18.427374+0000","last_became_peered":"2024-04-25T19:15:18.427374+0000","last_unstale":"2024-04-25T19:19:46.955652+0000","last_undegraded":"2024-04-25T19:19:46.955652+0000","last_fullsized":"2024-04-25T19:19:46.955652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7","last_scrub_stamp":"2024-04-25T19:19:46.955560+0000","last_deep_scrub":"23'7","last_deep_scrub_stamp":"2024-04-25T19:19:46.955560+0000","last_clean_scrub_stamp":"2024-04-25T19:19:46.955560+0000","objects_scrubbed":3,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:29:58.063543+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'144","reported_seq":171,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:45.186737+0000","last_change":"2024-04-25T19:19:45.186808+0000","last_active":"2024-04-25T19:19:45.186737+0000","last_peered":"2024-04-25T19:19:45.186737+0000","last_clean":"2024-04-25T19:19:45.186737+0000","last_became_active":"2024-04-25T19:15:19.421835+0000","last_became_peered":"2024-04-25T19:15:19.421835+0000","last_unstale":"2024-04-25T19:19:45.186737+0000","last_undegraded":"2024-04-25T19:19:45.186737+0000","last_fullsized":"2024-04-25T19:19:45.186737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'144","last_scrub_stamp":"2024-04-25T19:19:45.186625+0000","last_deep_scrub":"23'144","last_deep_scrub_stamp":"2024-04-25T19:19:45.186625+0000","last_clean_scrub_stamp":"2024-04-25T19:19:45.186625+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:29:27.012490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'27","reported_seq":56,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:37.164797+0000","last_change":"2024-04-25T19:19:37.164797+0000","last_active":"2024-04-25T19:19:37.164797+0000","last_peered":"2024-04-25T19:19:37.164797+0000","last_clean":"2024-04-25T19:19:37.164797+0000","last_became_active":"2024-04-25T19:15:18.418775+0000","last_became_peered":"2024-04-25T19:15:18.418775+0000","last_unstale":"2024-04-25T19:19:37.164797+0000","last_undegraded":"2024-04-25T19:19:37.164797+0000","last_fullsized":"2024-04-25T19:19:37.164797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'27","last_scrub_stamp":"2024-04-25T19:19:37.164683+0000","last_deep_scrub":"23'27","last_deep_scrub_stamp":"2024-04-25T19:19:37.164683+0000","last_clean_scrub_stamp":"2024-04-25T19:19:37.164683+0000","objects_scrubbed":4,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:16.469492+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'143","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:50.043199+0000","last_change":"2024-04-25T19:19:50.043238+0000","last_active":"2024-04-25T19:19:50.043199+0000","last_peered":"2024-04-25T19:19:50.043199+0000","last_clean":"2024-04-25T19:19:50.043199+0000","last_became_active":"2024-04-25T19:15:19.431905+0000","last_became_peered":"2024-04-25T19:15:19.431905+0000","last_unstale":"2024-04-25T19:19:50.043199+0000","last_undegraded":"2024-04-25T19:19:50.043199+0000","last_fullsized":"2024-04-25T19:19:50.043199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'143","last_scrub_stamp":"2024-04-25T19:19:50.043095+0000","last_deep_scrub":"23'143","last_deep_scrub_stamp":"2024-04-25T19:19:50.043095+0000","last_clean_scrub_stamp":"2024-04-25T19:19:50.043095+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:10:31.017754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'12","reported_seq":41,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:43.372811+0000","last_change":"2024-04-25T19:19:43.372849+0000","last_active":"2024-04-25T19:19:43.372811+0000","last_peered":"2024-04-25T19:19:43.372811+0000","last_clean":"2024-04-25T19:19:43.372811+0000","last_became_active":"2024-04-25T19:15:18.422838+0000","last_became_peered":"2024-04-25T19:15:18.422838+0000","last_unstale":"2024-04-25T19:19:43.372811+0000","last_undegraded":"2024-04-25T19:19:43.372811+0000","last_fullsized":"2024-04-25T19:19:43.372811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'12","last_scrub_stamp":"2024-04-25T19:19:43.372719+0000","last_deep_scrub":"23'12","last_deep_scrub_stamp":"2024-04-25T19:19:43.372719+0000","last_clean_scrub_stamp":"2024-04-25T19:19:43.372719+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:13:18.732766+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'163","reported_seq":190,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:42.786732+0000","last_change":"2024-04-25T19:19:42.786732+0000","last_active":"2024-04-25T19:19:42.786732+0000","last_peered":"2024-04-25T19:19:42.786732+0000","last_clean":"2024-04-25T19:19:42.786732+0000","last_became_active":"2024-04-25T19:15:19.420270+0000","last_became_peered":"2024-04-25T19:15:19.420270+0000","last_unstale":"2024-04-25T19:19:42.786732+0000","last_undegraded":"2024-04-25T19:19:42.786732+0000","last_fullsized":"2024-04-25T19:19:42.786732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'163","last_scrub_stamp":"2024-04-25T19:19:42.786599+0000","last_deep_scrub":"23'163","last_deep_scrub_stamp":"2024-04-25T19:19:42.786599+0000","last_clean_scrub_stamp":"2024-04-25T19:19:42.786599+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:52:29.458587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'3","reported_seq":32,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:45.004413+0000","last_change":"2024-04-25T19:19:45.004446+0000","last_active":"2024-04-25T19:19:45.004413+0000","last_peered":"2024-04-25T19:19:45.004413+0000","last_clean":"2024-04-25T19:19:45.004413+0000","last_became_active":"2024-04-25T19:15:18.425084+0000","last_became_peered":"2024-04-25T19:15:18.425084+0000","last_unstale":"2024-04-25T19:19:45.004413+0000","last_undegraded":"2024-04-25T19:19:45.004413+0000","last_fullsized":"2024-04-25T19:19:45.004413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3","last_scrub_stamp":"2024-04-25T19:19:45.004324+0000","last_deep_scrub":"23'3","last_deep_scrub_stamp":"2024-04-25T19:19:45.004324+0000","last_clean_scrub_stamp":"2024-04-25T19:19:45.004324+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:40.126780+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"23'134","reported_seq":161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:42.336585+0000","last_change":"2024-04-25T19:19:42.336585+0000","last_active":"2024-04-25T19:19:42.336585+0000","last_peered":"2024-04-25T19:19:42.336585+0000","last_clean":"2024-04-25T19:19:42.336585+0000","last_became_active":"2024-04-25T19:15:19.424847+0000","last_became_peered":"2024-04-25T19:15:19.424847+0000","last_unstale":"2024-04-25T19:19:42.336585+0000","last_undegraded":"2024-04-25T19:19:42.336585+0000","last_fullsized":"2024-04-25T19:19:42.336585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'134","last_scrub_stamp":"2024-04-25T19:19:42.336425+0000","last_deep_scrub":"23'134","last_deep_scrub_stamp":"2024-04-25T19:19:42.336425+0000","last_clean_scrub_stamp":"2024-04-25T19:19:42.336425+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:53:30.780172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"23'11","reported_seq":40,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:40.136696+0000","last_change":"2024-04-25T19:19:40.136749+0000","last_active":"2024-04-25T19:19:40.136696+0000","last_peered":"2024-04-25T19:19:40.136696+0000","last_clean":"2024-04-25T19:19:40.136696+0000","last_became_active":"2024-04-25T19:15:18.424241+0000","last_became_peered":"2024-04-25T19:15:18.424241+0000","last_unstale":"2024-04-25T19:19:40.136696+0000","last_undegraded":"2024-04-25T19:19:40.136696+0000","last_fullsized":"2024-04-25T19:19:40.136696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'11","last_scrub_stamp":"2024-04-25T19:19:40.136556+0000","last_deep_scrub":"23'11","last_deep_scrub_stamp":"2024-04-25T19:19:40.136556+0000","last_clean_scrub_stamp":"2024-04-25T19:19:40.136556+0000","objects_scrubbed":4,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:45.208291+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"23'153","reported_seq":180,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:45.975450+0000","last_change":"2024-04-25T19:19:45.975480+0000","last_active":"2024-04-25T19:19:45.975450+0000","last_peered":"2024-04-25T19:19:45.975450+0000","last_clean":"2024-04-25T19:19:45.975450+0000","last_became_active":"2024-04-25T19:15:19.420795+0000","last_became_peered":"2024-04-25T19:15:19.420795+0000","last_unstale":"2024-04-25T19:19:45.975450+0000","last_undegraded":"2024-04-25T19:19:45.975450+0000","last_fullsized":"2024-04-25T19:19:45.975450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'153","last_scrub_stamp":"2024-04-25T19:19:45.975356+0000","last_deep_scrub":"23'153","last_deep_scrub_stamp":"2024-04-25T19:19:45.975356+0000","last_clean_scrub_stamp":"2024-04-25T19:19:45.975356+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:13.251655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"23'5","reported_seq":34,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:39.116530+0000","last_change":"2024-04-25T19:19:39.116559+0000","last_active":"2024-04-25T19:19:39.116530+0000","last_peered":"2024-04-25T19:19:39.116530+0000","last_clean":"2024-04-25T19:19:39.116530+0000","last_became_active":"2024-04-25T19:15:18.423500+0000","last_became_peered":"2024-04-25T19:15:18.423500+0000","last_unstale":"2024-04-25T19:19:39.116530+0000","last_undegraded":"2024-04-25T19:19:39.116530+0000","last_fullsized":"2024-04-25T19:19:39.116530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5","last_scrub_stamp":"2024-04-25T19:19:39.116455+0000","last_deep_scrub":"23'5","last_deep_scrub_stamp":"2024-04-25T19:19:39.116455+0000","last_clean_scrub_stamp":"2024-04-25T19:19:39.116455+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:34.309231+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"23'143","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:41.910510+0000","last_change":"2024-04-25T19:19:41.910510+0000","last_active":"2024-04-25T19:19:41.910510+0000","last_peered":"2024-04-25T19:19:41.910510+0000","last_clean":"2024-04-25T19:19:41.910510+0000","last_became_active":"2024-04-25T19:15:19.415828+0000","last_became_peered":"2024-04-25T19:15:19.415828+0000","last_unstale":"2024-04-25T19:19:41.910510+0000","last_undegraded":"2024-04-25T19:19:41.910510+0000","last_fullsized":"2024-04-25T19:19:41.910510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'143","last_scrub_stamp":"2024-04-25T19:19:41.910379+0000","last_deep_scrub":"23'143","last_deep_scrub_stamp":"2024-04-25T19:19:41.910379+0000","last_clean_scrub_stamp":"2024-04-25T19:19:41.910379+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:47:02.255920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'7","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:43.019893+0000","last_change":"2024-04-25T19:19:43.019893+0000","last_active":"2024-04-25T19:19:43.019893+0000","last_peered":"2024-04-25T19:19:43.019893+0000","last_clean":"2024-04-25T19:19:43.019893+0000","last_became_active":"2024-04-25T19:15:18.423588+0000","last_became_peered":"2024-04-25T19:15:18.423588+0000","last_unstale":"2024-04-25T19:19:43.019893+0000","last_undegraded":"2024-04-25T19:19:43.019893+0000","last_fullsized":"2024-04-25T19:19:43.019893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7","last_scrub_stamp":"2024-04-25T19:19:43.019750+0000","last_deep_scrub":"23'7","last_deep_scrub_stamp":"2024-04-25T19:19:43.019750+0000","last_clean_scrub_stamp":"2024-04-25T19:19:43.019750+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:00:58.057453+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"23'128","reported_seq":155,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:48.414356+0000","last_change":"2024-04-25T19:19:48.414408+0000","last_active":"2024-04-25T19:19:48.414356+0000","last_peered":"2024-04-25T19:19:48.414356+0000","last_clean":"2024-04-25T19:19:48.414356+0000","last_became_active":"2024-04-25T19:15:19.422651+0000","last_became_peered":"2024-04-25T19:15:19.422651+0000","last_unstale":"2024-04-25T19:19:48.414356+0000","last_undegraded":"2024-04-25T19:19:48.414356+0000","last_fullsized":"2024-04-25T19:19:48.414356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'128","last_scrub_stamp":"2024-04-25T19:19:48.414236+0000","last_deep_scrub":"23'128","last_deep_scrub_stamp":"2024-04-25T19:19:48.414236+0000","last_clean_scrub_stamp":"2024-04-25T19:19:48.414236+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:07.612583+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"23'183","reported_seq":209,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:46.784602+0000","last_change":"2024-04-25T19:19:46.784637+0000","last_active":"2024-04-25T19:19:46.784602+0000","last_peered":"2024-04-25T19:19:46.784602+0000","last_clean":"2024-04-25T19:19:46.784602+0000","last_became_active":"2024-04-25T19:15:19.423341+0000","last_became_peered":"2024-04-25T19:15:19.423341+0000","last_unstale":"2024-04-25T19:19:46.784602+0000","last_undegraded":"2024-04-25T19:19:46.784602+0000","last_fullsized":"2024-04-25T19:19:46.784602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'183","last_scrub_stamp":"2024-04-25T19:19:46.784489+0000","last_deep_scrub":"23'183","last_deep_scrub_stamp":"2024-04-25T19:19:46.784489+0000","last_clean_scrub_stamp":"2024-04-25T19:19:46.784489+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:39:41.859172+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"23'8","reported_seq":36,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:44.026504+0000","last_change":"2024-04-25T19:19:44.026545+0000","last_active":"2024-04-25T19:19:44.026504+0000","last_peered":"2024-04-25T19:19:44.026504+0000","last_clean":"2024-04-25T19:19:44.026504+0000","last_became_active":"2024-04-25T19:15:18.424119+0000","last_became_peered":"2024-04-25T19:15:18.424119+0000","last_unstale":"2024-04-25T19:19:44.026504+0000","last_undegraded":"2024-04-25T19:19:44.026504+0000","last_fullsized":"2024-04-25T19:19:44.026504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8","last_scrub_stamp":"2024-04-25T19:19:44.026408+0000","last_deep_scrub":"23'8","last_deep_scrub_stamp":"2024-04-25T19:19:44.026408+0000","last_clean_scrub_stamp":"2024-04-25T19:19:44.026408+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:03:02.304666+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'141","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:47.755436+0000","last_change":"2024-04-25T19:19:47.755527+0000","last_active":"2024-04-25T19:19:47.755436+0000","last_peered":"2024-04-25T19:19:47.755436+0000","last_clean":"2024-04-25T19:19:47.755436+0000","last_became_active":"2024-04-25T19:15:19.423187+0000","last_became_peered":"2024-04-25T19:15:19.423187+0000","last_unstale":"2024-04-25T19:19:47.755436+0000","last_undegraded":"2024-04-25T19:19:47.755436+0000","last_fullsized":"2024-04-25T19:19:47.755436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'141","last_scrub_stamp":"2024-04-25T19:19:47.755311+0000","last_deep_scrub":"23'141","last_deep_scrub_stamp":"2024-04-25T19:19:47.755311+0000","last_clean_scrub_stamp":"2024-04-25T19:19:47.755311+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:57:30.427625+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"23'20","reported_seq":48,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:41.974105+0000","last_change":"2024-04-25T19:19:41.974151+0000","last_active":"2024-04-25T19:19:41.974105+0000","last_peered":"2024-04-25T19:19:41.974105+0000","last_clean":"2024-04-25T19:19:41.974105+0000","last_became_active":"2024-04-25T19:15:18.422831+0000","last_became_peered":"2024-04-25T19:15:18.422831+0000","last_unstale":"2024-04-25T19:19:41.974105+0000","last_undegraded":"2024-04-25T19:19:41.974105+0000","last_fullsized":"2024-04-25T19:19:41.974105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'20","last_scrub_stamp":"2024-04-25T19:19:41.974000+0000","last_deep_scrub":"23'20","last_deep_scrub_stamp":"2024-04-25T19:19:41.974000+0000","last_clean_scrub_stamp":"2024-04-25T19:19:41.974000+0000","objects_scrubbed":7,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:17:32.649735+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"23'135","reported_seq":162,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:40.760214+0000","last_change":"2024-04-25T19:19:40.760258+0000","last_active":"2024-04-25T19:19:40.760214+0000","last_peered":"2024-04-25T19:19:40.760214+0000","last_clean":"2024-04-25T19:19:40.760214+0000","last_became_active":"2024-04-25T19:15:19.416163+0000","last_became_peered":"2024-04-25T19:15:19.416163+0000","last_unstale":"2024-04-25T19:19:40.760214+0000","last_undegraded":"2024-04-25T19:19:40.760214+0000","last_fullsized":"2024-04-25T19:19:40.760214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'135","last_scrub_stamp":"2024-04-25T19:19:40.760108+0000","last_deep_scrub":"23'135","last_deep_scrub_stamp":"2024-04-25T19:19:40.760108+0000","last_clean_scrub_stamp":"2024-04-25T19:19:40.760108+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:24.813931+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:39.974202+0000","last_change":"2024-04-25T19:19:39.974226+0000","last_active":"2024-04-25T19:19:39.974202+0000","last_peered":"2024-04-25T19:19:39.974202+0000","last_clean":"2024-04-25T19:19:39.974202+0000","last_became_active":"2024-04-25T19:15:18.422489+0000","last_became_peered":"2024-04-25T19:15:18.422489+0000","last_unstale":"2024-04-25T19:19:39.974202+0000","last_undegraded":"2024-04-25T19:19:39.974202+0000","last_fullsized":"2024-04-25T19:19:39.974202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'2","last_scrub_stamp":"2024-04-25T19:19:39.974126+0000","last_deep_scrub":"21'2","last_deep_scrub_stamp":"2024-04-25T19:19:39.974126+0000","last_clean_scrub_stamp":"2024-04-25T19:19:39.974126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:36.228257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"23'152","reported_seq":179,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:42.929215+0000","last_change":"2024-04-25T19:19:42.929242+0000","last_active":"2024-04-25T19:19:42.929215+0000","last_peered":"2024-04-25T19:19:42.929215+0000","last_clean":"2024-04-25T19:19:42.929215+0000","last_became_active":"2024-04-25T19:15:19.414173+0000","last_became_peered":"2024-04-25T19:15:19.414173+0000","last_unstale":"2024-04-25T19:19:42.929215+0000","last_undegraded":"2024-04-25T19:19:42.929215+0000","last_fullsized":"2024-04-25T19:19:42.929215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'152","last_scrub_stamp":"2024-04-25T19:19:42.929114+0000","last_deep_scrub":"23'152","last_deep_scrub_stamp":"2024-04-25T19:19:42.929114+0000","last_clean_scrub_stamp":"2024-04-25T19:19:42.929114+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:22.285921+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"23'19","reported_seq":48,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-25T19:19:37.724482+0000","last_change":"2024-04-25T19:19:37.724514+0000","last_active":"2024-04-25T19:19:37.724482+0000","last_peered":"2024-04-25T19:19:37.724482+0000","last_clean":"2024-04-25T19:19:37.724482+0000","last_became_active":"2024-04-25T19:15:18.423532+0000","last_became_peered":"2024-04-25T19:15:18.423532+0000","last_unstale":"2024-04-25T19:19:37.724482+0000","last_undegraded":"2024-04-25T19:19:37.724482+0000","last_fullsized":"2024-04-25T19:19:37.724482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'19","last_scrub_stamp":"2024-04-25T19:19:37.724409+0000","last_deep_scrub":"23'19","last_deep_scrub_stamp":"2024-04-25T19:19:37.724409+0000","last_clean_scrub_stamp":"2024-04-25T19:19:37.724409+0000","objects_scrubbed":6,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:35.236962+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9717,"ondisk_log_size":9717,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":225467,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":353,"num_write_kb":342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25000,"num_omap_keys":50,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":696320,"data_stored":450934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":403,"ondisk_log_size":403,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64838,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39832,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332008,"statfs":{"total":96636764160,"available":96595976192,"internally_reserved":0,"allocated":344064,"data_stored":680845,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35808,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336032,"statfs":{"total":96636764160,"available":96600096768,"internally_reserved":0,"allocated":483328,"data_stored":305961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574908,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39780,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332060,"statfs":{"total":96636764160,"available":96596029440,"internally_reserved":0,"allocated":290816,"data_stored":92309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574908,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39756,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332084,"statfs":{"total":96636764160,"available":96596054016,"internally_reserved":0,"allocated":266240,"data_stored":89707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39976,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331864,"statfs":{"total":96636764160,"available":96595828736,"internally_reserved":0,"allocated":491520,"data_stored":306500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35608,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336232,"statfs":{"total":96636764160,"available":96600301568,"internally_reserved":0,"allocated":278528,"data_stored":90344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35600,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336240,"statfs":{"total":96636764160,"available":96600309760,"internally_reserved":0,"allocated":270336,"data_stored":89327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39824,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94332016,"statfs":{"total":96636764160,"available":96595984384,"internally_reserved":0,"allocated":335872,"data_stored":679037,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32419,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":249856,"data_stored":218705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":218166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T19:20:00.372 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T19:20:00.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.514+0000 7fbf4ce5a640 1 Processor -- start 2024-04-25T19:20:00.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.514+0000 7fbf4ce5a640 1 -- start start 2024-04-25T19:20:00.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.516+0000 7fbf4ce5a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf4806d120 0x7fbf4815e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:00.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.516+0000 7fbf4ce5a640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fbf4815e9c0 0x7fbf480c8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:00.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.516+0000 7fbf4ce5a640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf4806c7b0 con 0x7fbf480714c0 2024-04-25T19:20:00.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.516+0000 7fbf4ce5a640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fbf480cb490 con 0x7fbf4815e9c0 2024-04-25T19:20:00.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.516+0000 7fbf4ce5a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fbf480ca7f0 con 0x7fbf4806d120 2024-04-25T19:20:00.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.516+0000 7fbf4659c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fbf480714c0 0x7fbf48154a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33086/0 (socket says 172.21.15.76:33086) 2024-04-25T19:20:00.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.516+0000 7fbf4659c640 1 -- 172.21.15.76:0/3150217092 learned_addr learned my addr 172.21.15.76:0/3150217092 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:20:00.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.516+0000 7fbf46d9d640 1 --2- 172.21.15.76:0/3150217092 >> v2:172.21.15.105:3300/0 conn(0x7fbf4815e9c0 0x7fbf480c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:00.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.516+0000 7fbf45d9b640 1 --2- 172.21.15.76:0/3150217092 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf4806d120 0x7fbf4815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:00.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.516+0000 7fbf4559a640 1 -- 172.21.15.76:0/3150217092 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 559885356 0 0) 0x7fbf4806c7b0 con 0x7fbf480714c0 2024-04-25T19:20:00.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.516+0000 7fbf4559a640 1 -- 172.21.15.76:0/3150217092 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf30003520 con 0x7fbf480714c0 2024-04-25T19:20:00.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.516+0000 7fbf46d9d640 1 -- 172.21.15.76:0/3150217092 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf4806d120 msgr2=0x7fbf4815e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:00.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.516+0000 7fbf46d9d640 1 --2- 172.21.15.76:0/3150217092 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf4806d120 0x7fbf4815e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:00.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.516+0000 7fbf46d9d640 1 -- 172.21.15.76:0/3150217092 >> v1:172.21.15.76:6789/0 conn(0x7fbf480714c0 legacy=0x7fbf48154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:00.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.516+0000 7fbf46d9d640 1 -- 172.21.15.76:0/3150217092 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf480ca230 con 0x7fbf4815e9c0 2024-04-25T19:20:00.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.517+0000 7fbf46d9d640 1 --2- 172.21.15.76:0/3150217092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf4815e9c0 0x7fbf480c8440 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbf400097a0 tx=0x7fbf4002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fd969e47609f277 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:00.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.517+0000 7fbf4559a640 1 -- 172.21.15.76:0/3150217092 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf40037070 con 0x7fbf4815e9c0 2024-04-25T19:20:00.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.517+0000 7fbf4559a640 1 -- 172.21.15.76:0/3150217092 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf40032440 con 0x7fbf4815e9c0 2024-04-25T19:20:00.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.517+0000 7fbf4559a640 1 -- 172.21.15.76:0/3150217092 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf400327f0 con 0x7fbf4815e9c0 2024-04-25T19:20:00.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.518+0000 7fbf4ce5a640 1 -- 172.21.15.76:0/3150217092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf4815e9c0 msgr2=0x7fbf480c8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:00.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.518+0000 7fbf4ce5a640 1 --2- 172.21.15.76:0/3150217092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf4815e9c0 0x7fbf480c8440 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbf400097a0 tx=0x7fbf4002a210 comp rx=0 tx=0).stop 2024-04-25T19:20:00.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.518+0000 7fbf4ce5a640 1 -- 172.21.15.76:0/3150217092 shutdown_connections 2024-04-25T19:20:00.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.518+0000 7fbf4ce5a640 1 --2- 172.21.15.76:0/3150217092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf4815e9c0 0x7fbf480c8440 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:00.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.518+0000 7fbf4ce5a640 1 --2- 172.21.15.76:0/3150217092 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf4806d120 0x7fbf4815e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:00.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.518+0000 7fbf4ce5a640 1 -- 172.21.15.76:0/3150217092 >> 172.21.15.76:0/3150217092 conn(0x7fbf4806d6d0 msgr2=0x7fbf4806dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:00.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.518+0000 7fbf4ce5a640 1 -- 172.21.15.76:0/3150217092 shutdown_connections 2024-04-25T19:20:00.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.518+0000 7fbf4ce5a640 1 -- 172.21.15.76:0/3150217092 wait complete. 2024-04-25T19:20:00.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.520+0000 7fbf4ce5a640 1 Processor -- start 2024-04-25T19:20:00.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.520+0000 7fbf4ce5a640 1 -- start start 2024-04-25T19:20:00.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.521+0000 7fbf4ce5a640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf4806d120 0x7fbf481f4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:00.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.521+0000 7fbf4ce5a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf480714c0 0x7fbf48150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:00.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.521+0000 7fbf4ce5a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf4815e9c0 0x7fbf4814a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:00.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.521+0000 7fbf4ce5a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbf480724b0 con 0x7fbf480714c0 2024-04-25T19:20:00.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.521+0000 7fbf4ce5a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4815df60 con 0x7fbf4815e9c0 2024-04-25T19:20:00.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.521+0000 7fbf4ce5a640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fbf4806cb80 con 0x7fbf4806d120 2024-04-25T19:20:00.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.521+0000 7fbf45d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf480714c0 0x7fbf48150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:00.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.521+0000 7fbf45d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf480714c0 0x7fbf48150e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58146/0 (socket says 172.21.15.76:58146) 2024-04-25T19:20:00.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.521+0000 7fbf45d9b640 1 -- 172.21.15.76:0/3260190841 learned_addr learned my addr 172.21.15.76:0/3260190841 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:20:00.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.521+0000 7fbf46d9d640 1 --2- 172.21.15.76:0/3260190841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf4815e9c0 0x7fbf4814a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:00.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.521+0000 7fbf4659c640 1 --2- 172.21.15.76:0/3260190841 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf4806d120 0x7fbf481f4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:00.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.522+0000 7fbf45d9b640 1 -- 172.21.15.76:0/3260190841 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf4806d120 msgr2=0x7fbf481f4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:00.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.522+0000 7fbf45d9b640 1 --2- 172.21.15.76:0/3260190841 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf4806d120 0x7fbf481f4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:00.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.522+0000 7fbf45d9b640 1 -- 172.21.15.76:0/3260190841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf4815e9c0 msgr2=0x7fbf4814a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:00.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.522+0000 7fbf45d9b640 1 --2- 172.21.15.76:0/3260190841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf4815e9c0 0x7fbf4814a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:00.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.522+0000 7fbf45d9b640 1 -- 172.21.15.76:0/3260190841 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf481f4fb0 con 0x7fbf480714c0 2024-04-25T19:20:00.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.522+0000 7fbf46d9d640 1 --2- 172.21.15.76:0/3260190841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf4815e9c0 0x7fbf4814a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:20:00.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.522+0000 7fbf45d9b640 1 --2- 172.21.15.76:0/3260190841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf480714c0 0x7fbf48150e80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00e8a0 tx=0x7fbf3c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:00.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.522+0000 7fbf36ffd640 1 -- 172.21.15.76:0/3260190841 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c017070 con 0x7fbf480714c0 2024-04-25T19:20:00.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.522+0000 7fbf36ffd640 1 -- 172.21.15.76:0/3260190841 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf3c00c940 con 0x7fbf480714c0 2024-04-25T19:20:00.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.522+0000 7fbf4ce5a640 1 -- 172.21.15.76:0/3260190841 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf48150a70 con 0x7fbf480714c0 2024-04-25T19:20:00.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.522+0000 7fbf36ffd640 1 -- 172.21.15.76:0/3260190841 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c00cd30 con 0x7fbf480714c0 2024-04-25T19:20:00.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.522+0000 7fbf4ce5a640 1 -- 172.21.15.76:0/3260190841 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf481504b0 con 0x7fbf480714c0 2024-04-25T19:20:00.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.523+0000 7fbf4ce5a640 1 -- 172.21.15.76:0/3260190841 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf4806c7b0 con 0x7fbf480714c0 2024-04-25T19:20:00.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.524+0000 7fbf36ffd640 1 -- 172.21.15.76:0/3260190841 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbf3c01a070 con 0x7fbf480714c0 2024-04-25T19:20:00.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.524+0000 7fbf36ffd640 1 --2- 172.21.15.76:0/3260190841 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbf280744b0 0x7fbf280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:00.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.524+0000 7fbf36ffd640 1 -- 172.21.15.76:0/3260190841 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fbf3c095090 con 0x7fbf480714c0 2024-04-25T19:20:00.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.524+0000 7fbf4659c640 1 --2- 172.21.15.76:0/3260190841 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbf280744b0 0x7fbf280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:00.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.525+0000 7fbf4659c640 1 --2- 172.21.15.76:0/3260190841 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbf280744b0 0x7fbf280769c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbf38005c00 tx=0x7fbf38009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:00.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.529+0000 7fbf36ffd640 1 -- 172.21.15.76:0/3260190841 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbf4806c7b0 con 0x7fbf480714c0 2024-04-25T19:20:00.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.706+0000 7fbf4ce5a640 1 -- 172.21.15.76:0/3260190841 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fbf4806a390 con 0x7fbf480714c0 2024-04-25T19:20:00.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.710+0000 7fbf36ffd640 1 -- 172.21.15.76:0/3260190841 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) ==== 123+0+0 (secure 0 0 0) 0x7fbf3c05f8c0 con 0x7fbf480714c0 2024-04-25T19:20:00.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.711+0000 7fbf36ffd640 1 -- 172.21.15.76:0/3260190841 <== mon.0 v2:172.21.15.76:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fbf3c05ecd0 con 0x7fbf480714c0 2024-04-25T19:20:00.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.712+0000 7fbf4ce5a640 1 -- 172.21.15.76:0/3260190841 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbf280744b0 msgr2=0x7fbf280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:00.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.712+0000 7fbf4ce5a640 1 --2- 172.21.15.76:0/3260190841 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbf280744b0 0x7fbf280769c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbf38005c00 tx=0x7fbf38009040 comp rx=0 tx=0).stop 2024-04-25T19:20:00.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.712+0000 7fbf4ce5a640 1 -- 172.21.15.76:0/3260190841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf480714c0 msgr2=0x7fbf48150e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:00.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.712+0000 7fbf4ce5a640 1 --2- 172.21.15.76:0/3260190841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf480714c0 0x7fbf48150e80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00e8a0 tx=0x7fbf3c00ed60 comp rx=0 tx=0).stop 2024-04-25T19:20:00.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.712+0000 7fbf4ce5a640 1 -- 172.21.15.76:0/3260190841 shutdown_connections 2024-04-25T19:20:00.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.712+0000 7fbf4ce5a640 1 --2- 172.21.15.76:0/3260190841 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fbf280744b0 0x7fbf280769c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:00.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.712+0000 7fbf4ce5a640 1 --2- 172.21.15.76:0/3260190841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbf4815e9c0 0x7fbf4814a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:00.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.713+0000 7fbf4ce5a640 1 --2- 172.21.15.76:0/3260190841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf480714c0 0x7fbf48150e80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:00.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.713+0000 7fbf4ce5a640 1 --2- 172.21.15.76:0/3260190841 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fbf4806d120 0x7fbf481f4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:00.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.713+0000 7fbf4ce5a640 1 -- 172.21.15.76:0/3260190841 >> 172.21.15.76:0/3260190841 conn(0x7fbf4806d6d0 msgr2=0x7fbf48152560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:00.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.713+0000 7fbf4ce5a640 1 -- 172.21.15.76:0/3260190841 shutdown_connections 2024-04-25T19:20:00.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.713+0000 7fbf4ce5a640 1 -- 172.21.15.76:0/3260190841 wait complete. 2024-04-25T19:20:00.725 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T19:20:00.725 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:20:00.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.831+0000 7fcd510e0640 1 Processor -- start 2024-04-25T19:20:00.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.832+0000 7fcd510e0640 1 -- start start 2024-04-25T19:20:00.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.833+0000 7fcd510e0640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcd4c0cc840 0x7fcd4c0cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:00.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.833+0000 7fcd510e0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd4c06e0e0 0x7fcd4c06e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:00.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.833+0000 7fcd510e0640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd4c0cb830 con 0x7fcd4c0cab50 2024-04-25T19:20:00.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.833+0000 7fcd510e0640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7fcd4c0cbb10 con 0x7fcd4c0cc840 2024-04-25T19:20:00.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.833+0000 7fcd510e0640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fcd4c0ce080 con 0x7fcd4c06e0e0 2024-04-25T19:20:00.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.833+0000 7fcd4ad9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcd4c0cc840 0x7fcd4c0cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:00.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.833+0000 7fcd4b59e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd4c06e0e0 0x7fcd4c06e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:00.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.833+0000 7fcd4ad9d640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fcd4c0cc840 0x7fcd4c0cc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.76:60838/0 (socket says 172.21.15.76:60838) 2024-04-25T19:20:00.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.833+0000 7fcd4ad9d640 1 -- 172.21.15.76:0/2608548087 learned_addr learned my addr 172.21.15.76:0/2608548087 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:20:00.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.833+0000 7fcd4b59e640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd4c06e0e0 0x7fcd4c06e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:42812/0 (socket says 172.21.15.76:42812) 2024-04-25T19:20:00.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.833+0000 7fcd4ad9d640 1 -- 172.21.15.76:0/2608548087 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd4c06e0e0 msgr2=0x7fcd4c06e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:00.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.833+0000 7fcd4ad9d640 1 --2- 172.21.15.76:0/2608548087 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd4c06e0e0 0x7fcd4c06e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:00.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.833+0000 7fcd4ad9d640 1 -- 172.21.15.76:0/2608548087 >> v1:172.21.15.105:6789/0 conn(0x7fcd4c0cab50 legacy=0x7fcd4c0ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:00.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.833+0000 7fcd4ad9d640 1 -- 172.21.15.76:0/2608548087 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd4c0ce360 con 0x7fcd4c0cc840 2024-04-25T19:20:00.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.833+0000 7fcd4b59e640 1 --2- 172.21.15.76:0/2608548087 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd4c06e0e0 0x7fcd4c06e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:20:00.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.834+0000 7fcd4ad9d640 1 --2- 172.21.15.76:0/2608548087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd4c0cc840 0x7fcd4c0cc1d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c009870 tx=0x7fcd3c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e84fb05a6e4b0a17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:00.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.834+0000 7fcd49d9b640 1 -- 172.21.15.76:0/2608548087 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c037070 con 0x7fcd4c0cc840 2024-04-25T19:20:00.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.834+0000 7fcd49d9b640 1 -- 172.21.15.76:0/2608548087 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fcd3c032440 con 0x7fcd4c0cc840 2024-04-25T19:20:00.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.834+0000 7fcd49d9b640 1 -- 172.21.15.76:0/2608548087 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c032830 con 0x7fcd4c0cc840 2024-04-25T19:20:00.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.835+0000 7fcd510e0640 1 -- 172.21.15.76:0/2608548087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd4c0cc840 msgr2=0x7fcd4c0cc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:00.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.835+0000 7fcd510e0640 1 --2- 172.21.15.76:0/2608548087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd4c0cc840 0x7fcd4c0cc1d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c009870 tx=0x7fcd3c02a270 comp rx=0 tx=0).stop 2024-04-25T19:20:00.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.835+0000 7fcd510e0640 1 -- 172.21.15.76:0/2608548087 shutdown_connections 2024-04-25T19:20:00.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.835+0000 7fcd510e0640 1 --2- 172.21.15.76:0/2608548087 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd4c06e0e0 0x7fcd4c06e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:00.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.835+0000 7fcd510e0640 1 --2- 172.21.15.76:0/2608548087 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd4c0cc840 0x7fcd4c0cc1d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:00.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.835+0000 7fcd510e0640 1 -- 172.21.15.76:0/2608548087 >> 172.21.15.76:0/2608548087 conn(0x7fcd4c072b60 msgr2=0x7fcd4c072f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:00.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.835+0000 7fcd510e0640 1 -- 172.21.15.76:0/2608548087 shutdown_connections 2024-04-25T19:20:00.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.835+0000 7fcd510e0640 1 -- 172.21.15.76:0/2608548087 wait complete. 2024-04-25T19:20:00.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.837+0000 7fcd510e0640 1 Processor -- start 2024-04-25T19:20:00.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.837+0000 7fcd510e0640 1 -- start start 2024-04-25T19:20:00.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.838+0000 7fcd510e0640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd4c06e0e0 0x7fcd4c1524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:00.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.838+0000 7fcd510e0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd4c0cab50 0x7fcd4c1f4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:00.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.838+0000 7fcd510e0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd4c0cc840 0x7fcd4c14d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:00.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.838+0000 7fcd510e0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c0cb0e0 con 0x7fcd4c0cab50 2024-04-25T19:20:00.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.838+0000 7fcd510e0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c0caf60 con 0x7fcd4c0cc840 2024-04-25T19:20:00.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.838+0000 7fcd510e0640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fcd4c160050 con 0x7fcd4c06e0e0 2024-04-25T19:20:00.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.838+0000 7fcd4a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd4c0cab50 0x7fcd4c1f4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:00.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.838+0000 7fcd4a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd4c0cab50 0x7fcd4c1f4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58154/0 (socket says 172.21.15.76:58154) 2024-04-25T19:20:00.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.838+0000 7fcd4a59c640 1 -- 172.21.15.76:0/2491935854 learned_addr learned my addr 172.21.15.76:0/2491935854 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:20:00.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.839+0000 7fcd4b59e640 1 --2- 172.21.15.76:0/2491935854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd4c0cc840 0x7fcd4c14d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:00.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.839+0000 7fcd4ad9d640 1 --2- 172.21.15.76:0/2491935854 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd4c06e0e0 0x7fcd4c1524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:00.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.839+0000 7fcd4a59c640 1 -- 172.21.15.76:0/2491935854 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd4c06e0e0 msgr2=0x7fcd4c1524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:00.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.839+0000 7fcd4a59c640 1 --2- 172.21.15.76:0/2491935854 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd4c06e0e0 0x7fcd4c1524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:00.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.839+0000 7fcd4a59c640 1 -- 172.21.15.76:0/2491935854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd4c0cc840 msgr2=0x7fcd4c14d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:00.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.839+0000 7fcd4a59c640 1 --2- 172.21.15.76:0/2491935854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd4c0cc840 0x7fcd4c14d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:00.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.839+0000 7fcd4a59c640 1 -- 172.21.15.76:0/2491935854 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd4c0b42f0 con 0x7fcd4c0cab50 2024-04-25T19:20:00.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.839+0000 7fcd4a59c640 1 --2- 172.21.15.76:0/2491935854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd4c0cab50 0x7fcd4c1f4120 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000c3f0 tx=0x7fcd4000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:00.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.839+0000 7fcd337fe640 1 -- 172.21.15.76:0/2491935854 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd400052d0 con 0x7fcd4c0cab50 2024-04-25T19:20:00.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.839+0000 7fcd337fe640 1 -- 172.21.15.76:0/2491935854 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fcd40005490 con 0x7fcd4c0cab50 2024-04-25T19:20:00.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.839+0000 7fcd510e0640 1 -- 172.21.15.76:0/2491935854 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd4c1f3660 con 0x7fcd4c0cab50 2024-04-25T19:20:00.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.839+0000 7fcd337fe640 1 -- 172.21.15.76:0/2491935854 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40007280 con 0x7fcd4c0cab50 2024-04-25T19:20:00.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.839+0000 7fcd510e0640 1 -- 172.21.15.76:0/2491935854 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd4c153120 con 0x7fcd4c0cab50 2024-04-25T19:20:00.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.840+0000 7fcd510e0640 1 -- 172.21.15.76:0/2491935854 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd4c153320 con 0x7fcd4c0cab50 2024-04-25T19:20:00.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.841+0000 7fcd337fe640 1 -- 172.21.15.76:0/2491935854 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcd40007420 con 0x7fcd4c0cab50 2024-04-25T19:20:00.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.841+0000 7fcd337fe640 1 --2- 172.21.15.76:0/2491935854 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fcd380744b0 0x7fcd380769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:00.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.841+0000 7fcd337fe640 1 -- 172.21.15.76:0/2491935854 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fcd400947f0 con 0x7fcd4c0cab50 2024-04-25T19:20:00.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.842+0000 7fcd4ad9d640 1 --2- 172.21.15.76:0/2491935854 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fcd380744b0 0x7fcd380769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:00.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.842+0000 7fcd4ad9d640 1 --2- 172.21.15.76:0/2491935854 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fcd380744b0 0x7fcd380769c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c034000 tx=0x7fcd3c02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:00.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:00.846+0000 7fcd337fe640 1 -- 172.21.15.76:0/2491935854 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd4005ee80 con 0x7fcd4c0cab50 2024-04-25T19:20:01.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.058+0000 7fcd510e0640 1 -- 172.21.15.76:0/2491935854 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd4c06a390 con 0x7fcd4c0cab50 2024-04-25T19:20:01.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.058+0000 7fcd337fe640 1 -- 172.21.15.76:0/2491935854 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6915 (secure 0 0 0) 0x7fcd4005e780 con 0x7fcd4c0cab50 2024-04-25T19:20:01.060 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:20:01.060 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/3003966182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3003966182},{"type":"v1","addr":"172.21.15.76:6836","nonce":3003966182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T19:15:18.769449+0000","modified":"2024-04-25T19:15:48.500543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4284,"mds_1":4278,"mds_2":4241,"mds_3":4238,"mds_4":4235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4235":{"gid":4235,"name":"b","rank":4,"incarnation":15,"state":"up:active","state_seq":10,"addr":"172.21.15.105:6836/1809877606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1809877606},{"type":"v1","addr":"172.21.15.105:6836","nonce":1809877606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4238":{"gid":4238,"name":"f","rank":3,"incarnation":13,"state":"up:active","state_seq":9,"addr":"172.21.15.105:6838/2724142282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2724142282},{"type":"v1","addr":"172.21.15.105:6838","nonce":2724142282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4241":{"gid":4241,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.105:6839/1798232582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6837","nonce":1798232582},{"type":"v1","addr":"172.21.15.105:6839","nonce":1798232582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.76:6833/2101494685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2101494685},{"type":"v1","addr":"172.21.15.76:6833","nonce":2101494685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4284":{"gid":4284,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6837/4226277412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":4226277412},{"type":"v1","addr":"172.21.15.76:6837","nonce":4226277412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4235,4238,4241,4278,4284]},"id":1}]} 2024-04-25T19:20:01.060 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 16 2024-04-25T19:20:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.060+0000 7fcd510e0640 1 -- 172.21.15.76:0/2491935854 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fcd380744b0 msgr2=0x7fcd380769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:01.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.060+0000 7fcd510e0640 1 --2- 172.21.15.76:0/2491935854 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fcd380744b0 0x7fcd380769c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c034000 tx=0x7fcd3c02e000 comp rx=0 tx=0).stop 2024-04-25T19:20:01.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.060+0000 7fcd510e0640 1 -- 172.21.15.76:0/2491935854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd4c0cab50 msgr2=0x7fcd4c1f4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:01.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.060+0000 7fcd510e0640 1 --2- 172.21.15.76:0/2491935854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd4c0cab50 0x7fcd4c1f4120 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000c3f0 tx=0x7fcd4000c8b0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.060+0000 7fcd510e0640 1 -- 172.21.15.76:0/2491935854 shutdown_connections 2024-04-25T19:20:01.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.060+0000 7fcd510e0640 1 --2- 172.21.15.76:0/2491935854 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fcd380744b0 0x7fcd380769c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.060+0000 7fcd510e0640 1 --2- 172.21.15.76:0/2491935854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcd4c0cc840 0x7fcd4c14d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.060+0000 7fcd510e0640 1 --2- 172.21.15.76:0/2491935854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd4c0cab50 0x7fcd4c1f4120 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.060+0000 7fcd510e0640 1 --2- 172.21.15.76:0/2491935854 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fcd4c06e0e0 0x7fcd4c1524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.060+0000 7fcd510e0640 1 -- 172.21.15.76:0/2491935854 >> 172.21.15.76:0/2491935854 conn(0x7fcd4c072b60 msgr2=0x7fcd4c155f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:01.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.060+0000 7fcd510e0640 1 -- 172.21.15.76:0/2491935854 shutdown_connections 2024-04-25T19:20:01.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.060+0000 7fcd510e0640 1 -- 172.21.15.76:0/2491935854 wait complete. 2024-04-25T19:20:01.073 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T19:20:01.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.180+0000 7f00d6bc5640 1 Processor -- start 2024-04-25T19:20:01.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.180+0000 7f00d6bc5640 1 -- start start 2024-04-25T19:20:01.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.181+0000 7f00d6bc5640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f00d00d0810 0x7f00d006f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:01.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.181+0000 7f00d6bc5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f00d006e870 0x7f00d00cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:01.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.181+0000 7f00d6bc5640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00d015ce80 con 0x7f00d00cea70 2024-04-25T19:20:01.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.181+0000 7f00d6bc5640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7f00d015d160 con 0x7f00d00d0810 2024-04-25T19:20:01.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.181+0000 7f00d6bc5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f00d015d440 con 0x7f00d006e870 2024-04-25T19:20:01.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.181+0000 7f00d5162640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f00d006e870 0x7f00d00cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:01.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.181+0000 7f00d4961640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f00d00d0810 0x7f00d006f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:01.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.181+0000 7f00d5162640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f00d006e870 0x7f00d00cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3301/0 says I am v2:172.21.15.76:42830/0 (socket says 172.21.15.76:42830) 2024-04-25T19:20:01.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.181+0000 7f00d5162640 1 -- 172.21.15.76:0/687155081 learned_addr learned my addr 172.21.15.76:0/687155081 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:20:01.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.182+0000 7f00d4961640 1 -- 172.21.15.76:0/687155081 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f00d006e870 msgr2=0x7f00d00cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:01.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.182+0000 7f00d4961640 1 --2- 172.21.15.76:0/687155081 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f00d006e870 0x7f00d00cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.182+0000 7f00d4961640 1 -- 172.21.15.76:0/687155081 >> v1:172.21.15.105:6789/0 conn(0x7f00d00cea70 legacy=0x7f00d006e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:01.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.182+0000 7f00d4961640 1 -- 172.21.15.76:0/687155081 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d015d720 con 0x7f00d00d0810 2024-04-25T19:20:01.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.182+0000 7f00d5162640 1 --2- 172.21.15.76:0/687155081 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f00d006e870 0x7f00d00cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:20:01.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.182+0000 7f00d4961640 1 --2- 172.21.15.76:0/687155081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00d00d0810 0x7f00d006f000 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f00b80097a0 tx=0x7f00b802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a208681323c88fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:01.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.183+0000 7f00cf7fe640 1 -- 172.21.15.76:0/687155081 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b8037070 con 0x7f00d00d0810 2024-04-25T19:20:01.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.183+0000 7f00cf7fe640 1 -- 172.21.15.76:0/687155081 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f00b8032440 con 0x7f00d00d0810 2024-04-25T19:20:01.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.183+0000 7f00cf7fe640 1 -- 172.21.15.76:0/687155081 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b8032830 con 0x7f00d00d0810 2024-04-25T19:20:01.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.183+0000 7f00d6bc5640 1 -- 172.21.15.76:0/687155081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00d00d0810 msgr2=0x7f00d006f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:01.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.183+0000 7f00d6bc5640 1 --2- 172.21.15.76:0/687155081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00d00d0810 0x7f00d006f000 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f00b80097a0 tx=0x7f00b802a180 comp rx=0 tx=0).stop 2024-04-25T19:20:01.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.183+0000 7f00d6bc5640 1 -- 172.21.15.76:0/687155081 shutdown_connections 2024-04-25T19:20:01.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.183+0000 7f00d6bc5640 1 --2- 172.21.15.76:0/687155081 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f00d006e870 0x7f00d00cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.183+0000 7f00d6bc5640 1 --2- 172.21.15.76:0/687155081 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00d00d0810 0x7f00d006f000 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.183+0000 7f00d6bc5640 1 -- 172.21.15.76:0/687155081 >> 172.21.15.76:0/687155081 conn(0x7f00d00b3c00 msgr2=0x7f00d00b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:01.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.183+0000 7f00d6bc5640 1 -- 172.21.15.76:0/687155081 shutdown_connections 2024-04-25T19:20:01.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.183+0000 7f00d6bc5640 1 -- 172.21.15.76:0/687155081 wait complete. 2024-04-25T19:20:01.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.185+0000 7f00d6bc5640 1 Processor -- start 2024-04-25T19:20:01.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.185+0000 7f00d6bc5640 1 -- start start 2024-04-25T19:20:01.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.186+0000 7f00d6bc5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00d006e870 0x7f00d01f3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:01.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.186+0000 7f00d6bc5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00d00cea70 0x7f00d0157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:01.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.186+0000 7f00d6bc5640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f00d00d0810 0x7f00d01607a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:01.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.186+0000 7f00d6bc5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f00d015cb70 con 0x7f00d00cea70 2024-04-25T19:20:01.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.186+0000 7f00d6bc5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f00d006dc40 con 0x7f00d006e870 2024-04-25T19:20:01.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.186+0000 7f00d6bc5640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f00d006f540 con 0x7f00d00d0810 2024-04-25T19:20:01.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.186+0000 7f00cffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00d00cea70 0x7f00d0157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:01.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.186+0000 7f00cffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00d00cea70 0x7f00d0157410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58158/0 (socket says 172.21.15.76:58158) 2024-04-25T19:20:01.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.186+0000 7f00cffff640 1 -- 172.21.15.76:0/1286197855 learned_addr learned my addr 172.21.15.76:0/1286197855 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:20:01.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.187+0000 7f00d4961640 1 --2- 172.21.15.76:0/1286197855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00d006e870 0x7f00d01f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:01.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.187+0000 7f00d5162640 1 --2- 172.21.15.76:0/1286197855 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f00d00d0810 0x7f00d01607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:01.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.187+0000 7f00cffff640 1 -- 172.21.15.76:0/1286197855 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f00d00d0810 msgr2=0x7f00d01607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:01.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.187+0000 7f00cffff640 1 --2- 172.21.15.76:0/1286197855 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f00d00d0810 0x7f00d01607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.187+0000 7f00cffff640 1 -- 172.21.15.76:0/1286197855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00d006e870 msgr2=0x7f00d01f3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:01.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.187+0000 7f00cffff640 1 --2- 172.21.15.76:0/1286197855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00d006e870 0x7f00d01f3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.187+0000 7f00cffff640 1 -- 172.21.15.76:0/1286197855 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d01f36a0 con 0x7f00d00cea70 2024-04-25T19:20:01.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.187+0000 7f00d4961640 1 --2- 172.21.15.76:0/1286197855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00d006e870 0x7f00d01f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:20:01.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.187+0000 7f00cffff640 1 --2- 172.21.15.76:0/1286197855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00d00cea70 0x7f00d0157410 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f00c000c3c0 tx=0x7f00c000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:01.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.187+0000 7f00cd7fa640 1 -- 172.21.15.76:0/1286197855 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0017070 con 0x7f00d00cea70 2024-04-25T19:20:01.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.187+0000 7f00cd7fa640 1 -- 172.21.15.76:0/1286197855 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f00c00052f0 con 0x7f00d00cea70 2024-04-25T19:20:01.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.187+0000 7f00cd7fa640 1 -- 172.21.15.76:0/1286197855 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0003850 con 0x7f00d00cea70 2024-04-25T19:20:01.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.187+0000 7f00d6bc5640 1 -- 172.21.15.76:0/1286197855 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00d015e470 con 0x7f00d00cea70 2024-04-25T19:20:01.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.187+0000 7f00d6bc5640 1 -- 172.21.15.76:0/1286197855 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00d015ea30 con 0x7f00d00cea70 2024-04-25T19:20:01.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.188+0000 7f00d6bc5640 1 -- 172.21.15.76:0/1286197855 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00d006a390 con 0x7f00d00cea70 2024-04-25T19:20:01.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.189+0000 7f00cd7fa640 1 -- 172.21.15.76:0/1286197855 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f00c001a070 con 0x7f00d00cea70 2024-04-25T19:20:01.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.189+0000 7f00cd7fa640 1 --2- 172.21.15.76:0/1286197855 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f00b00744e0 0x7f00b00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:01.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.189+0000 7f00cd7fa640 1 -- 172.21.15.76:0/1286197855 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f00c0094560 con 0x7f00d00cea70 2024-04-25T19:20:01.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.190+0000 7f00d4961640 1 --2- 172.21.15.76:0/1286197855 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f00b00744e0 0x7f00b00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:01.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.190+0000 7f00d4961640 1 --2- 172.21.15.76:0/1286197855 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f00b00744e0 0x7f00b00769f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f00b8034000 tx=0x7f00b802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:01.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.194+0000 7f00cd7fa640 1 -- 172.21.15.76:0/1286197855 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f00c005e620 con 0x7f00d00cea70 2024-04-25T19:20:01.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.406+0000 7f00d6bc5640 1 -- 172.21.15.76:0/1286197855 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00d015c920 con 0x7f00d00cea70 2024-04-25T19:20:01.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.406+0000 7f00cd7fa640 1 -- 172.21.15.76:0/1286197855 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6915 (secure 0 0 0) 0x7f00c0062920 con 0x7f00d00cea70 2024-04-25T19:20:01.408 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:20:01.408 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/3003966182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3003966182},{"type":"v1","addr":"172.21.15.76:6836","nonce":3003966182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T19:15:18.769449+0000","modified":"2024-04-25T19:15:48.500543+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4284,"mds_1":4278,"mds_2":4241,"mds_3":4238,"mds_4":4235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4235":{"gid":4235,"name":"b","rank":4,"incarnation":15,"state":"up:active","state_seq":10,"addr":"172.21.15.105:6836/1809877606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1809877606},{"type":"v1","addr":"172.21.15.105:6836","nonce":1809877606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4238":{"gid":4238,"name":"f","rank":3,"incarnation":13,"state":"up:active","state_seq":9,"addr":"172.21.15.105:6838/2724142282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6835","nonce":2724142282},{"type":"v1","addr":"172.21.15.105:6838","nonce":2724142282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4241":{"gid":4241,"name":"d","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.105:6839/1798232582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6837","nonce":1798232582},{"type":"v1","addr":"172.21.15.105:6839","nonce":1798232582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"a","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.76:6833/2101494685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":2101494685},{"type":"v1","addr":"172.21.15.76:6833","nonce":2101494685}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4284":{"gid":4284,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6837/4226277412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":4226277412},{"type":"v1","addr":"172.21.15.76:6837","nonce":4226277412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4235,4238,4241,4278,4284]},"id":1}]} 2024-04-25T19:20:01.408 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 16 2024-04-25T19:20:01.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.409+0000 7f00d6bc5640 1 -- 172.21.15.76:0/1286197855 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f00b00744e0 msgr2=0x7f00b00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:01.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.409+0000 7f00d6bc5640 1 --2- 172.21.15.76:0/1286197855 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f00b00744e0 0x7f00b00769f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f00b8034000 tx=0x7f00b802e000 comp rx=0 tx=0).stop 2024-04-25T19:20:01.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.409+0000 7f00d6bc5640 1 -- 172.21.15.76:0/1286197855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00d00cea70 msgr2=0x7f00d0157410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:01.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.409+0000 7f00d6bc5640 1 --2- 172.21.15.76:0/1286197855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00d00cea70 0x7f00d0157410 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f00c000c3c0 tx=0x7f00c000c880 comp rx=0 tx=0).stop 2024-04-25T19:20:01.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.409+0000 7f00d6bc5640 1 -- 172.21.15.76:0/1286197855 shutdown_connections 2024-04-25T19:20:01.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.409+0000 7f00d6bc5640 1 --2- 172.21.15.76:0/1286197855 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f00b00744e0 0x7f00b00769f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.409+0000 7f00d6bc5640 1 --2- 172.21.15.76:0/1286197855 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f00d00d0810 0x7f00d01607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.409+0000 7f00d6bc5640 1 --2- 172.21.15.76:0/1286197855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00d00cea70 0x7f00d0157410 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.409+0000 7f00d6bc5640 1 --2- 172.21.15.76:0/1286197855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00d006e870 0x7f00d01f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.409+0000 7f00d6bc5640 1 -- 172.21.15.76:0/1286197855 >> 172.21.15.76:0/1286197855 conn(0x7f00d00b3c00 msgr2=0x7f00d015b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:01.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.409+0000 7f00d6bc5640 1 -- 172.21.15.76:0/1286197855 shutdown_connections 2024-04-25T19:20:01.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.409+0000 7f00d6bc5640 1 -- 172.21.15.76:0/1286197855 wait complete. 2024-04-25T19:20:01.422 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:20:01.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.529+0000 7efe4a815640 1 Processor -- start 2024-04-25T19:20:01.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.529+0000 7efe4a815640 1 -- start start 2024-04-25T19:20:01.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.530+0000 7efe4a815640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe440714c0 0x7efe4406c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:01.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.530+0000 7efe4a815640 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7efe440c9730 0x7efe4406cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:01.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.530+0000 7efe4a815640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7efe44156bf0 0x7efe440c7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:01.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.530+0000 7efe4a815640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7efe4406d160 con 0x7efe44156bf0 2024-04-25T19:20:01.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.530+0000 7efe4a815640 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 -- 0x7efe4415d660 con 0x7efe440c9730 2024-04-25T19:20:01.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.530+0000 7efe4a815640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7efe44071fa0 con 0x7efe440714c0 2024-04-25T19:20:01.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.530+0000 7efe48db2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7efe44156bf0 0x7efe440c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:01.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.530+0000 7efe48db2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7efe44156bf0 0x7efe440c7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58172/0 (socket says 172.21.15.76:58172) 2024-04-25T19:20:01.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.530+0000 7efe48db2640 1 -- 172.21.15.76:0/2230491041 learned_addr learned my addr 172.21.15.76:0/2230491041 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:20:01.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.531+0000 7efe437fe640 1 --2- 172.21.15.76:0/2230491041 >> v2:172.21.15.105:3300/0 conn(0x7efe440c9730 0x7efe4406cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:01.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.531+0000 7efe43fff640 1 --2- 172.21.15.76:0/2230491041 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe440714c0 0x7efe4406c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:01.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.531+0000 7efe48db2640 1 -- 172.21.15.76:0/2230491041 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe440714c0 msgr2=0x7efe4406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:01.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.531+0000 7efe48db2640 1 --2- 172.21.15.76:0/2230491041 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe440714c0 0x7efe4406c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.531+0000 7efe48db2640 1 -- 172.21.15.76:0/2230491041 >> v2:172.21.15.105:3300/0 conn(0x7efe440c9730 msgr2=0x7efe4406cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:01.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.531+0000 7efe48db2640 1 --2- 172.21.15.76:0/2230491041 >> v2:172.21.15.105:3300/0 conn(0x7efe440c9730 0x7efe4406cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.531+0000 7efe48db2640 1 -- 172.21.15.76:0/2230491041 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4415d940 con 0x7efe44156bf0 2024-04-25T19:20:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.531+0000 7efe48db2640 1 --2- 172.21.15.76:0/2230491041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe44156bf0 0x7efe440c7710 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efe4415dbd0 tx=0x7efe3c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae16de14067a91a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.532+0000 7efe42ffd640 1 -- 172.21.15.76:0/2230491041 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c036070 con 0x7efe44156bf0 2024-04-25T19:20:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.532+0000 7efe42ffd640 1 -- 172.21.15.76:0/2230491041 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7efe3c031600 con 0x7efe44156bf0 2024-04-25T19:20:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.532+0000 7efe42ffd640 1 -- 172.21.15.76:0/2230491041 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c0319f0 con 0x7efe44156bf0 2024-04-25T19:20:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.532+0000 7efe4a815640 1 -- 172.21.15.76:0/2230491041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe44156bf0 msgr2=0x7efe440c7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.532+0000 7efe4a815640 1 --2- 172.21.15.76:0/2230491041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe44156bf0 0x7efe440c7710 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efe4415dbd0 tx=0x7efe3c02a680 comp rx=0 tx=0).stop 2024-04-25T19:20:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.532+0000 7efe4a815640 1 -- 172.21.15.76:0/2230491041 shutdown_connections 2024-04-25T19:20:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.532+0000 7efe4a815640 1 --2- 172.21.15.76:0/2230491041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe44156bf0 0x7efe440c7710 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.532+0000 7efe4a815640 1 --2- 172.21.15.76:0/2230491041 >> v2:172.21.15.105:3300/0 conn(0x7efe440c9730 0x7efe4406cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.532+0000 7efe4a815640 1 --2- 172.21.15.76:0/2230491041 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe440714c0 0x7efe4406c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.532+0000 7efe4a815640 1 -- 172.21.15.76:0/2230491041 >> 172.21.15.76:0/2230491041 conn(0x7efe4406d6d0 msgr2=0x7efe4406dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.532+0000 7efe4a815640 1 -- 172.21.15.76:0/2230491041 shutdown_connections 2024-04-25T19:20:01.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.532+0000 7efe4a815640 1 -- 172.21.15.76:0/2230491041 wait complete. 2024-04-25T19:20:01.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.534+0000 7efe4a815640 1 Processor -- start 2024-04-25T19:20:01.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.534+0000 7efe4a815640 1 -- start start 2024-04-25T19:20:01.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.535+0000 7efe4a815640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe440c9730 0x7efe441f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:01.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.536+0000 7efe4a815640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe44156bf0 0x7efe440ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:01.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.536+0000 7efe4a815640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe4415a840 0x7efe44147480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:01.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.536+0000 7efe4a815640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efe440cc6f0 con 0x7efe4415a840 2024-04-25T19:20:01.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.536+0000 7efe4a815640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7efe440cc570 con 0x7efe440c9730 2024-04-25T19:20:01.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.536+0000 7efe4a815640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7efe440c7c50 con 0x7efe44156bf0 2024-04-25T19:20:01.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.536+0000 7efe48db2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe4415a840 0x7efe44147480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:01.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.536+0000 7efe48db2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe4415a840 0x7efe44147480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58178/0 (socket says 172.21.15.76:58178) 2024-04-25T19:20:01.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.536+0000 7efe48db2640 1 -- 172.21.15.76:0/856207784 learned_addr learned my addr 172.21.15.76:0/856207784 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:20:01.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.536+0000 7efe43fff640 1 --2- 172.21.15.76:0/856207784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe440c9730 0x7efe441f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:01.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.536+0000 7efe437fe640 1 --2- 172.21.15.76:0/856207784 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe44156bf0 0x7efe440ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:01.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.536+0000 7efe48db2640 1 -- 172.21.15.76:0/856207784 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe44156bf0 msgr2=0x7efe440ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:01.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.536+0000 7efe48db2640 1 --2- 172.21.15.76:0/856207784 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe44156bf0 0x7efe440ce0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.536+0000 7efe48db2640 1 -- 172.21.15.76:0/856207784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe440c9730 msgr2=0x7efe441f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:01.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.536+0000 7efe48db2640 1 --2- 172.21.15.76:0/856207784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe440c9730 0x7efe441f5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.536+0000 7efe48db2640 1 -- 172.21.15.76:0/856207784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe441f6070 con 0x7efe4415a840 2024-04-25T19:20:01.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.536+0000 7efe48db2640 1 --2- 172.21.15.76:0/856207784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe4415a840 0x7efe44147480 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efe3c034040 tx=0x7efe3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:01.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.536+0000 7efe40ff9640 1 -- 172.21.15.76:0/856207784 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c036050 con 0x7efe4415a840 2024-04-25T19:20:01.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.536+0000 7efe40ff9640 1 -- 172.21.15.76:0/856207784 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7efe3c042070 con 0x7efe4415a840 2024-04-25T19:20:01.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.536+0000 7efe4a815640 1 -- 172.21.15.76:0/856207784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe441f34d0 con 0x7efe4415a840 2024-04-25T19:20:01.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.536+0000 7efe40ff9640 1 -- 172.21.15.76:0/856207784 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c009510 con 0x7efe4415a840 2024-04-25T19:20:01.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.537+0000 7efe4a815640 1 -- 172.21.15.76:0/856207784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe4414d0f0 con 0x7efe4415a840 2024-04-25T19:20:01.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.537+0000 7efe4a815640 1 -- 172.21.15.76:0/856207784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe4406a390 con 0x7efe4415a840 2024-04-25T19:20:01.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.538+0000 7efe40ff9640 1 -- 172.21.15.76:0/856207784 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efe3c007bf0 con 0x7efe4415a840 2024-04-25T19:20:01.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.539+0000 7efe40ff9640 1 --2- 172.21.15.76:0/856207784 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7efe300744b0 0x7efe300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:01.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.539+0000 7efe40ff9640 1 -- 172.21.15.76:0/856207784 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7efe3c0b6180 con 0x7efe4415a840 2024-04-25T19:20:01.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.539+0000 7efe43fff640 1 --2- 172.21.15.76:0/856207784 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7efe300744b0 0x7efe300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:01.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.539+0000 7efe43fff640 1 --2- 172.21.15.76:0/856207784 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7efe300744b0 0x7efe300769c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efe44161110 tx=0x7efe34008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:01.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.543+0000 7efe40ff9640 1 -- 172.21.15.76:0/856207784 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efe3c080240 con 0x7efe4415a840 2024-04-25T19:20:01.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.720+0000 7efe4a815640 1 -- 172.21.15.76:0/856207784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe44077fd0 con 0x7efe4415a840 2024-04-25T19:20:01.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.721+0000 7efe40ff9640 1 -- 172.21.15.76:0/856207784 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16855 (secure 0 0 0) 0x7efe3c084540 con 0x7efe4415a840 2024-04-25T19:20:01.723 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-04-25T19:20:01.723 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":23,"fsid":"d5adf865-03c7-4588-a151-044ededbff0b","created":"2024-04-25T19:15:01.693137+0000","modified":"2024-04-25T19:16:17.545036+0000","last_up_change":"2024-04-25T19:15:11.891681+0000","last_in_change":"2024-04-25T19:15:03.898065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T19:15:13.846514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T19:15:16.606533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T19:15:17.719197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72a16c02-48f9-45aa-9447-fcf755ebf936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6817","nonce":1409574998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6819","nonce":1409574998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6823","nonce":1409574998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1409574998},{"type":"v1","addr":"172.21.15.76:6821","nonce":1409574998}]},"public_addr":"172.21.15.76:6817/1409574998","cluster_addr":"172.21.15.76:6819/1409574998","heartbeat_back_addr":"172.21.15.76:6823/1409574998","heartbeat_front_addr":"172.21.15.76:6821/1409574998","state":["exists","up"]},{"osd":1,"uuid":"818d50f0-76c6-45f8-bf79-5e304f14c5cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6801","nonce":1302663821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6803","nonce":1302663821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6807","nonce":1302663821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":1302663821},{"type":"v1","addr":"172.21.15.76:6805","nonce":1302663821}]},"public_addr":"172.21.15.76:6801/1302663821","cluster_addr":"172.21.15.76:6803/1302663821","heartbeat_back_addr":"172.21.15.76:6807/1302663821","heartbeat_front_addr":"172.21.15.76:6805/1302663821","state":["exists","up"]},{"osd":2,"uuid":"5714c8f5-ae23-473c-8cf1-db73811acded","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6809","nonce":1992938181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6811","nonce":1992938181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6815","nonce":1992938181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1992938181},{"type":"v1","addr":"172.21.15.76:6813","nonce":1992938181}]},"public_addr":"172.21.15.76:6809/1992938181","cluster_addr":"172.21.15.76:6811/1992938181","heartbeat_back_addr":"172.21.15.76:6815/1992938181","heartbeat_front_addr":"172.21.15.76:6813/1992938181","state":["exists","up"]},{"osd":3,"uuid":"13204cdb-d8e6-4f97-bc6e-9561f442decf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6825","nonce":3418257163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6827","nonce":3418257163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6831","nonce":3418257163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3418257163},{"type":"v1","addr":"172.21.15.76:6829","nonce":3418257163}]},"public_addr":"172.21.15.76:6825/3418257163","cluster_addr":"172.21.15.76:6827/3418257163","heartbeat_back_addr":"172.21.15.76:6831/3418257163","heartbeat_front_addr":"172.21.15.76:6829/3418257163","state":["exists","up"]},{"osd":4,"uuid":"5bea5166-1986-4d56-b5f5-ecc8cbe7f3ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6801","nonce":2125002468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6803","nonce":2125002468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6807","nonce":2125002468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2125002468},{"type":"v1","addr":"172.21.15.105:6805","nonce":2125002468}]},"public_addr":"172.21.15.105:6801/2125002468","cluster_addr":"172.21.15.105:6803/2125002468","heartbeat_back_addr":"172.21.15.105:6807/2125002468","heartbeat_front_addr":"172.21.15.105:6805/2125002468","state":["exists","up"]},{"osd":5,"uuid":"96f89dcc-270d-4421-b8e8-3b92ba5f9844","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6817","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6819","nonce":3413103121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6821","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6823","nonce":3413103121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6831","nonce":3413103121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3413103121},{"type":"v1","addr":"172.21.15.105:6828","nonce":3413103121}]},"public_addr":"172.21.15.105:6819/3413103121","cluster_addr":"172.21.15.105:6823/3413103121","heartbeat_back_addr":"172.21.15.105:6831/3413103121","heartbeat_front_addr":"172.21.15.105:6828/3413103121","state":["exists","up"]},{"osd":6,"uuid":"93eb087a-d9ba-4096-b85d-b74d96e819a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6809","nonce":2721832954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6811","nonce":2721832954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6815","nonce":2721832954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":2721832954},{"type":"v1","addr":"172.21.15.105:6813","nonce":2721832954}]},"public_addr":"172.21.15.105:6809/2721832954","cluster_addr":"172.21.15.105:6811/2721832954","heartbeat_back_addr":"172.21.15.105:6815/2721832954","heartbeat_front_addr":"172.21.15.105:6813/2721832954","state":["exists","up"]},{"osd":7,"uuid":"2f80f790-ad2d-460e-a37d-47bf1771ef43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6818","nonce":548350430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6822","nonce":548350430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6827","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6829","nonce":548350430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":548350430},{"type":"v1","addr":"172.21.15.105:6825","nonce":548350430}]},"public_addr":"172.21.15.105:6818/548350430","cluster_addr":"172.21.15.105:6822/548350430","heartbeat_back_addr":"172.21.15.105:6829/548350430","heartbeat_front_addr":"172.21.15.105:6825/548350430","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.106901+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.072819+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.126865+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.053464+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:08.982614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.104825+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.112714+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T19:15:09.107387+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":1}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:20:01.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.723+0000 7efe4a815640 1 -- 172.21.15.76:0/856207784 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7efe300744b0 msgr2=0x7efe300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:01.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.723+0000 7efe4a815640 1 --2- 172.21.15.76:0/856207784 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7efe300744b0 0x7efe300769c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efe44161110 tx=0x7efe34008040 comp rx=0 tx=0).stop 2024-04-25T19:20:01.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.723+0000 7efe4a815640 1 -- 172.21.15.76:0/856207784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe4415a840 msgr2=0x7efe44147480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:01.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.723+0000 7efe4a815640 1 --2- 172.21.15.76:0/856207784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe4415a840 0x7efe44147480 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efe3c034040 tx=0x7efe3c004a40 comp rx=0 tx=0).stop 2024-04-25T19:20:01.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.724+0000 7efe4a815640 1 -- 172.21.15.76:0/856207784 shutdown_connections 2024-04-25T19:20:01.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.724+0000 7efe4a815640 1 --2- 172.21.15.76:0/856207784 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7efe300744b0 0x7efe300769c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.724+0000 7efe4a815640 1 --2- 172.21.15.76:0/856207784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe4415a840 0x7efe44147480 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.724+0000 7efe4a815640 1 --2- 172.21.15.76:0/856207784 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7efe44156bf0 0x7efe440ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.724+0000 7efe4a815640 1 --2- 172.21.15.76:0/856207784 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7efe440c9730 0x7efe441f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.724+0000 7efe4a815640 1 -- 172.21.15.76:0/856207784 >> 172.21.15.76:0/856207784 conn(0x7efe4406d6d0 msgr2=0x7efe44148b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:01.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.724+0000 7efe4a815640 1 -- 172.21.15.76:0/856207784 shutdown_connections 2024-04-25T19:20:01.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.724+0000 7efe4a815640 1 -- 172.21.15.76:0/856207784 wait complete. 2024-04-25T19:20:01.737 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T19:20:01.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.845+0000 7f94765ee640 1 Processor -- start 2024-04-25T19:20:01.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.845+0000 7f94765ee640 1 -- start start 2024-04-25T19:20:01.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.846+0000 7f94765ee640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f94700c7d10 0x7f947015b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:01.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.846+0000 7f94765ee640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9470154a60 0x7f94700cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:01.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.846+0000 7f94765ee640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9470072bc0 con 0x7f94700ce2a0 2024-04-25T19:20:01.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.846+0000 7f94765ee640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f94700ca3d0 con 0x7f94700c7d10 2024-04-25T19:20:01.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.846+0000 7f94765ee640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f94700ca5f0 con 0x7f9470154a60 2024-04-25T19:20:01.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.847+0000 7f946ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f94700c7d10 0x7f947015b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:01.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.847+0000 7f9474b8b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f94700ce2a0 0x7f94700caaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33098/0 (socket says 172.21.15.76:33098) 2024-04-25T19:20:01.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.847+0000 7f9474b8b640 1 -- 172.21.15.76:0/3093174153 learned_addr learned my addr 172.21.15.76:0/3093174153 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-04-25T19:20:01.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.847+0000 7f946f7fe640 1 --2- 172.21.15.76:0/3093174153 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9470154a60 0x7f94700cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:01.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.847+0000 7f946ffff640 1 -- 172.21.15.76:0/3093174153 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9470154a60 msgr2=0x7f94700cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:01.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.847+0000 7f946ffff640 1 --2- 172.21.15.76:0/3093174153 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9470154a60 0x7f94700cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.847+0000 7f946ffff640 1 -- 172.21.15.76:0/3093174153 >> v1:172.21.15.76:6789/0 conn(0x7f94700ce2a0 legacy=0x7f94700caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:01.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.847+0000 7f946ffff640 1 -- 172.21.15.76:0/3093174153 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94700c8c40 con 0x7f94700c7d10 2024-04-25T19:20:01.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.847+0000 7f946effd640 1 -- 172.21.15.76:0/3093174153 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 457780559 0 0) 0x7f9468001490 con 0x7f94700ce2a0 2024-04-25T19:20:01.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.848+0000 7f946ffff640 1 --2- 172.21.15.76:0/3093174153 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94700c7d10 0x7f947015b270 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f94600097a0 tx=0x7f946002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcfd83e69bcac8f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:01.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.848+0000 7f946effd640 1 -- 172.21.15.76:0/3093174153 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9460037070 con 0x7f94700c7d10 2024-04-25T19:20:01.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.848+0000 7f946effd640 1 -- 172.21.15.76:0/3093174153 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f9460032440 con 0x7f94700c7d10 2024-04-25T19:20:01.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.848+0000 7f946effd640 1 -- 172.21.15.76:0/3093174153 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9460032830 con 0x7f94700c7d10 2024-04-25T19:20:01.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.848+0000 7f94765ee640 1 -- 172.21.15.76:0/3093174153 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94700c7d10 msgr2=0x7f947015b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:01.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.848+0000 7f94765ee640 1 --2- 172.21.15.76:0/3093174153 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94700c7d10 0x7f947015b270 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f94600097a0 tx=0x7f946002a180 comp rx=0 tx=0).stop 2024-04-25T19:20:01.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.849+0000 7f94765ee640 1 -- 172.21.15.76:0/3093174153 shutdown_connections 2024-04-25T19:20:01.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.849+0000 7f94765ee640 1 --2- 172.21.15.76:0/3093174153 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9470154a60 0x7f94700cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.849+0000 7f94765ee640 1 --2- 172.21.15.76:0/3093174153 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94700c7d10 0x7f947015b270 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.849+0000 7f94765ee640 1 -- 172.21.15.76:0/3093174153 >> 172.21.15.76:0/3093174153 conn(0x7f9470070800 msgr2=0x7f9470070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:01.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.849+0000 7f94765ee640 1 -- 172.21.15.76:0/3093174153 shutdown_connections 2024-04-25T19:20:01.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.849+0000 7f94765ee640 1 -- 172.21.15.76:0/3093174153 wait complete. 2024-04-25T19:20:01.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.850+0000 7f94765ee640 1 Processor -- start 2024-04-25T19:20:01.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.850+0000 7f94765ee640 1 -- start start 2024-04-25T19:20:01.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.852+0000 7f94765ee640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94700c7d10 0x7f94701568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:01.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.852+0000 7f94765ee640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94700ce2a0 0x7f9470149180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:01.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.852+0000 7f94765ee640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9470154a60 0x7f94701507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:01.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.852+0000 7f94765ee640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f94700c94e0 con 0x7f9470154a60 2024-04-25T19:20:01.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.852+0000 7f94765ee640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f94700c9200 con 0x7f94700ce2a0 2024-04-25T19:20:01.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.852+0000 7f94765ee640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f9470154e70 con 0x7f94700c7d10 2024-04-25T19:20:01.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.852+0000 7f9474b8b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9470154a60 0x7f94701507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:01.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.852+0000 7f9474b8b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9470154a60 0x7f94701507d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58192/0 (socket says 172.21.15.76:58192) 2024-04-25T19:20:01.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.852+0000 7f9474b8b640 1 -- 172.21.15.76:0/367768299 learned_addr learned my addr 172.21.15.76:0/367768299 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:20:01.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.852+0000 7f946ffff640 1 --2- 172.21.15.76:0/367768299 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94700c7d10 0x7f94701568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:01.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.852+0000 7f946f7fe640 1 --2- 172.21.15.76:0/367768299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94700ce2a0 0x7f9470149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:01.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.852+0000 7f9474b8b640 1 -- 172.21.15.76:0/367768299 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94700c7d10 msgr2=0x7f94701568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:01.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.852+0000 7f9474b8b640 1 --2- 172.21.15.76:0/367768299 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94700c7d10 0x7f94701568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.852+0000 7f9474b8b640 1 -- 172.21.15.76:0/367768299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94700ce2a0 msgr2=0x7f9470149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:01.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.852+0000 7f9474b8b640 1 --2- 172.21.15.76:0/367768299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94700ce2a0 0x7f9470149180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:01.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.852+0000 7f9474b8b640 1 -- 172.21.15.76:0/367768299 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94701552c0 con 0x7f9470154a60 2024-04-25T19:20:01.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.852+0000 7f946f7fe640 1 --2- 172.21.15.76:0/367768299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94700ce2a0 0x7f9470149180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:20:01.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.852+0000 7f9474b8b640 1 --2- 172.21.15.76:0/367768299 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9470154a60 0x7f94701507d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f946800c6d0 tx=0x7f946800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:01.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.852+0000 7f946cff9640 1 -- 172.21.15.76:0/367768299 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9468018070 con 0x7f9470154a60 2024-04-25T19:20:01.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.852+0000 7f946cff9640 1 -- 172.21.15.76:0/367768299 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f9468004d60 con 0x7f9470154a60 2024-04-25T19:20:01.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.852+0000 7f946cff9640 1 -- 172.21.15.76:0/367768299 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946800a8a0 con 0x7f9470154a60 2024-04-25T19:20:01.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.852+0000 7f94765ee640 1 -- 172.21.15.76:0/367768299 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9470156630 con 0x7f9470154a60 2024-04-25T19:20:01.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.853+0000 7f94765ee640 1 -- 172.21.15.76:0/367768299 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9470155860 con 0x7f9470154a60 2024-04-25T19:20:01.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.854+0000 7f94765ee640 1 -- 172.21.15.76:0/367768299 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f947006a390 con 0x7f9470154a60 2024-04-25T19:20:01.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.854+0000 7f946cff9640 1 -- 172.21.15.76:0/367768299 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f946801b070 con 0x7f9470154a60 2024-04-25T19:20:01.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.855+0000 7f946cff9640 1 --2- 172.21.15.76:0/367768299 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f945c0744e0 0x7f945c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:01.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.855+0000 7f946cff9640 1 -- 172.21.15.76:0/367768299 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f9468094a30 con 0x7f9470154a60 2024-04-25T19:20:01.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.855+0000 7f946ffff640 1 --2- 172.21.15.76:0/367768299 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f945c0744e0 0x7f945c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:01.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.855+0000 7f946ffff640 1 --2- 172.21.15.76:0/367768299 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f945c0744e0 0x7f945c0769f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9460034000 tx=0x7f946002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:01.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:01.860+0000 7f946cff9640 1 -- 172.21.15.76:0/367768299 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9470072bc0 con 0x7f9470154a60 2024-04-25T19:20:02.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.073+0000 7f94765ee640 1 -- 172.21.15.76:0/367768299 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9470073160 con 0x7f9470154a60 2024-04-25T19:20:02.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.076+0000 7f946cff9640 1 -- 172.21.15.76:0/367768299 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) ==== 161+0+0 (secure 0 0 0) 0x7f946805f140 con 0x7f9470154a60 2024-04-25T19:20:02.077 INFO:teuthology.orchestra.run.smithi076.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T19:20:02.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.078+0000 7f94765ee640 1 -- 172.21.15.76:0/367768299 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f945c0744e0 msgr2=0x7f945c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:02.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.078+0000 7f94765ee640 1 --2- 172.21.15.76:0/367768299 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f945c0744e0 0x7f945c0769f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9460034000 tx=0x7f946002e000 comp rx=0 tx=0).stop 2024-04-25T19:20:02.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.078+0000 7f94765ee640 1 -- 172.21.15.76:0/367768299 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9470154a60 msgr2=0x7f94701507d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:02.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.078+0000 7f94765ee640 1 --2- 172.21.15.76:0/367768299 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9470154a60 0x7f94701507d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f946800c6d0 tx=0x7f946800cb90 comp rx=0 tx=0).stop 2024-04-25T19:20:02.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.078+0000 7f94765ee640 1 -- 172.21.15.76:0/367768299 shutdown_connections 2024-04-25T19:20:02.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.078+0000 7f94765ee640 1 --2- 172.21.15.76:0/367768299 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f945c0744e0 0x7f945c0769f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:02.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.078+0000 7f94765ee640 1 --2- 172.21.15.76:0/367768299 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9470154a60 0x7f94701507d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:02.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.078+0000 7f94765ee640 1 --2- 172.21.15.76:0/367768299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f94700ce2a0 0x7f9470149180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:02.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.078+0000 7f94765ee640 1 --2- 172.21.15.76:0/367768299 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f94700c7d10 0x7f94701568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:02.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.078+0000 7f94765ee640 1 -- 172.21.15.76:0/367768299 >> 172.21.15.76:0/367768299 conn(0x7f9470070800 msgr2=0x7f9470159bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:02.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.079+0000 7f94765ee640 1 -- 172.21.15.76:0/367768299 shutdown_connections 2024-04-25T19:20:02.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.079+0000 7f94765ee640 1 -- 172.21.15.76:0/367768299 wait complete. 2024-04-25T19:20:02.085 INFO:tasks.ceph.mds.f.smithi105.stderr: -8681> 2024-04-25T19:15:12.377+0000 7f9f417bb640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6837/0: (98) Address already in use 2024-04-25T19:20:02.093 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T19:20:02.141 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2024-04-25T19:20:02.145 INFO:tasks.ceph.mds.e.smithi076.stdout:starting mds.e at 2024-04-25T19:20:02.148 INFO:tasks.ceph.mds.d.smithi105.stdout:starting mds.d at 2024-04-25T19:20:02.155 INFO:tasks.ceph.mds.f.smithi105.stdout:starting mds.f at 2024-04-25T19:20:02.156 INFO:tasks.ceph.mds.b.smithi105.stdout:starting mds.b at 2024-04-25T19:20:02.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.208+0000 7f79a1234640 1 Processor -- start 2024-04-25T19:20:02.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.208+0000 7f79a1234640 1 -- start start 2024-04-25T19:20:02.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.210+0000 7f79a1234640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f799c159840 0x7f799c153b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:02.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.210+0000 7f79a1234640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f799c15d880 0x7f799c0c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:02.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.210+0000 7f79a1234640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f799c0715d0 con 0x7f799c0c5b20 2024-04-25T19:20:02.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.210+0000 7f79a1234640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f799c15ce30 con 0x7f799c159840 2024-04-25T19:20:02.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.210+0000 7f79a1234640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f799c0718b0 con 0x7f799c15d880 2024-04-25T19:20:02.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.210+0000 7f799ad9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f799c159840 0x7f799c153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:02.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.210+0000 7f799ad9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f799c159840 0x7f799c153b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58238/0 (socket says 172.21.15.76:58238) 2024-04-25T19:20:02.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.210+0000 7f799ad9d640 1 -- 172.21.15.76:0/3903500218 learned_addr learned my addr 172.21.15.76:0/3903500218 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:20:02.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.210+0000 7f799b59e640 1 --2- 172.21.15.76:0/3903500218 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f799c15d880 0x7f799c0c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:02.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.210+0000 7f799ad9d640 1 -- 172.21.15.76:0/3903500218 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f799c15d880 msgr2=0x7f799c0c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:02.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.210+0000 7f799ad9d640 1 --2- 172.21.15.76:0/3903500218 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f799c15d880 0x7f799c0c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:02.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.210+0000 7f799ad9d640 1 -- 172.21.15.76:0/3903500218 >> v1:172.21.15.105:6789/0 conn(0x7f799c0c5b20 legacy=0x7f799c0c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:02.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.210+0000 7f7999d9b640 1 -- 172.21.15.76:0/3903500218 <== mon.1 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 513487505 0 0) 0x7f799c0715d0 con 0x7f799c0c5b20 2024-04-25T19:20:02.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.210+0000 7f799ad9d640 1 -- 172.21.15.76:0/3903500218 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799c071b90 con 0x7f799c159840 2024-04-25T19:20:02.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.211+0000 7f799ad9d640 1 --2- 172.21.15.76:0/3903500218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f799c159840 0x7f799c153b70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7990009870 tx=0x7f799002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8249958150c88b79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:02.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.211+0000 7f7999d9b640 1 -- 172.21.15.76:0/3903500218 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7990037070 con 0x7f799c159840 2024-04-25T19:20:02.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.211+0000 7f7999d9b640 1 -- 172.21.15.76:0/3903500218 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f7990032440 con 0x7f799c159840 2024-04-25T19:20:02.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.211+0000 7f7999d9b640 1 -- 172.21.15.76:0/3903500218 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7990032830 con 0x7f799c159840 2024-04-25T19:20:02.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.212+0000 7f79a1234640 1 -- 172.21.15.76:0/3903500218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f799c159840 msgr2=0x7f799c153b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:02.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.212+0000 7f79a1234640 1 --2- 172.21.15.76:0/3903500218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f799c159840 0x7f799c153b70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7990009870 tx=0x7f799002a270 comp rx=0 tx=0).stop 2024-04-25T19:20:02.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.212+0000 7f79a1234640 1 -- 172.21.15.76:0/3903500218 shutdown_connections 2024-04-25T19:20:02.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.212+0000 7f79a1234640 1 --2- 172.21.15.76:0/3903500218 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f799c15d880 0x7f799c0c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:02.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.212+0000 7f79a1234640 1 --2- 172.21.15.76:0/3903500218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f799c159840 0x7f799c153b70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:02.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.212+0000 7f79a1234640 1 -- 172.21.15.76:0/3903500218 >> 172.21.15.76:0/3903500218 conn(0x7f799c0cecc0 msgr2=0x7f799c0cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:02.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.212+0000 7f79a1234640 1 -- 172.21.15.76:0/3903500218 shutdown_connections 2024-04-25T19:20:02.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.212+0000 7f79a1234640 1 -- 172.21.15.76:0/3903500218 wait complete. 2024-04-25T19:20:02.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.214+0000 7f79a1234640 1 Processor -- start 2024-04-25T19:20:02.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.214+0000 7f79a1234640 1 -- start start 2024-04-25T19:20:02.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.215+0000 7f79a1234640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f799c0c5b20 0x7f799c155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:02.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.215+0000 7f79a1234640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f799c159840 0x7f799c1f68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:02.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.215+0000 7f79a1234640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f799c15d880 0x7f799c15bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:02.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.215+0000 7f79a1234640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f799c15dc90 con 0x7f799c15d880 2024-04-25T19:20:02.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.215+0000 7f79a1234640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f799c071270 con 0x7f799c159840 2024-04-25T19:20:02.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.215+0000 7f79a1234640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f799c0c8e30 con 0x7f799c0c5b20 2024-04-25T19:20:02.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.215+0000 7f799b59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f799c15d880 0x7f799c15bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:02.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.215+0000 7f799b59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f799c15d880 0x7f799c15bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58250/0 (socket says 172.21.15.76:58250) 2024-04-25T19:20:02.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.215+0000 7f799b59e640 1 -- 172.21.15.76:0/1762197788 learned_addr learned my addr 172.21.15.76:0/1762197788 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:20:02.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.215+0000 7f799ad9d640 1 --2- 172.21.15.76:0/1762197788 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f799c0c5b20 0x7f799c155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:02.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.215+0000 7f799a59c640 1 --2- 172.21.15.76:0/1762197788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f799c159840 0x7f799c1f68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:02.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.216+0000 7f799b59e640 1 -- 172.21.15.76:0/1762197788 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f799c0c5b20 msgr2=0x7f799c155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:02.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.216+0000 7f799b59e640 1 --2- 172.21.15.76:0/1762197788 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f799c0c5b20 0x7f799c155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:02.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.216+0000 7f799b59e640 1 -- 172.21.15.76:0/1762197788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f799c159840 msgr2=0x7f799c1f68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:02.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.216+0000 7f799b59e640 1 --2- 172.21.15.76:0/1762197788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f799c159840 0x7f799c1f68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:02.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.216+0000 7f799b59e640 1 -- 172.21.15.76:0/1762197788 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799c1f6e00 con 0x7f799c15d880 2024-04-25T19:20:02.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.216+0000 7f799b59e640 1 --2- 172.21.15.76:0/1762197788 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f799c15d880 0x7f799c15bf70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7994009cb0 tx=0x7f799400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:02.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.216+0000 7f79837fe640 1 -- 172.21.15.76:0/1762197788 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994018020 con 0x7f799c15d880 2024-04-25T19:20:02.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.216+0000 7f79837fe640 1 -- 172.21.15.76:0/1762197788 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f7994014480 con 0x7f799c15d880 2024-04-25T19:20:02.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.216+0000 7f79837fe640 1 -- 172.21.15.76:0/1762197788 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994014890 con 0x7f799c15d880 2024-04-25T19:20:02.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.216+0000 7f79a1234640 1 -- 172.21.15.76:0/1762197788 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799c14c450 con 0x7f799c15d880 2024-04-25T19:20:02.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.216+0000 7f79a1234640 1 -- 172.21.15.76:0/1762197788 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f799c14ca10 con 0x7f799c15d880 2024-04-25T19:20:02.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.217+0000 7f79a1234640 1 -- 172.21.15.76:0/1762197788 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f799c06a390 con 0x7f799c15d880 2024-04-25T19:20:02.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.218+0000 7f79837fe640 1 -- 172.21.15.76:0/1762197788 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7994005d10 con 0x7f799c15d880 2024-04-25T19:20:02.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.218+0000 7f79837fe640 1 --2- 172.21.15.76:0/1762197788 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f79880744b0 0x7f79880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:02.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.218+0000 7f79837fe640 1 -- 172.21.15.76:0/1762197788 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6255+0+0 (secure 0 0 0) 0x7f7994096080 con 0x7f799c15d880 2024-04-25T19:20:02.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.218+0000 7f799ad9d640 1 --2- 172.21.15.76:0/1762197788 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f79880744b0 0x7f79880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:02.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.219+0000 7f799ad9d640 1 --2- 172.21.15.76:0/1762197788 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f79880744b0 0x7f79880769c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7990034000 tx=0x7f799002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:02.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.223+0000 7f79837fe640 1 -- 172.21.15.76:0/1762197788 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7994022030 con 0x7f799c15d880 2024-04-25T19:20:02.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:02.434+0000 7f79a1234640 1 -- 172.21.15.76:0/1762197788 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f799c0ceaf0 con 0x7f799c15d880 2024-04-25T19:20:03.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.079+0000 7f79837fe640 1 -- 172.21.15.76:0/1762197788 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) ==== 104+0+0 (secure 0 0 0) 0x7f7994010040 con 0x7f799c15d880 2024-04-25T19:20:03.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.081+0000 7f79a1234640 1 -- 172.21.15.76:0/1762197788 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f79880744b0 msgr2=0x7f79880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:03.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.081+0000 7f79a1234640 1 --2- 172.21.15.76:0/1762197788 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f79880744b0 0x7f79880769c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7990034000 tx=0x7f799002e000 comp rx=0 tx=0).stop 2024-04-25T19:20:03.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.081+0000 7f79a1234640 1 -- 172.21.15.76:0/1762197788 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f799c15d880 msgr2=0x7f799c15bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:03.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.081+0000 7f79a1234640 1 --2- 172.21.15.76:0/1762197788 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f799c15d880 0x7f799c15bf70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7994009cb0 tx=0x7f799400c490 comp rx=0 tx=0).stop 2024-04-25T19:20:03.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.081+0000 7f79a1234640 1 -- 172.21.15.76:0/1762197788 shutdown_connections 2024-04-25T19:20:03.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.081+0000 7f79a1234640 1 --2- 172.21.15.76:0/1762197788 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f79880744b0 0x7f79880769c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:03.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.082+0000 7f79a1234640 1 --2- 172.21.15.76:0/1762197788 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f799c15d880 0x7f799c15bf70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:03.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.082+0000 7f79a1234640 1 --2- 172.21.15.76:0/1762197788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f799c159840 0x7f799c1f68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:03.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.082+0000 7f79a1234640 1 --2- 172.21.15.76:0/1762197788 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f799c0c5b20 0x7f799c155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:03.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.082+0000 7f79a1234640 1 -- 172.21.15.76:0/1762197788 >> 172.21.15.76:0/1762197788 conn(0x7f799c0cecc0 msgr2=0x7f799c1496a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:03.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.082+0000 7f79a1234640 1 -- 172.21.15.76:0/1762197788 shutdown_connections 2024-04-25T19:20:03.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.082+0000 7f79a1234640 1 -- 172.21.15.76:0/1762197788 wait complete. 2024-04-25T19:20:03.097 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T19:20:03.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.210+0000 7fa0545ce640 1 Processor -- start 2024-04-25T19:20:03.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.210+0000 7fa0545ce640 1 -- start start 2024-04-25T19:20:03.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.211+0000 7fa0545ce640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa04c072b60 0x7fa04c072f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:03.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.211+0000 7fa0545ce640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa04c15e740 0x7fa04c15eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:03.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.211+0000 7fa0545ce640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa04c15af40 con 0x7fa04c154d60 2024-04-25T19:20:03.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.211+0000 7fa0545ce640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa04c15bac0 con 0x7fa04c072b60 2024-04-25T19:20:03.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.211+0000 7fa0545ce640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa04c15e2b0 con 0x7fa04c15e740 2024-04-25T19:20:03.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.212+0000 7fa051b69640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa04c072b60 0x7fa04c072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:03.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.212+0000 7fa051b69640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa04c072b60 0x7fa04c072f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58256/0 (socket says 172.21.15.76:58256) 2024-04-25T19:20:03.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.212+0000 7fa051b69640 1 -- 172.21.15.76:0/247686324 learned_addr learned my addr 172.21.15.76:0/247686324 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:20:03.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.212+0000 7fa052b6b640 1 --2- 172.21.15.76:0/247686324 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa04c15e740 0x7fa04c15eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:03.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.212+0000 7fa051b69640 1 -- 172.21.15.76:0/247686324 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa04c15e740 msgr2=0x7fa04c15eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:03.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.212+0000 7fa051b69640 1 --2- 172.21.15.76:0/247686324 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa04c15e740 0x7fa04c15eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:03.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.212+0000 7fa051b69640 1 -- 172.21.15.76:0/247686324 >> v1:172.21.15.105:6789/0 conn(0x7fa04c154d60 legacy=0x7fa04c0ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:03.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.212+0000 7fa051b69640 1 -- 172.21.15.76:0/247686324 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04c15b220 con 0x7fa04c072b60 2024-04-25T19:20:03.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.213+0000 7fa051b69640 1 --2- 172.21.15.76:0/247686324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa04c072b60 0x7fa04c072f70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009870 tx=0x7fa03c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af77c59b02dc665 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:03.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.213+0000 7fa051368640 1 -- 172.21.15.76:0/247686324 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c037070 con 0x7fa04c072b60 2024-04-25T19:20:03.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.213+0000 7fa051368640 1 -- 172.21.15.76:0/247686324 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa03c032440 con 0x7fa04c072b60 2024-04-25T19:20:03.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.213+0000 7fa051368640 1 -- 172.21.15.76:0/247686324 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c032830 con 0x7fa04c072b60 2024-04-25T19:20:03.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.213+0000 7fa0545ce640 1 -- 172.21.15.76:0/247686324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa04c072b60 msgr2=0x7fa04c072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:03.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.213+0000 7fa0545ce640 1 --2- 172.21.15.76:0/247686324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa04c072b60 0x7fa04c072f70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009870 tx=0x7fa03c02a1c0 comp rx=0 tx=0).stop 2024-04-25T19:20:03.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.214+0000 7fa0545ce640 1 -- 172.21.15.76:0/247686324 shutdown_connections 2024-04-25T19:20:03.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.214+0000 7fa0545ce640 1 --2- 172.21.15.76:0/247686324 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa04c15e740 0x7fa04c15eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:03.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.214+0000 7fa0545ce640 1 --2- 172.21.15.76:0/247686324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa04c072b60 0x7fa04c072f70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:03.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.214+0000 7fa0545ce640 1 -- 172.21.15.76:0/247686324 >> 172.21.15.76:0/247686324 conn(0x7fa04c070b00 msgr2=0x7fa04c070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:03.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.214+0000 7fa0545ce640 1 -- 172.21.15.76:0/247686324 shutdown_connections 2024-04-25T19:20:03.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.214+0000 7fa0545ce640 1 -- 172.21.15.76:0/247686324 wait complete. 2024-04-25T19:20:03.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.215+0000 7fa0545ce640 1 Processor -- start 2024-04-25T19:20:03.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.215+0000 7fa0545ce640 1 -- start start 2024-04-25T19:20:03.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.217+0000 7fa0545ce640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa04c072b60 0x7fa04c1e3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:03.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.217+0000 7fa0545ce640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa04c154d60 0x7fa04c158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:03.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.217+0000 7fa0545ce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa04c15e740 0x7fa04c0b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:03.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.217+0000 7fa0545ce640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c155170 con 0x7fa04c154d60 2024-04-25T19:20:03.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.217+0000 7fa0545ce640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c15b7e0 con 0x7fa04c15e740 2024-04-25T19:20:03.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.217+0000 7fa0545ce640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7fa04c0cb0e0 con 0x7fa04c072b60 2024-04-25T19:20:03.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.217+0000 7fa051b69640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa04c154d60 0x7fa04c158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:03.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.217+0000 7fa051b69640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa04c154d60 0x7fa04c158e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58262/0 (socket says 172.21.15.76:58262) 2024-04-25T19:20:03.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.217+0000 7fa051b69640 1 -- 172.21.15.76:0/2299630465 learned_addr learned my addr 172.21.15.76:0/2299630465 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:20:03.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.217+0000 7fa05236a640 1 --2- 172.21.15.76:0/2299630465 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa04c072b60 0x7fa04c1e3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:03.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.217+0000 7fa052b6b640 1 --2- 172.21.15.76:0/2299630465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa04c15e740 0x7fa04c0b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:03.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.217+0000 7fa051b69640 1 -- 172.21.15.76:0/2299630465 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa04c072b60 msgr2=0x7fa04c1e3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:03.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.217+0000 7fa051b69640 1 --2- 172.21.15.76:0/2299630465 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa04c072b60 0x7fa04c1e3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:03.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.217+0000 7fa051b69640 1 -- 172.21.15.76:0/2299630465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa04c15e740 msgr2=0x7fa04c0b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:03.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.217+0000 7fa051b69640 1 --2- 172.21.15.76:0/2299630465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa04c15e740 0x7fa04c0b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:03.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.217+0000 7fa051b69640 1 -- 172.21.15.76:0/2299630465 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04c1593d0 con 0x7fa04c154d60 2024-04-25T19:20:03.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.217+0000 7fa05236a640 1 --2- 172.21.15.76:0/2299630465 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa04c072b60 0x7fa04c1e3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:20:03.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.217+0000 7fa051b69640 1 --2- 172.21.15.76:0/2299630465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa04c154d60 0x7fa04c158e90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa03c000c00 tx=0x7fa03c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:03.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.217+0000 7fa042ffd640 1 -- 172.21.15.76:0/2299630465 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c037040 con 0x7fa04c154d60 2024-04-25T19:20:03.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.217+0000 7fa042ffd640 1 -- 172.21.15.76:0/2299630465 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa03c042070 con 0x7fa04c154d60 2024-04-25T19:20:03.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.217+0000 7fa0545ce640 1 -- 172.21.15.76:0/2299630465 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa04c15a0b0 con 0x7fa04c154d60 2024-04-25T19:20:03.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.217+0000 7fa042ffd640 1 -- 172.21.15.76:0/2299630465 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c030bd0 con 0x7fa04c154d60 2024-04-25T19:20:03.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.217+0000 7fa0545ce640 1 -- 172.21.15.76:0/2299630465 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa04c15a650 con 0x7fa04c154d60 2024-04-25T19:20:03.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.218+0000 7fa0545ce640 1 -- 172.21.15.76:0/2299630465 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa04c06a390 con 0x7fa04c154d60 2024-04-25T19:20:03.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.219+0000 7fa042ffd640 1 -- 172.21.15.76:0/2299630465 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa03c02ad10 con 0x7fa04c154d60 2024-04-25T19:20:03.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.219+0000 7fa042ffd640 1 --2- 172.21.15.76:0/2299630465 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa0300744b0 0x7fa0300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:03.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.220+0000 7fa042ffd640 1 -- 172.21.15.76:0/2299630465 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6255+0+0 (secure 0 0 0) 0x7fa03c0b5d60 con 0x7fa04c154d60 2024-04-25T19:20:03.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.220+0000 7fa05236a640 1 --2- 172.21.15.76:0/2299630465 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa0300744b0 0x7fa0300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:03.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.220+0000 7fa05236a640 1 --2- 172.21.15.76:0/2299630465 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa0300744b0 0x7fa0300769c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa048001d60 tx=0x7fa048009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:03.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.224+0000 7fa042ffd640 1 -- 172.21.15.76:0/2299630465 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa03c032e20 con 0x7fa04c154d60 2024-04-25T19:20:03.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:03.398+0000 7fa0545ce640 1 -- 172.21.15.76:0/2299630465 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa04c15d640 con 0x7fa04c154d60 2024-04-25T19:20:04.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.083+0000 7fa042ffd640 1 -- 172.21.15.76:0/2299630465 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) ==== 181+0+0 (secure 0 0 0) 0x7fa03c080360 con 0x7fa04c154d60 2024-04-25T19:20:04.084 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' removed 2024-04-25T19:20:04.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.085+0000 7fa0545ce640 1 -- 172.21.15.76:0/2299630465 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa0300744b0 msgr2=0x7fa0300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:04.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.085+0000 7fa0545ce640 1 --2- 172.21.15.76:0/2299630465 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa0300744b0 0x7fa0300769c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa048001d60 tx=0x7fa048009040 comp rx=0 tx=0).stop 2024-04-25T19:20:04.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.085+0000 7fa0545ce640 1 -- 172.21.15.76:0/2299630465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa04c154d60 msgr2=0x7fa04c158e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:04.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.085+0000 7fa0545ce640 1 --2- 172.21.15.76:0/2299630465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa04c154d60 0x7fa04c158e90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa03c000c00 tx=0x7fa03c0046c0 comp rx=0 tx=0).stop 2024-04-25T19:20:04.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.085+0000 7fa0545ce640 1 -- 172.21.15.76:0/2299630465 shutdown_connections 2024-04-25T19:20:04.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.085+0000 7fa0545ce640 1 --2- 172.21.15.76:0/2299630465 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7fa0300744b0 0x7fa0300769c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:04.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.085+0000 7fa0545ce640 1 --2- 172.21.15.76:0/2299630465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa04c15e740 0x7fa04c0b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:04.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.085+0000 7fa0545ce640 1 --2- 172.21.15.76:0/2299630465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa04c154d60 0x7fa04c158e90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:04.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.085+0000 7fa0545ce640 1 --2- 172.21.15.76:0/2299630465 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7fa04c072b60 0x7fa04c1e3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:04.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.085+0000 7fa0545ce640 1 -- 172.21.15.76:0/2299630465 >> 172.21.15.76:0/2299630465 conn(0x7fa04c070b00 msgr2=0x7fa04c072f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:04.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.085+0000 7fa0545ce640 1 -- 172.21.15.76:0/2299630465 shutdown_connections 2024-04-25T19:20:04.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.085+0000 7fa0545ce640 1 -- 172.21.15.76:0/2299630465 wait complete. 2024-04-25T19:20:04.101 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T19:20:04.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.215+0000 7f9c7e7b2640 1 Processor -- start 2024-04-25T19:20:04.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.215+0000 7f9c7e7b2640 1 -- start start 2024-04-25T19:20:04.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.216+0000 7f9c7e7b2640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c78073b10 0x7f9c780cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:04.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.216+0000 7f9c7e7b2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9c7815e4f0 0x7f9c780cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:04.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.216+0000 7f9c7e7b2640 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c780715a0 con 0x7f9c780721f0 2024-04-25T19:20:04.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.216+0000 7f9c7e7b2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f9c78071ae0 con 0x7f9c7815e4f0 2024-04-25T19:20:04.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.216+0000 7f9c7e7b2640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f9c780cb1c0 con 0x7f9c78073b10 2024-04-25T19:20:04.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.217+0000 7f9c7cd4f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9c7815e4f0 0x7f9c780cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:04.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.217+0000 7f9c7cd4f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9c7815e4f0 0x7f9c780cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58266/0 (socket says 172.21.15.76:58266) 2024-04-25T19:20:04.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.217+0000 7f9c7cd4f640 1 -- 172.21.15.76:0/3844176051 learned_addr learned my addr 172.21.15.76:0/3844176051 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:20:04.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.217+0000 7f9c77fff640 1 --2- 172.21.15.76:0/3844176051 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c78073b10 0x7f9c780cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:04.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.217+0000 7f9c7cd4f640 1 -- 172.21.15.76:0/3844176051 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c78073b10 msgr2=0x7f9c780cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:20:04.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.217+0000 7f9c7cd4f640 1 --2- 172.21.15.76:0/3844176051 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c78073b10 0x7f9c780cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:04.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.217+0000 7f9c7cd4f640 1 -- 172.21.15.76:0/3844176051 >> v1:172.21.15.105:6789/0 conn(0x7f9c780721f0 legacy=0x7f9c7815dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:04.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.217+0000 7f9c7cd4f640 1 -- 172.21.15.76:0/3844176051 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c78071840 con 0x7f9c7815e4f0 2024-04-25T19:20:04.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.218+0000 7f9c7cd4f640 1 --2- 172.21.15.76:0/3844176051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c7815e4f0 0x7f9c780cc230 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9c70009e50 tx=0x7f9c7002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55d29cc9955886e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:04.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.218+0000 7f9c76ffd640 1 -- 172.21.15.76:0/3844176051 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c70036070 con 0x7f9c7815e4f0 2024-04-25T19:20:04.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.218+0000 7f9c76ffd640 1 -- 172.21.15.76:0/3844176051 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f9c70031600 con 0x7f9c7815e4f0 2024-04-25T19:20:04.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.218+0000 7f9c76ffd640 1 -- 172.21.15.76:0/3844176051 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c700319f0 con 0x7f9c7815e4f0 2024-04-25T19:20:04.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.218+0000 7f9c7e7b2640 1 -- 172.21.15.76:0/3844176051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c7815e4f0 msgr2=0x7f9c780cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:04.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.218+0000 7f9c7e7b2640 1 --2- 172.21.15.76:0/3844176051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c7815e4f0 0x7f9c780cc230 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9c70009e50 tx=0x7f9c7002a680 comp rx=0 tx=0).stop 2024-04-25T19:20:04.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.218+0000 7f9c7e7b2640 1 -- 172.21.15.76:0/3844176051 shutdown_connections 2024-04-25T19:20:04.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.218+0000 7f9c7e7b2640 1 --2- 172.21.15.76:0/3844176051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c7815e4f0 0x7f9c780cc230 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:04.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.218+0000 7f9c7e7b2640 1 --2- 172.21.15.76:0/3844176051 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c78073b10 0x7f9c780cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:04.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.218+0000 7f9c7e7b2640 1 -- 172.21.15.76:0/3844176051 >> 172.21.15.76:0/3844176051 conn(0x7f9c7806fb50 msgr2=0x7f9c7806ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:04.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.219+0000 7f9c7e7b2640 1 -- 172.21.15.76:0/3844176051 shutdown_connections 2024-04-25T19:20:04.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.219+0000 7f9c7e7b2640 1 -- 172.21.15.76:0/3844176051 wait complete. 2024-04-25T19:20:04.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.220+0000 7f9c7e7b2640 1 Processor -- start 2024-04-25T19:20:04.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.220+0000 7f9c7e7b2640 1 -- start start 2024-04-25T19:20:04.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.222+0000 7f9c7e7b2640 1 --2- >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c780721f0 0x7f9c7814a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:04.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.222+0000 7f9c7e7b2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c78073b10 0x7f9c7815e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:04.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.222+0000 7f9c7e7b2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c781f3e60 0x7f9c7815e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:04.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.222+0000 7f9c7e7b2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9c78074840 con 0x7f9c78073b10 2024-04-25T19:20:04.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.222+0000 7f9c7e7b2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9c78074560 con 0x7f9c781f3e60 2024-04-25T19:20:04.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.222+0000 7f9c7e7b2640 1 -- --> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] -- mon_getmap magic: 0 -- 0x7f9c78071d70 con 0x7f9c780721f0 2024-04-25T19:20:04.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.222+0000 7f9c777fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c78073b10 0x7f9c7815e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:04.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.222+0000 7f9c777fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c78073b10 0x7f9c7815e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58276/0 (socket says 172.21.15.76:58276) 2024-04-25T19:20:04.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.222+0000 7f9c777fe640 1 -- 172.21.15.76:0/1075031028 learned_addr learned my addr 172.21.15.76:0/1075031028 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-04-25T19:20:04.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.222+0000 7f9c7cd4f640 1 --2- 172.21.15.76:0/1075031028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c781f3e60 0x7f9c7815e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:04.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.222+0000 7f9c77fff640 1 --2- 172.21.15.76:0/1075031028 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c780721f0 0x7f9c7814a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:04.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.222+0000 7f9c777fe640 1 -- 172.21.15.76:0/1075031028 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c780721f0 msgr2=0x7f9c7814a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:04.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.222+0000 7f9c777fe640 1 --2- 172.21.15.76:0/1075031028 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c780721f0 0x7f9c7814a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:04.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.222+0000 7f9c777fe640 1 -- 172.21.15.76:0/1075031028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c781f3e60 msgr2=0x7f9c7815e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:04.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.222+0000 7f9c777fe640 1 --2- 172.21.15.76:0/1075031028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c781f3e60 0x7f9c7815e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:04.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.222+0000 7f9c777fe640 1 -- 172.21.15.76:0/1075031028 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7814add0 con 0x7f9c78073b10 2024-04-25T19:20:04.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.222+0000 7f9c777fe640 1 --2- 172.21.15.76:0/1075031028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c78073b10 0x7f9c7815e240 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c00c3f0 tx=0x7f9c6c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:04.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.222+0000 7f9c74ff9640 1 -- 172.21.15.76:0/1075031028 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c0052d0 con 0x7f9c78073b10 2024-04-25T19:20:04.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.222+0000 7f9c74ff9640 1 -- 172.21.15.76:0/1075031028 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f9c6c005490 con 0x7f9c78073b10 2024-04-25T19:20:04.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.222+0000 7f9c7e7b2640 1 -- 172.21.15.76:0/1075031028 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c7815b9c0 con 0x7f9c78073b10 2024-04-25T19:20:04.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.222+0000 7f9c74ff9640 1 -- 172.21.15.76:0/1075031028 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c007280 con 0x7f9c78073b10 2024-04-25T19:20:04.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.223+0000 7f9c7e7b2640 1 -- 172.21.15.76:0/1075031028 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c781f1370 con 0x7f9c78073b10 2024-04-25T19:20:04.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.224+0000 7f9c7e7b2640 1 -- 172.21.15.76:0/1075031028 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c7806a390 con 0x7f9c78073b10 2024-04-25T19:20:04.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.224+0000 7f9c74ff9640 1 -- 172.21.15.76:0/1075031028 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9c6c007420 con 0x7f9c78073b10 2024-04-25T19:20:04.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.225+0000 7f9c74ff9640 1 --2- 172.21.15.76:0/1075031028 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9c640744e0 0x7f9c640769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:20:04.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.225+0000 7f9c74ff9640 1 -- 172.21.15.76:0/1075031028 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5817+0+0 (secure 0 0 0) 0x7f9c6c094750 con 0x7f9c78073b10 2024-04-25T19:20:04.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.225+0000 7f9c77fff640 1 --2- 172.21.15.76:0/1075031028 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9c640744e0 0x7f9c640769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:20:04.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.225+0000 7f9c77fff640 1 --2- 172.21.15.76:0/1075031028 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9c640744e0 0x7f9c640769f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9c68009aa0 tx=0x7f9c68009190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:20:04.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.229+0000 7f9c74ff9640 1 -- 172.21.15.76:0/1075031028 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c6c05e820 con 0x7f9c78073b10 2024-04-25T19:20:04.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:04.403+0000 7f9c7e7b2640 1 -- 172.21.15.76:0/1075031028 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9c78071fd0 con 0x7f9c78073b10 2024-04-25T19:20:05.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:05.084+0000 7f9c74ff9640 1 -- 172.21.15.76:0/1075031028 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) ==== 169+0+0 (secure 0 0 0) 0x7f9c6c062b20 con 0x7f9c78073b10 2024-04-25T19:20:05.085 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' removed 2024-04-25T19:20:05.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:05.088+0000 7f9c7e7b2640 1 -- 172.21.15.76:0/1075031028 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9c640744e0 msgr2=0x7f9c640769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:05.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:05.088+0000 7f9c7e7b2640 1 --2- 172.21.15.76:0/1075031028 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9c640744e0 0x7f9c640769f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9c68009aa0 tx=0x7f9c68009190 comp rx=0 tx=0).stop 2024-04-25T19:20:05.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:05.088+0000 7f9c7e7b2640 1 -- 172.21.15.76:0/1075031028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c78073b10 msgr2=0x7f9c7815e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:20:05.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:05.088+0000 7f9c7e7b2640 1 --2- 172.21.15.76:0/1075031028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c78073b10 0x7f9c7815e240 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c00c3f0 tx=0x7f9c6c00c8b0 comp rx=0 tx=0).stop 2024-04-25T19:20:05.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:05.090+0000 7f9c7e7b2640 1 -- 172.21.15.76:0/1075031028 shutdown_connections 2024-04-25T19:20:05.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:05.090+0000 7f9c7e7b2640 1 --2- 172.21.15.76:0/1075031028 >> [v2:172.21.15.105:6832/2585775531,v1:172.21.15.105:6833/2585775531] conn(0x7f9c640744e0 0x7f9c640769f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:05.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:05.090+0000 7f9c7e7b2640 1 --2- 172.21.15.76:0/1075031028 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9c781f3e60 0x7f9c7815e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:05.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:05.090+0000 7f9c7e7b2640 1 --2- 172.21.15.76:0/1075031028 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9c78073b10 0x7f9c7815e240 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:05.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:05.090+0000 7f9c7e7b2640 1 --2- 172.21.15.76:0/1075031028 >> [v2:172.21.15.105:3301/0,v1:172.21.15.105:6790/0] conn(0x7f9c780721f0 0x7f9c7814a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:20:05.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:05.090+0000 7f9c7e7b2640 1 -- 172.21.15.76:0/1075031028 >> 172.21.15.76:0/1075031028 conn(0x7f9c7806fb50 msgr2=0x7f9c7814f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:20:05.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:05.091+0000 7f9c7e7b2640 1 -- 172.21.15.76:0/1075031028 shutdown_connections 2024-04-25T19:20:05.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-04-25T19:20:05.092+0000 7f9c7e7b2640 1 -- 172.21.15.76:0/1075031028 wait complete. 2024-04-25T19:20:05.109 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T19:20:05.109 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T19:20:05.109 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:20:05.122 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T19:20:05.122 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T19:20:05.122 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:20:05.171 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T19:20:05.171 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-04-25T19:20:05.171 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:20:05.224 INFO:tasks.ceph.mds.e:Stopped 2024-04-25T19:20:05.225 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T19:20:05.225 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:20:05.237 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T19:20:05.237 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T19:20:05.237 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:20:05.290 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T19:20:05.290 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-04-25T19:20:05.290 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:20:05.343 INFO:tasks.ceph.mds.f:Stopped 2024-04-25T19:20:05.343 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T19:20:05.343 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T19:20:05.343 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:20:05.379 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T19:20:05.379 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T19:20:05.379 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:20:05.447 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T19:20:05.447 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T19:20:05.447 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:20:05.521 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T19:20:05.521 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T19:20:05.521 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:20:05.588 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T19:20:05.588 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T19:20:05.588 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:20:05.626 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T19:20:05.626 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T19:20:05.626 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:20:05.707 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T19:20:05.707 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T19:20:05.707 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:20:05.778 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T19:20:05.779 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T19:20:05.779 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:20:05.859 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T19:20:05.859 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T19:20:05.860 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T19:20:05.860 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:20:05.884 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T19:20:05.885 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T19:20:05.885 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:20:05.927 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T19:20:05.928 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T19:20:05.928 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T19:20:05.928 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:20:05.939 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T19:20:05.939 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T19:20:05.939 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:20:05.980 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T19:20:05.980 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T19:20:05.980 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T19:20:06.032 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T19:20:06.032 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T19:20:06.032 DEBUG:teuthology.orchestra.run.smithi076:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_DOWN\)' | head -n 1 2024-04-25T19:20:06.087 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi076.front.sepia.ceph.com 2024-04-25T19:20:06.087 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T19:20:06.400 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi076.front.sepia.ceph.com 2024-04-25T19:20:06.400 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T19:20:06.490 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi076.front.sepia.ceph.com 2024-04-25T19:20:06.490 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T19:20:06.547 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi076.front.sepia.ceph.com 2024-04-25T19:20:06.547 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T19:20:06.598 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi105.front.sepia.ceph.com 2024-04-25T19:20:06.598 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T19:20:06.933 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi105.front.sepia.ceph.com 2024-04-25T19:20:06.933 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T19:20:07.036 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi105.front.sepia.ceph.com 2024-04-25T19:20:07.037 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T19:20:07.094 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi105.front.sepia.ceph.com 2024-04-25T19:20:07.094 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T19:20:07.169 INFO:tasks.ceph:Archiving mon data... 2024-04-25T19:20:07.170 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673301/data/mon.a.tgz 2024-04-25T19:20:07.171 DEBUG:teuthology.orchestra.run.smithi076:> mktemp 2024-04-25T19:20:07.187 INFO:teuthology.orchestra.run.smithi076.stdout:/tmp/tmp.m87duT0wZe 2024-04-25T19:20:07.188 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.m87duT0wZe 2024-04-25T19:20:07.325 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0666 /tmp/tmp.m87duT0wZe 2024-04-25T19:20:07.407 DEBUG:teuthology.orchestra.remote:smithi076:/tmp/tmp.m87duT0wZe is 355KB 2024-04-25T19:20:07.502 DEBUG:teuthology.orchestra.run.smithi076:> rm -fr /tmp/tmp.m87duT0wZe 2024-04-25T19:20:07.517 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673301/data/mon.b.tgz 2024-04-25T19:20:07.517 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2024-04-25T19:20:07.534 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.EfOmx98jMP 2024-04-25T19:20:07.534 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.EfOmx98jMP 2024-04-25T19:20:07.706 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.EfOmx98jMP 2024-04-25T19:20:07.789 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.EfOmx98jMP is 366KB 2024-04-25T19:20:07.885 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.EfOmx98jMP 2024-04-25T19:20:07.900 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673301/data/mon.c.tgz 2024-04-25T19:20:07.900 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2024-04-25T19:20:07.954 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.NV3l3qsQFb 2024-04-25T19:20:07.954 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.NV3l3qsQFb 2024-04-25T19:20:08.081 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.NV3l3qsQFb 2024-04-25T19:20:08.161 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.NV3l3qsQFb is 364KB 2024-04-25T19:20:08.256 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.NV3l3qsQFb 2024-04-25T19:20:08.270 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T19:20:08.270 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T19:20:08.273 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T19:20:08.349 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T19:20:08.387 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T19:20:08.388 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T19:20:08.389 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673301/remote/smithi076/crash 2024-04-25T19:20:08.390 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T19:20:08.414 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673301/remote/smithi105/crash 2024-04-25T19:20:08.417 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T19:20:08.451 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T19:20:08.451 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:20:08.460 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T19:20:08.549 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T19:20:08.572 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T19:20:08.595 INFO:tasks.ceph:Compressing logs... 2024-04-25T19:20:08.595 DEBUG:teuthology.orchestra.run.smithi076:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T19:20:08.615 DEBUG:teuthology.orchestra.run.smithi105:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T19:20:08.635 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42390.log 2024-04-25T19:20:08.635 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44153.log 2024-04-25T19:20:08.673 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34929.log 2024-04-25T19:20:08.673 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T19:20:08.673 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34889.log 2024-04-25T19:20:08.673 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34929.log.gz 2024-04-25T19:20:08.673 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34849.log 2024-04-25T19:20:08.674 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34889.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2024-04-25T19:20:08.674 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34849.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T19:20:08.674 INFO:teuthology.orchestra.run.smithi105.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34889.log.gz 2024-04-25T19:20:08.674 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mds.f.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34849.log.gz 2024-04-25T19:20:08.674 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T19:20:08.674 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T19:20:08.674 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T19:20:08.674 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T19:20:08.674 INFO:teuthology.orchestra.run.smithi105.stderr: 92.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T19:20:08.674 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T19:20:08.674 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T19:20:08.674 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mgr.x.log: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T19:20:08.674 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34758.log 2024-04-25T19:20:08.674 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T19:20:08.674 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34758.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34758.log.gz 2024-04-25T19:20:08.683 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35009.log 2024-04-25T19:20:08.694 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.6.log: 92.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T19:20:08.694 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34798.log 2024-04-25T19:20:08.694 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.35009.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35009.log.gz 2024-04-25T19:20:08.694 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T19:20:08.694 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34798.log: 91.9% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2024-04-25T19:20:08.694 INFO:teuthology.orchestra.run.smithi105.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.34798.log.gz 2024-04-25T19:20:08.694 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T19:20:08.694 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34718.log 2024-04-25T19:20:08.711 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34969.log 2024-04-25T19:20:08.711 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34718.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34718.log.gz 2024-04-25T19:20:08.715 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47047.log 2024-04-25T19:20:08.715 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42390.log: /var/log/ceph/ceph-client.admin.44153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39209.log 2024-04-25T19:20:08.715 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42390.log.gz 2024-04-25T19:20:08.715 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47047.log: gzip -5 89.7% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.44153.log.gz 2024-04-25T19:20:08.715 INFO:teuthology.orchestra.run.smithi076.stderr: /var/log/ceph/ceph-client.admin.39711.log 2024-04-25T19:20:08.715 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39209.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47047.log.gz 2024-04-25T19:20:08.715 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43727.log 2024-04-25T19:20:08.716 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39209.log.gz 2024-04-25T19:20:08.716 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47612.log 2024-04-25T19:20:08.716 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41134.log 2024-04-25T19:20:08.716 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39711.log.gz 2024-04-25T19:20:08.716 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43601.log 2024-04-25T19:20:08.716 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43727.log.gz 2024-04-25T19:20:08.716 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41134.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47612.log.gz 2024-04-25T19:20:08.716 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39712.log 2024-04-25T19:20:08.716 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43601.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41134.log.gz 2024-04-25T19:20:08.716 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46136.log 2024-04-25T19:20:08.716 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45278.log 2024-04-25T19:20:08.716 INFO:teuthology.orchestra.run.smithi076.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.43601.log.gz 2024-04-25T19:20:08.716 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39712.log.gz 2024-04-25T19:20:08.716 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46020.log 2024-04-25T19:20:08.716 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46136.log.gz 2024-04-25T19:20:08.717 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41386.log 2024-04-25T19:20:08.717 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45278.log: /var/log/ceph/ceph-client.admin.46020.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28388.log 2024-04-25T19:20:08.717 INFO:teuthology.orchestra.run.smithi076.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.45278.log.gz 2024-04-25T19:20:08.717 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41386.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46020.log.gz 2024-04-25T19:20:08.717 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38949.log 2024-04-25T19:20:08.717 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41386.log.gz 2024-04-25T19:20:08.717 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph.tmp-client.admin.28388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39709.log 2024-04-25T19:20:08.717 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28388.log.gz 2024-04-25T19:20:08.717 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40840.log 2024-04-25T19:20:08.717 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39709.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38949.log.gz 2024-04-25T19:20:08.717 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40711.log 2024-04-25T19:20:08.717 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40840.log: 87.7%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.39709.log.gz --verbose 2024-04-25T19:20:08.717 INFO:teuthology.orchestra.run.smithi076.stderr: -- /var/log/ceph/ceph-client.admin.47257.log 2024-04-25T19:20:08.717 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40711.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.40840.log.gz 2024-04-25T19:20:08.717 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44321.log 2024-04-25T19:20:08.718 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47257.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40711.log.gz 2024-04-25T19:20:08.718 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40022.log 2024-04-25T19:20:08.718 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47257.log.gz 2024-04-25T19:20:08.718 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38811.log 2024-04-25T19:20:08.718 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45708.log 2024-04-25T19:20:08.718 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44321.log.gz 2024-04-25T19:20:08.718 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41008.log 2024-04-25T19:20:08.718 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40022.log.gz 2024-04-25T19:20:08.718 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45708.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38811.log.gz 2024-04-25T19:20:08.718 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39705.log 2024-04-25T19:20:08.718 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43161.log 2024-04-25T19:20:08.718 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45708.log.gz 2024-04-25T19:20:08.718 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39705.log: 89.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.41008.log.gz -5 2024-04-25T19:20:08.718 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43348.log 2024-04-25T19:20:08.718 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39302.log 2024-04-25T19:20:08.719 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39705.log.gz 2024-04-25T19:20:08.719 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43161.log.gz 2024-04-25T19:20:08.719 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40924.log 2024-04-25T19:20:08.719 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39302.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43348.log.gz -5 2024-04-25T19:20:08.719 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46488.log 2024-04-25T19:20:08.719 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39302.log.gz 2024-04-25T19:20:08.719 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44662.log 2024-04-25T19:20:08.719 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40924.log.gz 2024-04-25T19:20:08.719 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46753.log 2024-04-25T19:20:08.719 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44662.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46488.log.gz 2024-04-25T19:20:08.719 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T19:20:08.719 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45073.log 2024-04-25T19:20:08.719 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46753.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38507.log 2024-04-25T19:20:08.720 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46753.log.gz 2024-04-25T19:20:08.720 INFO:teuthology.orchestra.run.smithi076.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.44662.log.gz 2024-04-25T19:20:08.720 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40882.log 2024-04-25T19:20:08.720 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41302.log 2024-04-25T19:20:08.720 INFO:teuthology.orchestra.run.smithi076.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.45073.log.gz 2024-04-25T19:20:08.720 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40882.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38507.log.gz 2024-04-25T19:20:08.720 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39701.log 2024-04-25T19:20:08.720 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40882.log.gz 2024-04-25T19:20:08.720 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42558.log 2024-04-25T19:20:08.720 INFO:teuthology.orchestra.run.smithi076.stderr: 85.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T19:20:08.720 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42247.log 2024-04-25T19:20:08.720 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41302.log.gz 2024-04-25T19:20:08.720 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39701.log.gz 2024-04-25T19:20:08.720 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47425.log 2024-04-25T19:20:08.720 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.0.42247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46669.log 2024-04-25T19:20:08.721 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42558.log.gz 2024-04-25T19:20:08.721 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47425.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T19:20:08.721 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44575.log 2024-04-25T19:20:08.721 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47425.log.gz 2024-04-25T19:20:08.721 INFO:teuthology.orchestra.run.smithi076.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46669.log.gz 2024-04-25T19:20:08.721 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39530.log 2024-04-25T19:20:08.721 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42684.log 2024-04-25T19:20:08.721 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39530.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44575.log.gz 2024-04-25T19:20:08.721 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42907.log 2024-04-25T19:20:08.721 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43685.log 2024-04-25T19:20:08.721 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42684.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39530.log.gz 2024-04-25T19:20:08.721 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42907.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42684.log.gz 2024-04-25T19:20:08.721 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43791.log 2024-04-25T19:20:08.721 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42907.log.gz 2024-04-25T19:20:08.722 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40244.log 2024-04-25T19:20:08.722 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43685.log: /var/log/ceph/ceph-client.admin.43791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40015.log 2024-04-25T19:20:08.722 INFO:teuthology.orchestra.run.smithi076.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.43685.log.gz 2024-04-25T19:20:08.722 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40244.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43791.log.gz 2024-04-25T19:20:08.722 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43034.log 2024-04-25T19:20:08.722 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40244.log.gz 2024-04-25T19:20:08.722 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45978.log 2024-04-25T19:20:08.722 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T19:20:08.722 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43034.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40015.log.gz 2024-04-25T19:20:08.722 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43474.log 2024-04-25T19:20:08.722 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43034.log.gz 2024-04-25T19:20:08.722 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45834.log 2024-04-25T19:20:08.722 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45978.log.gz 2024-04-25T19:20:08.722 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47131.log 2024-04-25T19:20:08.723 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46627.log 2024-04-25T19:20:08.723 INFO:teuthology.orchestra.run.smithi076.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.43474.log.gz 2024-04-25T19:20:08.723 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46879.log 2024-04-25T19:20:08.723 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45834.log.gz 2024-04-25T19:20:08.723 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T19:20:08.723 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45235.log 2024-04-25T19:20:08.723 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47131.log.gz 2024-04-25T19:20:08.723 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46879.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46627.log.gzgzip 2024-04-25T19:20:08.723 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43245.log 2024-04-25T19:20:08.723 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42950.log 2024-04-25T19:20:08.723 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45235.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46879.log.gz 2024-04-25T19:20:08.723 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41494.log 2024-04-25T19:20:08.723 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45235.log.gz 2024-04-25T19:20:08.723 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40669.log 2024-04-25T19:20:08.724 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43245.log.gz 2024-04-25T19:20:08.724 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41494.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42950.log.gz 2024-04-25T19:20:08.724 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46094.log 2024-04-25T19:20:08.724 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44766.log 2024-04-25T19:20:08.724 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40669.log: /var/log/ceph/ceph-client.admin.46094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38607.log 2024-04-25T19:20:08.724 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40669.log.gz 91.1% 2024-04-25T19:20:08.724 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41494.log.gz 2024-04-25T19:20:08.724 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44766.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46094.log.gz 2024-04-25T19:20:08.724 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T19:20:08.724 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40188.log 2024-04-25T19:20:08.724 INFO:teuthology.orchestra.run.smithi076.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44766.log.gz 2024-04-25T19:20:08.724 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-osd.3.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38607.log.gz 2024-04-25T19:20:08.724 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47299.log 2024-04-25T19:20:08.724 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43390.log 2024-04-25T19:20:08.724 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40188.log: /var/log/ceph/ceph-client.admin.47299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44195.log 2024-04-25T19:20:08.725 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40188.log.gz 2024-04-25T19:20:08.725 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47299.log.gz 2024-04-25T19:20:08.725 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45404.log 2024-04-25T19:20:08.725 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40627.log 2024-04-25T19:20:08.725 INFO:teuthology.orchestra.run.smithi076.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.43390.log.gz 2024-04-25T19:20:08.725 INFO:teuthology.orchestra.run.smithi076.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44195.log.gz 2024-04-25T19:20:08.725 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46711.log 2024-04-25T19:20:08.725 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39707.log 2024-04-25T19:20:08.725 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45404.log.gz 2024-04-25T19:20:08.725 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40627.log: /var/log/ceph/ceph-client.admin.46711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45498.log 2024-04-25T19:20:08.725 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40627.log.gz 2024-04-25T19:20:08.725 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39707.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46711.log.gz 2024-04-25T19:20:08.725 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43643.log 2024-04-25T19:20:08.725 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45498.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39707.log.gz 2024-04-25T19:20:08.725 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40756.log 2024-04-25T19:20:08.725 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43643.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T19:20:08.726 INFO:teuthology.orchestra.run.smithi076.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.45498.log.gz 2024-04-25T19:20:08.726 INFO:teuthology.orchestra.run.smithi076.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.43643.log.gz 2024-04-25T19:20:08.726 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46404.log 2024-04-25T19:20:08.726 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mds.c.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40756.log.gz 2024-04-25T19:20:08.726 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41092.log 2024-04-25T19:20:08.726 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39167.log 2024-04-25T19:20:08.726 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41092.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46404.log.gz 2024-04-25T19:20:08.726 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40431.log 2024-04-25T19:20:08.726 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39167.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41092.log.gz 2024-04-25T19:20:08.726 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46178.log 2024-04-25T19:20:08.726 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39167.log.gz 2024-04-25T19:20:08.726 INFO:teuthology.orchestra.run.smithi076.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T19:20:08.726 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40431.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T19:20:08.726 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46178.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T19:20:08.727 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40431.log.gz 2024-04-25T19:20:08.727 INFO:teuthology.orchestra.run.smithi076.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46178.log.gz 2024-04-25T19:20:08.727 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41560.log 2024-04-25T19:20:08.727 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45589.log 2024-04-25T19:20:08.727 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42992.log 2024-04-25T19:20:08.727 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47215.log 2024-04-25T19:20:08.727 INFO:teuthology.orchestra.run.smithi076.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.41560.log.gz 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45589.log.gz 2024-04-25T19:20:08.727 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-04-25T19:20:08.727 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46837.log 2024-04-25T19:20:08.727 INFO:teuthology.orchestra.run.smithi076.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.42992.log.gz/var/log/ceph/ceph-client.admin.47215.log: 2024-04-25T19:20:08.727 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T19:20:08.727 INFO:teuthology.orchestra.run.smithi076.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47215.log.gz 2024-04-25T19:20:08.727 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42432.log 2024-04-25T19:20:08.727 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1%/var/log/ceph/ceph-osd.0.log: -- replaced with /var/log/ceph/ceph-client.admin.46837.log.gz 2024-04-25T19:20:08.727 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44279.log 2024-04-25T19:20:08.727 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38741.log 2024-04-25T19:20:08.727 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42432.log: /var/log/ceph/ceph-client.admin.44279.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42432.log.gz 2024-04-25T19:20:08.728 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40966.log 2024-04-25T19:20:08.728 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44279.log.gz 2024-04-25T19:20:08.728 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38741.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38741.log.gz 2024-04-25T19:20:08.728 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42826.log 2024-04-25T19:20:08.728 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41344.log 2024-04-25T19:20:08.728 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40966.log: /var/log/ceph/ceph-client.admin.42826.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40966.log.gz 2024-04-25T19:20:08.728 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% 93.2% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T19:20:08.728 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42826.log.gz 2024-04-25T19:20:08.728 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46362.log 2024-04-25T19:20:08.728 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45362.log 2024-04-25T19:20:08.728 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41344.log: /var/log/ceph/ceph-client.admin.46362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42474.log 2024-04-25T19:20:08.728 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45362.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41344.log.gz 2024-04-25T19:20:08.728 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46362.log.gz 2024-04-25T19:20:08.728 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38653.log 2024-04-25T19:20:08.729 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45362.log.gz 2024-04-25T19:20:08.729 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40355.log 2024-04-25T19:20:08.729 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44860.log 2024-04-25T19:20:08.729 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42474.log.gz 2024-04-25T19:20:08.729 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38653.log: /var/log/ceph/ceph-client.admin.40355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43517.log 2024-04-25T19:20:08.729 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38653.log.gz 2024-04-25T19:20:08.729 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44860.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40355.log.gz 2024-04-25T19:20:08.729 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47341.log 2024-04-25T19:20:08.729 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44860.log.gz 2024-04-25T19:20:08.729 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45547.log 2024-04-25T19:20:08.729 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43517.log: /var/log/ceph/ceph-client.admin.47341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40024.log 2024-04-25T19:20:08.729 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45547.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43517.log.gz 2024-04-25T19:20:08.729 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47341.log.gz 2024-04-25T19:20:08.729 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 90.0% -- replaced with /var/log/ceph/ceph-client.admin.45547.log.gz 2024-04-25T19:20:08.730 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45193.log 2024-04-25T19:20:08.730 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47860.log 2024-04-25T19:20:08.730 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45193.log: /var/log/ceph/ceph-client.admin.40024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47173.log 2024-04-25T19:20:08.730 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45193.log.gz/var/log/ceph/ceph-client.admin.47860.log: 2024-04-25T19:20:08.730 INFO:teuthology.orchestra.run.smithi076.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40024.log.gz 2024-04-25T19:20:08.730 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47860.log.gz 2024-04-25T19:20:08.730 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44724.log 2024-04-25T19:20:08.730 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39404.log 2024-04-25T19:20:08.730 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47173.log: /var/log/ceph/ceph-client.admin.44724.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T19:20:08.730 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39404.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47173.log.gz 2024-04-25T19:20:08.730 INFO:teuthology.orchestra.run.smithi076.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44724.log.gz 2024-04-25T19:20:08.730 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39404.log.gz 2024-04-25T19:20:08.730 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47570.log 2024-04-25T19:20:08.730 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43875.log 2024-04-25T19:20:08.731 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.47570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38905.log 2024-04-25T19:20:08.731 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43875.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47570.log.gz 2024-04-25T19:20:08.731 INFO:teuthology.orchestra.run.smithi076.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.43875.log.gz 2024-04-25T19:20:08.731 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47780.log 2024-04-25T19:20:08.731 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47467.log 2024-04-25T19:20:08.731 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47780.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38905.log.gz 2024-04-25T19:20:08.731 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47530.log 2024-04-25T19:20:08.731 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47780.log.gz 2024-04-25T19:20:08.731 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47467.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47467.log.gz 2024-04-25T19:20:08.731 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42726.log 2024-04-25T19:20:08.731 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44406.log 2024-04-25T19:20:08.731 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42726.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47530.log.gz 2024-04-25T19:20:08.731 INFO:teuthology.orchestra.run.smithi076.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42726.log.gz 2024-04-25T19:20:08.731 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41626.log 2024-04-25T19:20:08.731 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44406.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44406.log.gz 2024-04-25T19:20:08.732 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45031.log 2024-04-25T19:20:08.732 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45918.log 2024-04-25T19:20:08.732 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45031.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.45031.log.gz 2024-04-25T19:20:08.732 INFO:teuthology.orchestra.run.smithi076.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.41626.log.gz 2024-04-25T19:20:08.732 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40585.log 2024-04-25T19:20:08.732 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44364.log 2024-04-25T19:20:08.732 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40585.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45918.log.gz 2024-04-25T19:20:08.732 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40585.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43076.log 2024-04-25T19:20:08.732 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-04-25T19:20:08.732 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44364.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44364.log.gz 2024-04-25T19:20:08.732 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42600.log 2024-04-25T19:20:08.732 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39710.log 2024-04-25T19:20:08.732 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42600.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43076.log.gz 2024-04-25T19:20:08.732 INFO:teuthology.orchestra.run.smithi076.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42600.log.gz 2024-04-25T19:20:08.732 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40364.log 2024-04-25T19:20:08.732 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34416.log 2024-04-25T19:20:08.733 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40364.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39710.log.gz 2024-04-25T19:20:08.733 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40364.log.gz 2024-04-25T19:20:08.733 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44490.log 2024-04-25T19:20:08.733 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.34416.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34416.log.gz 2024-04-25T19:20:08.733 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47383.log 2024-04-25T19:20:08.733 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41176.log 2024-04-25T19:20:08.733 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47383.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44490.log.gz 2024-04-25T19:20:08.733 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47383.log.gz 2024-04-25T19:20:08.733 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41050.log 2024-04-25T19:20:08.733 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41176.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41176.log.gz 2024-04-25T19:20:08.733 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40025.log 2024-04-25T19:20:08.733 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44532.log 2024-04-25T19:20:08.733 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40025.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41050.log.gz 2024-04-25T19:20:08.733 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40025.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46446.log 2024-04-25T19:20:08.734 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-04-25T19:20:08.734 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44532.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44532.log.gz 2024-04-25T19:20:08.734 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40798.log 2024-04-25T19:20:08.734 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43833.log 2024-04-25T19:20:08.734 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40798.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46446.log.gz 2024-04-25T19:20:08.734 INFO:teuthology.orchestra.run.smithi076.stderr: 89.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.40798.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.39125.log 2024-04-25T19:20:08.734 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-04-25T19:20:08.734 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43833.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43833.log.gz 2024-04-25T19:20:08.734 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42768.log 2024-04-25T19:20:08.734 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46262.log 2024-04-25T19:20:08.734 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42768.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39125.log.gz 2024-04-25T19:20:08.734 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42768.log.gzgzip 2024-04-25T19:20:08.734 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.38695.log 2024-04-25T19:20:08.734 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46262.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46262.log.gz 2024-04-25T19:20:08.735 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45750.log 2024-04-25T19:20:08.735 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40023.log 2024-04-25T19:20:08.735 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45750.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38695.log.gz 2024-04-25T19:20:08.735 INFO:teuthology.orchestra.run.smithi076.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45750.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46921.log 2024-04-25T19:20:08.735 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-04-25T19:20:08.735 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40023.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40023.log.gz 2024-04-25T19:20:08.735 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39706.log 2024-04-25T19:20:08.735 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40363.log 2024-04-25T19:20:08.735 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39706.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46921.log.gz 2024-04-25T19:20:08.735 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39706.log.gz 2024-04-25T19:20:08.735 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43917.log 2024-04-25T19:20:08.735 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40363.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40363.log.gz 2024-04-25T19:20:08.735 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34680.log 2024-04-25T19:20:08.735 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41260.log 2024-04-25T19:20:08.735 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.34680.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43917.log.gz 2024-04-25T19:20:08.736 INFO:teuthology.orchestra.run.smithi076.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.34680.log.gz 2024-04-25T19:20:08.736 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46795.log 2024-04-25T19:20:08.736 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41260.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41260.log.gz 2024-04-25T19:20:08.736 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46963.log 2024-04-25T19:20:08.736 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45792.log 2024-04-25T19:20:08.736 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46963.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46795.log.gz 2024-04-25T19:20:08.736 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39041.log 2024-04-25T19:20:08.736 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46963.log.gz 2024-04-25T19:20:08.736 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45792.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45792.log.gz 2024-04-25T19:20:08.736 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39083.log 2024-04-25T19:20:08.736 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39041.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2024-04-25T19:20:08.736 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39083.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39041.log.gz 2024-04-25T19:20:08.736 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39083.log.gz 2024-04-25T19:20:08.736 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39260.log 2024-04-25T19:20:08.736 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42642.log 2024-04-25T19:20:08.737 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39260.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39260.log.gz 2024-04-25T19:20:08.737 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44618.log 2024-04-25T19:20:08.737 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42642.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42642.log.gz 2024-04-25T19:20:08.737 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47005.log 2024-04-25T19:20:08.737 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44618.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44618.log.gz 2024-04-25T19:20:08.737 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45876.log 2024-04-25T19:20:08.737 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47005.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47005.log.gz 2024-04-25T19:20:08.737 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43119.log 2024-04-25T19:20:08.737 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45876.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45876.log.gz 2024-04-25T19:20:08.737 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40365.log 2024-04-25T19:20:08.737 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43119.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43119.log.gz 2024-04-25T19:20:08.737 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41428.log 2024-04-25T19:20:08.737 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40365.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40365.log.gz 2024-04-25T19:20:08.737 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44945.log 2024-04-25T19:20:08.738 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41428.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.41428.log.gz 2024-04-25T19:20:08.739 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.34969.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34969.log.gz 2024-04-25T19:20:08.739 INFO:teuthology.orchestra.run.smithi105.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T19:20:08.739 INFO:teuthology.orchestra.run.smithi105.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T19:20:08.747 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44448.log 2024-04-25T19:20:08.747 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44945.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44945.log.gz 2024-04-25T19:20:08.747 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40026.log 2024-04-25T19:20:08.748 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44448.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44448.log.gz 2024-04-25T19:20:08.748 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43203.log 2024-04-25T19:20:08.748 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40026.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40026.log.gz 2024-04-25T19:20:08.748 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47738.log 2024-04-25T19:20:08.748 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43203.log: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.43203.log.gz 2024-04-25T19:20:08.748 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40016.log 2024-04-25T19:20:08.748 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47738.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47738.log.gz 2024-04-25T19:20:08.748 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43287.log 2024-04-25T19:20:08.748 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40016.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40016.log.gz 2024-04-25T19:20:08.748 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39488.log 2024-04-25T19:20:08.748 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43287.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43287.log.gz 2024-04-25T19:20:08.748 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44988.log 2024-04-25T19:20:08.748 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39488.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39488.log.gz 2024-04-25T19:20:08.748 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41218.log 2024-04-25T19:20:08.748 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44988.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.44988.log.gz 2024-04-25T19:20:08.749 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38999.log 2024-04-25T19:20:08.749 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41218.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41218.log.gz 2024-04-25T19:20:08.749 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47696.log 2024-04-25T19:20:08.749 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38999.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38999.log.gz 2024-04-25T19:20:08.750 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41794.log 2024-04-25T19:20:08.751 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47696.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47696.log.gz 2024-04-25T19:20:08.751 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44237.log 2024-04-25T19:20:08.751 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41794.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41794.log.gz 2024-04-25T19:20:08.753 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43432.log 2024-04-25T19:20:08.754 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44237.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44237.log.gz 2024-04-25T19:20:08.754 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42516.log 2024-04-25T19:20:08.754 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43432.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.43432.log.gz 2024-04-25T19:20:08.755 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T19:20:08.755 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42516.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42516.log.gz 2024-04-25T19:20:08.756 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43959.log 2024-04-25T19:20:08.766 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45115.log 2024-04-25T19:20:08.766 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43959.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43959.log.gz 2024-04-25T19:20:08.775 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47089.log 2024-04-25T19:20:08.775 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45115.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45115.log.gz 2024-04-25T19:20:08.785 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46220.log 2024-04-25T19:20:08.785 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47089.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47089.log.gz 2024-04-25T19:20:08.786 INFO:teuthology.orchestra.run.smithi105.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T19:20:08.790 INFO:teuthology.orchestra.run.smithi076.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T19:20:08.791 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39344.log 2024-04-25T19:20:08.791 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46220.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46220.log.gz 2024-04-25T19:20:08.792 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46320.log 2024-04-25T19:20:08.793 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39344.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39344.log.gz 2024-04-25T19:20:08.793 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38861.log 2024-04-25T19:20:08.793 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46320.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46320.log.gz 2024-04-25T19:20:08.794 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34783.log 2024-04-25T19:20:08.794 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38861.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38861.log.gz 2024-04-25T19:20:08.795 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47654.log 2024-04-25T19:20:08.795 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.34783.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34783.log.gz 2024-04-25T19:20:08.796 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47902.log 2024-04-25T19:20:08.796 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47654.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47654.log.gz 2024-04-25T19:20:08.797 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41692.log 2024-04-25T19:20:08.797 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47902.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47902.log.gz 2024-04-25T19:20:08.798 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45320.log 2024-04-25T19:20:08.799 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41692.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.41692.log.gz 2024-04-25T19:20:08.799 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38392.log 2024-04-25T19:20:08.800 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45320.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45320.log.gz 2024-04-25T19:20:08.800 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39446.log 2024-04-25T19:20:08.801 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38392.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38392.log.gz 2024-04-25T19:20:08.801 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40284.log 2024-04-25T19:20:08.802 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39446.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39446.log.gz 2024-04-25T19:20:08.802 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43559.log 2024-04-25T19:20:08.803 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40284.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40284.log.gz 2024-04-25T19:20:08.803 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40283.log 2024-04-25T19:20:08.804 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43559.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43559.log.gz 2024-04-25T19:20:08.804 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45448.log 2024-04-25T19:20:08.805 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40283.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40283.log.gz 2024-04-25T19:20:08.806 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45448.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45448.log.gz 2024-04-25T19:20:08.821 INFO:teuthology.orchestra.run.smithi105.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T19:20:08.879 INFO:teuthology.orchestra.run.smithi076.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2024-04-25T19:20:09.000 INFO:teuthology.orchestra.run.smithi076.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T19:20:11.008 INFO:teuthology.orchestra.run.smithi076.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T19:20:11.512 INFO:teuthology.orchestra.run.smithi105.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T19:20:11.903 INFO:teuthology.orchestra.run.smithi076.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T19:20:12.261 INFO:teuthology.orchestra.run.smithi105.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T19:20:12.454 INFO:teuthology.orchestra.run.smithi076.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T19:20:12.576 INFO:teuthology.orchestra.run.smithi105.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T19:20:12.705 INFO:teuthology.orchestra.run.smithi105.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T19:20:12.707 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-04-25T19:20:12.708 INFO:teuthology.orchestra.run.smithi105.stderr:real 0m4.059s 2024-04-25T19:20:12.708 INFO:teuthology.orchestra.run.smithi105.stderr:user 0m14.286s 2024-04-25T19:20:12.708 INFO:teuthology.orchestra.run.smithi105.stderr:sys 0m0.532s 2024-04-25T19:20:13.225 INFO:teuthology.orchestra.run.smithi076.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T19:20:16.265 INFO:teuthology.orchestra.run.smithi076.stderr: 96.7% -- replaced with /var/log/ceph/ceph-client.0.42247.log.gz 2024-04-25T19:20:16.267 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-04-25T19:20:16.267 INFO:teuthology.orchestra.run.smithi076.stderr:real 0m7.641s 2024-04-25T19:20:16.267 INFO:teuthology.orchestra.run.smithi076.stderr:user 0m21.393s 2024-04-25T19:20:16.267 INFO:teuthology.orchestra.run.smithi076.stderr:sys 0m0.902s 2024-04-25T19:20:16.268 INFO:tasks.ceph:Archiving logs... 2024-04-25T19:20:16.268 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673301/remote/smithi076/log 2024-04-25T19:20:16.269 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T19:20:19.110 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673301/remote/smithi105/log 2024-04-25T19:20:19.111 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T19:20:21.328 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T19:20:21.338 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T19:20:21.338 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T19:20:21.341 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T19:20:21.420 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T19:20:21.430 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T19:20:21.430 DEBUG:teuthology.orchestra.run.smithi076:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T19:20:21.432 DEBUG:teuthology.orchestra.run.smithi105:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T19:20:21.445 INFO:teuthology.orchestra.run.smithi076.stderr:bash: line 1: ntpq: command not found 2024-04-25T19:20:21.448 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: ntpq: command not found 2024-04-25T19:20:21.450 INFO:teuthology.orchestra.run.smithi076.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T19:20:21.450 INFO:teuthology.orchestra.run.smithi076.stdout:=============================================================================== 2024-04-25T19:20:21.450 INFO:teuthology.orchestra.run.smithi076.stdout:^* hv01.front.sepia.ceph.com 3 6 377 57 +928us[ +922us] +/- 58ms 2024-04-25T19:20:21.450 INFO:teuthology.orchestra.run.smithi076.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 57 -1496us[-1496us] +/- 81ms 2024-04-25T19:20:21.450 INFO:teuthology.orchestra.run.smithi076.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 +190us[ +183us] +/- 91ms 2024-04-25T19:20:21.450 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T19:20:21.453 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T19:20:21.453 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2024-04-25T19:20:21.453 INFO:teuthology.orchestra.run.smithi105.stdout:^* hv01.front.sepia.ceph.com 3 6 377 54 +914us[ +933us] +/- 58ms 2024-04-25T19:20:21.453 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 55 -1519us[-1500us] +/- 81ms 2024-04-25T19:20:21.453 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 55 +166us[ +185us] +/- 91ms 2024-04-25T19:20:21.453 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T19:20:21.453 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T19:20:21.463 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T19:20:21.464 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T19:20:21.544 DEBUG:teuthology.orchestra.run.smithi076:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T19:20:21.614 DEBUG:teuthology.orchestra.run.smithi105:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T19:20:21.677 DEBUG:teuthology.orchestra.run.smithi076:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T19:20:21.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:20:21.707 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T19:20:21.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:20:21.735 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T19:20:21.745 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T19:20:21.755 INFO:teuthology.task.internal:Duration was 851.029233 seconds 2024-04-25T19:20:21.755 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T19:20:21.765 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T19:20:21.765 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T19:20:21.768 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T19:20:21.804 INFO:teuthology.orchestra.run.smithi076.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T19:20:21.813 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T19:20:22.147 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T19:20:22.147 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi076.front.sepia.ceph.com 2024-04-25T19:20:22.148 DEBUG:teuthology.orchestra.run.smithi076:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T19:20:22.209 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi105.front.sepia.ceph.com 2024-04-25T19:20:22.210 DEBUG:teuthology.orchestra.run.smithi105:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T19:20:22.267 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T19:20:22.267 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T19:20:22.270 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T19:20:22.295 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T19:20:22.295 DEBUG:teuthology.orchestra.run.smithi076:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T19:20:22.336 DEBUG:teuthology.orchestra.run.smithi105:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T19:20:22.438 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T19:20:22.449 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T19:20:22.449 DEBUG:teuthology.orchestra.run.smithi076:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T19:20:22.472 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T19:20:22.495 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T19:20:22.506 DEBUG:teuthology.orchestra.run.smithi076:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T19:20:22.515 DEBUG:teuthology.orchestra.run.smithi105:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T19:20:22.535 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = core 2024-04-25T19:20:22.558 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = core 2024-04-25T19:20:22.583 DEBUG:teuthology.orchestra.run.smithi076:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T19:20:22.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:20:22.598 DEBUG:teuthology.orchestra.run.smithi105:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T19:20:22.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:20:22.637 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T19:20:22.647 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T19:20:22.648 DEBUG:teuthology.misc:Transferring archived files from smithi076:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673301/remote/smithi076 2024-04-25T19:20:22.648 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T19:20:22.681 DEBUG:teuthology.misc:Transferring archived files from smithi105:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673301/remote/smithi105 2024-04-25T19:20:22.681 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T19:20:22.712 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T19:20:22.712 DEBUG:teuthology.orchestra.run.smithi076:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T19:20:22.717 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T19:20:22.762 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T19:20:22.774 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T19:20:22.775 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T19:20:22.785 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T19:20:22.785 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T19:20:22.787 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T19:20:22.799 INFO:teuthology.orchestra.run.smithi076.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 19:20 /home/ubuntu/cephtest 2024-04-25T19:20:22.816 INFO:teuthology.orchestra.run.smithi105.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 19:20 /home/ubuntu/cephtest 2024-04-25T19:20:22.817 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T19:20:22.827 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T19:20:22.852 INFO:teuthology.run:Summary data: description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mon 2-workunit/suites/iozone}} duration: 851.029233455658 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-04-25T19:20:22.852 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T19:20:22.921 INFO:teuthology.run:pass